GUIdebook: Graphical User Interface galleryHome > Articles > “LisaLearning”
GUIsTimelinesScreenshotsIconsSoundsSplashesApplicationsAdsVideosArticlesBooksTutorialsExtras
Go backArticlesLisaLearning

Reprinted from IEEE Computer, November 1986, pp. 35-49.

Six people learned to use Lisa on their own. Their problems with the interface can teach designers how to make their own systems easier to learn.

People who do not already know how to use computing systems often have a difficult time learning how. Over the past six years, research at the Watson Research Center has focused on these learning problems and on finding solutions to them in better interfaces and training tools.1-5 We have studied a variety of commercial systems and prototypes, and a variety of training approaches. Our goal has been to develop a thorough understanding of how new users interact with user interfaces – an understanding that can inform the design of future technology.

Much of this work has focused on office workers with no prior experience with computers. We also focused on “traditional” systems: systems with no on-line tutoring, with step-keys or contextual cursoring, with character-box displays, or with no graphics (and therefore no text/graphics integration). There are good reasons for focusing research in this way: Helping clerical users get started on traditional systems is a pressing problem in the computer industry.

But there are other user groups and other systems. Professionals, the people for whom the clerical users work, have more personal prerogative in determining which system they will use and what they will do with it. The problems of getting started with a system are quite different for professionals. A system difficult to learn will not merely be an obstacle and an inconvenience; it may actually be discarded or returned to the vendor for a refund.

User interfaces for professionals’ workstations have broken rather sharply with what we referred to earlier as traditional user interface designs. The Apple Lisa, first introduced in January 1983 and discontinued in April 1985, is one of the most important examples of this trend. Lisa incorporated an on-line tutor (LisaGuide), a mouse-driven pointer, a bit-mapped display, and a fundamentally graphic interface (icons) for graphics applications (charting, planning, and spreadsheets).

The key ideas in Lisa received an enthusiastic reception in the computer industry. For example, in a review in InfoWorld that named the Lisa as one of the top three personal computer/workstations of 1983, Chin6 specifically identified pointing and selection via the mouse, desktop windows and icons, and commands as outstanding features. She concluded that “Lisa has made giant steps in saving the user both time and frustration,” that it was truly “revolutionary.” These ideas also caused excitement in the research community under the rubric of “direct manipulation.”7-9

This was the context for our study. We wanted to research professionals’ workstations and we wanted to study a less traditional type of interface, but one commercially available and with obvious impact. We thus decided to study professionals learning to use Lisa. We focused on problems learners experienced because these indicated aspects of the design that needed improvement. We were not interested in Lisa per se, but in the class of systems of which Lisa was the best-known example.

Overview of the study

Our study of Lisa was exploratory, so we begin with disclaimers. We studied only six individuals, staff members at a research laboratory. These people may well have been atypical professionals. We did not perform any rigorous benchmark studies, since we did not care whether typing a mailing label was faster on Lisa than on a competitive system. Rather, we tried to focus qualitatively on the design elements of the Lisa interface and training, and on their implications for learners.

Our six learners responded to an on-line request for volunteers that we placed on our local computer network. We excluded volunteers who had substantial programming experience, but all of the six we accepted had some experience with computers. A few had had some programming experience. We asked our learners to volunteer for two three-hour sessions, promising free coffee and the opportunity to learn to use a “new integrated commercial workstation.” Our group included a lawyer, a graphics designer working in patents, a publications librarian, a documentation designer, a researcher in automation design, and a research team manager.

At the beginning of the first session we explained our ground rules: Learners were to work on their own; to imagine a real office situation in which they were learning to use their own new personal workstation (one that nobody else had yet); and were to regard us as inert (albeit keenly interested) observers. We presented them with the system and two Lisa manuals (the owner’s guide and the LisaProject book) and asked them to learn to use the system, and in particular the LisaProject part of the system. We picked LisaProject because it seemed the most professional-oriented of the Lisa applications, as well as alleged by some to be “the best” Lisa application.10,11 We asked them to think aloud as they worked, and prompted them with “What are you thinking?” throughout the sessions as necessary to keep this self-disclosing monolog going.

The two of us kept independent notes on what was said and done during the six hours. Later, we compared and corroborated our observations. The critical and typical episodes and incidents culled from our notes form the basis for our description of what it’s like to learn Lisa. Our plan was to have our participants spend roughly the first three-hour session covering basics and the second covering LisaProject and then performing a simple transfer-of-learning test (we asked them to use LisaProject to plan the schedule for nine phases in the construction of a three-bedroom house). Write-ups in the trade press and estimates in the Lisa documentation assured us that this would be more than enough time.

L1L2L3L4L5L6
Backgrounds
Prior experience with computers (years)164463
Number of different systems ever used233156
Average current computer use (hrs/wk)937281530
Performance
Performance Use of LisaGuide (hrs)2.31.1*3.101.21.1
Use of LisaProject (hrs)02.3004.3.7
Attempted LisaProject taskNYNNYN
Table 1. Learner background and performance summary. *Skipped three topics
In Table 1 we have summarized the backgrounds of our six learners (L1-6) and the amount of time they apportioned to two major aspects of the system.

On average, the five learners able to try the LisaGuide on-line tutorial spent over one and three quarters hours on it, or more than three times the half hour estimated by Apple and by the trade journals and newsletters (L4 made a single error that totally obstructed his progress). Moreover, two of the five were not able to use LisaGuide until their second session (that is, after three hours of previous experience with the system); and one of the five skipped three topic chapters of the tutorial. Only three of the six managed to try LisaProject, and one of them accomplished little. The other two spent an average of 3.3 hours learning the basics and accomplishing the project we had suggested.

In sum, we found a lower level of success than we had been led to expect. Although our learners had had experience (with an average of four systems over an average of three years), they did not all finish the tasks we suggested. Indeed, L4 and L6 – who both spent more than half their work time using computers – failed to get to our LisaProject transfer-of-learning task. But these performance benchmarks tell us little in and of themselves. Hereafter we focus on problems in the learning situation as we observed it. They help us understand why performance was poorer than we expected and how user interface design might benefit from addressing these problems.

Getting started

We saw many of the same problems over and over in prior work with other systems. For example, learners often fear that they will damage the system. As L1 said, “Will I damage this if I turn it off with the disk in? There is a terror of destroying things, so I’m still a bit tentative.” This kind of fear was typical at the start of learning, but in the case quoted L1 had already had four hours of experience with the system.

Reading

Most of our learners did not want to read – as they demonstrated. Nonetheless, most entered the experiment announcing that they were the type of person who carefully read everything before trying to do anything. Two of them brought paper and pencil with which to take notes. For example, L4 apologetically informed us that we would have to watch him read both the owner’s guide and the LisaProject book before he would use the system. Two minutes later he had turned on the Profile (the hard disk drive unit). L6 began the experiment by picking up the Lisa owner’s guide and announcing that his style was to first read everything thoroughly. Indeed, he actually read the manual for less than nine minutes before switching his attention to the system. He next referred to the manual almost two hours later.

L5 was the only participant successful at reading, but for an ironic reason. On her first day she inserted the LisaGuide disk in the wrong orientation and therefore turned to the manual. She spent most of her first session reading and following printed exercises. Curiously, L5 was also one of the two participants to get to the LisaProject transfer test. But her success was only relative. She had many problems coordinating the manual descriptions with events on the display. For example, only 20 minutes into her first session she exclaimed, “I’m wondering as usual why it’s not doing what the manual said it was going to.”

Learners had other specific problems using the manuals. L6 was afraid to turn the system off, thinking he might damage it. He checked the index of the Lisa owner’s guide for help and was directed to page 118. This turned out to be page I18 (that is, capital I-18). In his second session he wanted to review some concepts and could not find any index entries for “cursor” or “pointer,” nor any relevant entries for “mouse” or “moving.”

Misreading

Often when learners spent time and effort reading they became snarled in confused interpretations and cross-references. In his nine minutes of reading, L6 became troubled when he understood section B of the manual to have referred him to section D, only to have section D then refer him to section B. Later, he became tangled in a loop between D-18 and D-20. One page seemed to him to be saying that to create a document or folder (directory) you must make a stationery pad, and the other page seemed to be saying that to make a stationery pad you must create a document or folder.

L4 read a reference to Figure 1 on page E-2 and associated the reference with an unlabeled figure of a calculator icon that appeared on the same page. He never noticed that there was indeed a Figure 1 (labeled as such) on the very next page (E-3). He became momentarily confused, and then irritated, by the fact that page A-4 of the owner’s guide suggested that the user read section A. As L2 put it, “If I were a machine, I’d be in an infinite loop.”

L4 was also confused by references in the owner’s guide to “controller cards” and his discovery of quick-reference cards under the keyboard. This initial confusion went unresolved for almost 50 minutes. At that point he seemed to see the difference and became irritated: “What’s it going to help to know about controller cards? I’m not going to repair it; I want to use it!”

Skipping

When they did read, the learners tended to skip around in the manual. L4 simultaneously read from the owner’s guide and LisaProject, juggling the two in his lap, skipping between chapters and sections. L2 asked to see some of the other manuals, for example LisaList, and then skipped around in an even larger set of books. L5 also juggled manuals. At first she followed the owner’s guide, but failed to load the LisaGuide disk correctly. When the system booted, the desktop displayed. She recognized it from a figure in the LisaProject manual: “Oh! I have the wrong disk in. I thought I had LisaGuide in, but I have LisaProject in.” On the basis of this coincidence, she then skipped to the LisaProject manual for the remainder of the session.

Turning the system on

Problems associated with reading the manual first were generally less troublesome than those associated with coordinating a reading of the manual with use of the system. L6 tried to follow an instruction that said “When you hear a click, hold down the Apple key.” He wondered whether he had heard a click (there were of course a variety of background noises emanating from the system as well as from adjacent rooms of our laboratory). He concluded that he had never heard a click. However, he went ahead and pressed the Apple key combination anyway. By that time he had missed the cue and an alert box (error message) appeared: “Apple key combination not associated with available function.” This series of problems led him to conclude that he could not learn by doing, as he wished. He therefore decided to go through LisaGuide: “I was hoping to skip the LisaGuide, but I guess I need to go through it. I’m missing something.”

Profile

L2 and L5 both turned on the system before starting the Profile (hard disk drive). L5 restarted from scratch when she saw her mistake, but L2 merely worried: “That might make a difference or it might not make a difference.” He was presented with a diagnostic Start From dialog box (menu), but at that time was also reading in the book that the system took a long time to initialize, so he gave it more time. After several minutes, he began to consider other possibilities. He turned the system off, checked the orientation of a disk he wanted to load, and then tried to start up again. He read several troubleshooting sections of the owner’s guide and then tried to load his disk in the lower disk drive. By failing to turn the Profile on first, he had made the system code stored on the Profile hard disk unavailable. Finally, toward the end of a 47-minute period, he tried to load another disk. Apparently by chance, he chose Office System 1, a backup of the system code. At that point, he was presented with several further dialog boxes (menus), which he did traverse successfully to initialize the system.

Response time

Many frustrations arose from the slow response time of the system. All of the five learners who managed to get the system initialized at all complained about response time. And this was more than inconvenience. In more than one case it led to serious trouble. L3 had tried to work on his own early in his second session, but had not succeeded. He decided to reload LisaGuide to refresh his memory, and in order to do this had to turn the system off. He did that correctly, but became impatient and pressed the on/off button before the system had finished its shut down. In doing this he hung the system up. Since he had experienced relatively long response times before in the shut-down situation, he just read on in the manual. After 18 minutes, he was sure something was wrong and asked us to intervene.

L4 committed a fatal error in similar circumstances. He was able to start the Profile, but became impatient during the long initialization period after pressing the on/off switch. This impatience led him to press the switch several more times before the system had booted, and to press an assortment of keyboard keys. The Start From dialog box appeared on the screen. Since he had not yet developed any skill in manipulating the mouse, he could not progress with the Start From dialog box. However, he did try the mouse button, clicking it several times. This caused the lower disk drive to be specified as the primary input device (instead of the Profile, so specified by default).

At this point he was doomed as a new user. Each time he tried to start the system, he got error messages (alert boxes and dialog boxes). Since the LisaGuide disk did not contain its own system, code, he could not load it. Indeed, after committing his error, L4 continued only one more hour. He then gave up in despair and asked to be excused from the experiment.

Reviewers of Lisa10 as well as the developers12 acknowledged the slow response time, but considered it a minor problem. From the perspectives of L3 and L4, it seems that little problems for experts often mean disaster for new users.

On-line tutorial

Typically, the learners tried to go it on their own-at first. In every case, however, they made some obstructing error or became confused. For example, L6 was confused about the click he was supposed to hear while the system was booting. L4 wondered why all the disks were stamped “Release 10” and what had happened to 1 through 9. These uncertainties convinced them to try the on-line tutorial LisaGuide. As L6 put it, “I guess I need to go through it. I’m missing something.”

Five of the six learners tried to load LisaGuide after having already booted the system. This caused an alert box (an error message display) to appear on the screen telling them to turn off the system and then turn it on again with the LisaGuide disk already loaded. All five were surprised that the system had to be turned off to load a new disk. L4, reading in the owner’s guide “..’be sure the Lisa’s power is off’...” convinced himself that there was a typo. L5 and L2 had problems in loading the LisaGuide disk, which diverted them from using LisaGuide. Both, however, made a fresh start in their second sessions, and at that time managed to load and run LisaGuide.

Rote learning with reinforcement

LisaGuide consisted of on-line lessons, each a series of exercises. Exercises were introduced, one after another, with no intrinsic rationale. They were to be followed by rote. When told to move the mouse in circles, L1 exclaimed, “Why are they telling me to do this?” The exercise seemed pointless to him.

Problems

Sometimes it happened that the method prescribed by LisaGuide was less efficient than one stumbled upon by the learner. While practicing replacement of text, several learners successfully and spontaneously employed the method of backspacing and retyping (instead of selecting a text range with the mouse and then typing to replace). They were disturbed to find the system advising them to be inefficient. L5 simply couldn’t believe that transactions with the Wastebasket could be as complicated as they seemed in the LisaGuide exercise (she was later relieved to learn that they could be simplified by shortcuts).

Another problem: Exercise steps were written so that warnings occurred after the opportunity to make a mistake, and often information was provided after it was relevant. Throughout the early LisaGuide topics, L1 was surprised that the cursor changed shape (from an arrow to a pair of opposed brackets) as it moved about and certain there was an important reason for this. He was quite annoyed that not until the end of the first topic of LisaGuide was he actually told that the cursor could change shape (he was still not told why). Subsequently, he was instructed to select a demo and only then warned that he need not have selected it if he had done well on the preceding exercise. He had already selected the demo at that point.

Figure 1. Executing the alternatives in order caused an exit from the tutorial before the learner even saw the second or third alternatives.
This image can be zoomedFigure 1. Executing the alternatives in order caused an exit from the tutorial before the learner even saw the second or third alternatives.
A more costly example arose in the “Stopping LisaGuide” topic chapter, as in Figure 1. L1 executed the first step without reading ahead and accordingly ejected the LisaGuide disk. “Oh, I didn’t mean to do that. Now I’ll have to start all over.” He did in fact start over, going step by step through LisaGuide a second time. The tutorial did provide a Topics pull-down menu so that learners could accelerate the training if they wished (notice the field labelled “Topics” on the menu-bar at the top of Figure 1). L1 found the Topics pulldown menu and was able to use it, displaying a selectable index of all the LisaGuide topics. However, he used it merely to confirm his current state, selecting the topic he was already in (Topic 1). Instead of skipping back to the “Stopping LisaGuide” topic, he kept reinitiating Topic 1. Finally, “I’m in a loop here.”

The printed material in the owner’s guide manual (accompanying LisaGuide) also presented information out of order with respect to the needs of the learner. As L1 put it, “I’ve already loaded the disk – now they tell me not to touch the shiny plastic!”

L2 said it would have helped him coordinate sequences of instructions if the LisaGuide screens had been printed in the owner’s guide. The LisaProject manual also seemed to have order problems, although as we noted only two of the learners got to use it very much. In one example, L5 was following the LisaProject manual to draw a diagram from a figure in the manual. Upon turning the page she was shocked to find a step-by-step procedure for drawing that diagram, out of order and too late to help her.

More problems

Even without out-of-order instructions and warnings, learners still got lost within the exercises. For example, L2 named the same object twice in immediate succession. He was following the manual and simply overlapped himself. L5 reread instructions for opening a folder icon immediately after having completed them and became confused about whether she had in fact completed them. The instructions, after all, were still there on the screen directing her to follow them. She tried to comply, by opening another folder icon. However, the system had disabled that choice in the File/Print menu (Figure 4 below shows an example of this menu) because she had just opened an icon. Menu selection in Lisa was context dependent; menu choices inappropriate to the current context were dimmed and could not be selected. L5 knew something was wrong, because she had used the menu before to open folders. But she could not figure out what was wrong or what she could do about it.

Figure 2. Executing Step 1 before reading Step 2 caused an exit from the tutorial without any hints about how to return. The “Do this...” heading attracted interest from learners before it was appropriate.
This image can be zoomedFigure 2. Executing Step 1 before reading Step 2 caused an exit from the tutorial without any hints about how to return. The “Do this...” heading attracted interest from learners before it was appropriate.
L1 and L2 had trouble with a LisaGuide panel containing five steps, as in Figure 2. Just prior to the first step was a heading saying to read both instructions before doing anything. L1 executed the first step before reading the second and was shocked when everything, including the LisaGuide window, was closed and set aside on the desktop. L2 ignored the first two steps entirely and focused on a second heading just prior to the third step, a heading that began with “Do this....” He then executed the next two steps. Because he had skipped the first two steps, he was unable to execute the last step. Save and Put Away was dimmed (disabled) in the File/Print menu, so he selected Set Aside Everything as a nearest approximation. But the “read both instructions before doing anything” actually referred to the two headings, not to the two steps immediately beneath it.

Problems like the foregoing resemble those typical of learners following self-study manuals. They underscore the generality and importance of such problems, and call into question the LisaGuide interactive training approach of placing self-study training on-line.

Resistance to reinforcement and rote learning

The tutorial provided positive reinforcement messages to the user each time an exercise step was correctly completed. L6 found these fatuous: “It’s silly to say excellent when they just told you that you did it wrong!” L2 felt insulted to be commended for following a relatively simple instruction. L1 became snarled in an error loop of repeated Tear Off Stationery operations, but when he did finish the step the system said “Excellent!” to which he replied, “The Hell it is!”

The learners seemed to resist the rote learning whenever possible. For example, L1 and L6 routinely executed the exercise summaries (which were intended to be read only). L1 detested the demos (which allowed a learner to watch an exercise step performed before attempting to do it): “I can’t stand the demo because it’s set up for a 4th grade reading speed.” L3 expressed impatience with the tutorial, focusing on the summaries: “It might be nice to give you the opportunity to do more, without running through this trivial example.” On his second pass through LisaGuide, L3 often jumped the gun on the exercise of particular skills. In many of these cases, LisaGuide obstructed his practice because he was out of sequence in the tutorial. L1 resented alert box messages implying that although he had done something wrong, the system would let him go on anyway. In these cases, he preferred to go back and redo the exercise step – and thereby take control away from the tutorial.

It seemed that the learners wanted to do something more concrete than follow a series of rote exercises. L2 balked at instructions to read a passage but not to do anything: “I’m tempted to do it anyway and then see if I can get out.” Indeed, he moved the mouse during a demo when instructed explicitly not to do so, then wondered why LisaGuide didn’t reprimand him.

Learners often tried to practice on their own within LisaGuide, like L6: “I just read how I could make a stationery pad from the menu, but I went to the menu and I couldn’t do it. They should let you make a note here.” An array of icons at the end of LisaGuide identified what was available in Lisa (refer to Figure 1), but one could not actually select them. After an hour of LisaGuide, L1 complained, “I’m getting impatient. I want to do something, not learn how to do everything.” After a hour and a half, he said, “I could have typed 3000 words by now.”

Error shielding

In many places LisaGuide permitted small user errors, correcting them automatically as the user passed on to the next exercise. Sometimes it informed the user that such and such a response was being altered to whatever. At other times, it provided no feedback or incomplete feedback. The response was altered surreptitiously, as it were. A typical example occurred when participants used names they had created instead of the names suggested by LisaGuide. At the end of the exercise, LisaGuide put in the name it had suggested and went on, briefly putting up a panel that explained that the user’s name had been replaced by the name the system had originally suggested.

Over-shielding

After such an error and error correction, the system offered several choices: Practice, Demo, Back, and Continue. L6 initially found these too telegraphic: “Practice what? Demo of what? Back to where?” He picked Continue, the only choice he had not eliminated. Still he complained, “It doesn’t tell you to go to Continue at the end of an exercise.” Later, he tried the Back option to find out why a name he had typed in had been replaced. However, LisaGuide did not take him back to the explanation about names, but rather back to the prior exercise step (the point at which he was originally asked to type in a name). Indeed, the only way he could truly go back to the point he wished to go to was to recommit the same naming error. L5, who made very few mistakes in LisaGuide, wondered why the system couldn’t just go on and spare her the trouble of selecting Continue after each step.

L1 tried to rename an object, but misspecified it (using the mouse). He typed in the new name without consequence (no beep, no error message, no feedback at all). When the system asked if he was sure that he understood (intended as tactful and positive reinforcement), he laughed, but went on anyway. LisaGuide put in the name he had failed to put in, and several minutes later he was quite surprised to find it there.

In a converse example, L1 (as well as L2 and L5) typed in several fields of a practice memo that he had not been told to type in. LisaGuide deleted his extra material at the end of the exercise step. Surprised to find this material gone, he retyped all of it. (Of course, LisaGuide again removed it on the next exercise step.)

In another incident, L1 selected Set Aside instead of Save and Put Away in the File/Print menu. At the end of the exercise step, LisaGuide informed him of this error and gave him the option of going back and redoing the step. He did, and correctly used Save and Put Away, but ignored the rest of the step – the operations he had already correctly practiced on his first time through. Having finished the step the second time, he was again informed that he had incorrectly performed the step.

L2 encountered a similar problem selecting multiple icons. He correctly completed the LisaGuide exercise step on this, but began to wonder why one would ever want to select multiple icons in the first place. He backed up to the relevant step and reread the panel. Satisfied, he moved on again, but at that point LisaGuide put up an alert box because he had not actually done the exercise step. He felt compelled to work through everything again. Accordingly, he backed up and selected a new group of icons (for variety). But of course he had again failed to do just what the exercise step said, and LisaGuide treated his new effort as an error, too.

LisaGuide’s error shielding also prevented learners from practicing individual skills they had mastered. L2, having just learned about scroll arrows, tried to operate them on the LisaGuide window. However, scrolling operations do not work in the LisaGuide window, preventing this sort of practice. This caused frustration and confusion since it made the scrolling operations appear to work unreliably.

Under-shielding

The foregoing examples suggest that error shielding creates some problems for new users, but we also saw cases where correcting errors only at the ends of exercise steps provided too little shielding. In the second chapter of LisaGuide, L1 spontaneously decided to practice icon selection. In the course of this he positioned an icon so that it partially occluded another icon. He was then unable to select the occluded icon, which was required for the next exercise step. He tried to correct this in several ways, including performing the exercise step on another icon. But all of his attempts failed. When he gave up and tried to go on, LisaGuide automatically corrected the error, but in a sense it was too late.

L2 resized a window so that it was wholly within (on top of) the LisaGuide window. Later, when he activated the LisaGuide window, this other window became wholly buried under the LisaGuide window (the currently active window moves to the “top” of the display area). Hence his other window became inaccessible for further use.

Likewise, L3 was practicing moving icons around the desktop and accidentally released a folder icon while it was superimposed over another folder icon: “I lost Supplies!” By releasing the icon over a folder, he had placed that data object into the folder, hence it became invisible on the desktop.

Error diagnosis and recovery

The feedback LisaGuide provided for learner errors was in many cases too general to help the learner adequately understand the problem. For example, most learners made errors coordinating the selection of icons from the display. LisaGuide delivered identical error feedback for naming the correctly selected icon with the “wrong” name as it did for selecting the wrong icon but naming it with the “correct” name (the name LisaGuide suggested). In the second case, the error feedback was all but uninterpretable.

When directed to put a folder into the Wastebasket and then to recover it, L1 put it first into the Wastebasket, then into a disk window, and then back into the Wastebasket. The alert box said “You put it into the diskette instead of onto the desktop” – an inadequate description of what actually transpired.

As in the case of the selection/naming errors, LisaGuide provided some support for error diagnosis and recovery, but not enough. However, in both cases the system purported to be providing complete support, which led to confusion for learners.

Figure 3. Overly terse error diagnosis and recovery help, as in Step 3, invited learners to create ad hoc interpretations.
This image can be zoomedFigure 3. Overly terse error diagnosis and recovery help, as in Step 3, invited learners to create ad hoc interpretations.
In dealing with error recovery, LisaGuide often employed somewhat “cute” checkpoint questions, such as “Doesn’t say Set Aside ‘Clipboard’? Redo Step 2” (see Figure 3). This message meant that if the File/Print pull-down menu did not list as a choice “Set Aside ’Clipboard’” the user should redo Step 2 in the exercise. L3 read and reread this message. Even though he had successfully completed the exercise, he backed up a step and began repeating part of his work. At that point he did make a mistake. In response, he backed up another step. At that point he chanced to notice that “Set Aside ’Clipboard’” was listed as a choice in the File/Print menu. From this sequence of events he drew this conclusion: “Maybe they’re telling me that it really doesn’t mean set aside Clipboard.” Unfortunately, this conclusion was entirely wrong.

Summary

The tutorial was far less successful than we had either hoped or expected. The learners did not like it, did not cooperate with its rote learning approach, and worst of all did not seem to gain much. Most of them spent most of their time running and rerunning LisaGuide, but when they went to apply what they had learned they could not. Minutes after finishing LisaGuide, L1 became puzzled by the icons at the bottom of the screen: “What’s all this junk at the bottom?” After two further passes through LisaGuide, he complained that he hadn’t learned about any of the pull-down menu headings on the menu bar (he had in fact used some of them). He summed up: “I am tremendously frustrated. I’ve done a lot of work and I can’t do a damn thing.”

L3, who had finished LisaGuide 16 minutes earlier, tried to create his own document and simply could not get started: “I have completely forgotten how to open, enter, to create a document!” He could only open and close icons and experiment with choices in the pull-down menus. After several cycles, as if in desperation he typed a burst of characters in an open window. Fortunately, the window was a blank document, so he succeeded.

Ironically, the only two learners who succeeded in making serious progress with LisaProject (the overall goal we gave the learners) were L2 and L5, who initially inserted the LisaGuide disk incorrectly and therefore could not use it at all during their first sessions. These incidents give a somewhat ironic twist to Merkin’s claim10 that “Learning how to use Lisa by trial and error is so much fun that to learn it methodically by going through the tutorials provided for each software package seems insipid.”

The desktop

Figure 4. The Lisa desktop. The Profile icon has been selected (it is highlighted in the lower right of the desktop) and the File/Print menu has been pulled down.
This image can be zoomedFigure 4. The Lisa desktop. The Profile icon has been selected (it is highlighted in the lower right of the desktop) and the File/Print menu has been pulled down.
The Lisa interface was one of the first to systematically and pervasively exploit a coherent interface metaphor: the desktop (see Figure 4). The user was encouraged to think of the display as a desktop containing objects that could be physically manipulated. But this interface posed problems for the new user, some of them terminological complexities peripheral to the desktop interface metaphor.

For example, L4 read the description of the Calculator and became confused by its technicalities. He did not know the terms “reverse Polish calculator” or “four function calculator,” nor did L1 or L6. L3 was confused by the term “highlighted.” After several minutes of puzzlement, he exclaimed, “Oh! When it turns from white to black it’s highlighted.”

Other problem terms were “profile” (which refers to the system’s hard disk), “preferences” (which refers to the default display settings), and “tools” (which refers to the Lisa applications, like LisaProject, and icon functions, like the Clock and the Calculator).

The term “tool” serves as a good example of an ancillary metaphor too general to imply anything useful. L5 became snarled in confusion between untitled icons and application “tools”: “They tell you to use the tool to tear off a piece of stationery, but when you go to do it you’re not allowed to in this screen.” The confusing message about using the tool suggested to her that the tool was indeed like a stationery pad, or even like a piece of stationery. She looped through a sequence of selections, failed renamings, openings, closings, etc., for over half an hour before giving up. The “tool” here was the application function (LisaWrite text-editing operations), not the application objects to which the function applied (pads, stationery).

Unfortunately, such conceptual problems arose even more frequently for terms more central to the interface, such as “icon” and “clicking.” L4 noted that “icons” are religious figures, and in fact never made the connection between that term and the system’s little interface pictorials. He confused “mouse” with “icon” when reading the instruction “click twice on the calculator icon,” concluding that the thing he was to click twice (namely, the mouse and its button) was the calculator icon.

L2, L4, and L6 had trouble with the term “clicking.” As L6 put it, “What does ‘clicking in the little picture of a diskette’ mean?” Not until the end of the second chapter of LisaGuide did he find out: “Now they tell me....” L2 pursued several references to the “clock” in the owner’s guide when he first encountered the term “click.” Later, he encountered a dialog box instruction to “check” boxes and wondered whether “click” could be the same as “check” (it was in this case, which reinforced his confusion).

The metaphor

Other troublesome concepts and labels exposed problems with the desktop metaphor itself. The first class of these involved problems people had understanding the desktop meaning for terms like “clipboard,” “stationery pad,” “typing,” “tear off stationery,” and “folders.” For example, L1 read the references to the “Samples diskette” in LisaGuide (which referred to the diskette icon on the desktop but not to any real disk) and looked for a floppy disk.

The manipulation of stationery pads seemed to the learners to be mere overhead. To get a piece of memo paper or a folder in Lisa they had to select a template icon (called a stationery pad), “tear off” a copy using the File/Print menu, and only then open the new icon for work (assigning a name to the new icon was optional). L1 complained, “I write my memos before tearing them off. If they’re going to use analogies they ought to do them right!” L1 particularly objected to the metaphor as applied to folders: “Tear Off Stationery has no relation to empty folders – it’s being cutesy beyond understanding.” L6 complained about the necessity for selection, tearing off stationery, reselection, renaming, and opening just to create space for data: “Why must I tear off just to get at something?”

The metaphor distinction between stationery paper and stationery pads often caused confusion. For example, after L3 selected and opened an icon he was unable to edit it because it turned out to be a stationery pad. After tearing off a copy of a stationery pad and opening the new icon, L3 was still thwarted in trying to type anything because he had been working with a folder stationery pad. To him, stationery pad suggested something you could write on, but in Lisa “stationery pad” was a technical term.

L5 also created a plethora of accidental icons as she tried to sort out “duplicate,” “tear off,” and “open.” Later, when she cleaned up her desktop (by moving these icons to the Wastebasket) she discarded her (renamed) LisaProject Paper icon. The system allowed her to do this even though it obstructed her further progress with LisaProject.

L1 and L6 both tangled aspects of the Tear Off Stationery problem with a problem interpreting the Clipboard metaphor, when they tried to Tear Off Stationery from the Clipboard to type something. L1 also tried to type directly onto the Clipboard and then Edit it, evoking alert box error messages both times. Note that from the standpoint of the desktop metaphor, all of these errors and confusions were perfectly reasonable. In fact, the only purpose of the Clipboard was to temporarily store things during cutting and pasting; it was not really a general scratchpad (as its name implies). L6 wondered why there was a Clipboard on the desktop at all.

Even the window concept, critical to the interface, did not derive from the desktop metaphor. Neither did the scroll bar or elevator concepts (icons that appeared in a Lisa window and were used to control scrolling). These misfits, gaps, and misleading cases weakened the efficacy of the metaphor.13 Indeed, since they were never addressed as concepts in the interface or the documentation, they created additional burdens of metaphor comprehension and metaphor confusion.

L1 balked at the LisaGuide description “...closes a window and places it on the desktop.” He said this meant nothing to him, but conjectured that it might mean that something was deleted (a wrong guess). L2 learned about scroll bars, but could not keep the concepts together. Within minutes of learning it, he could not remember what a scroll bar was. (L1 had a similar problem.)

Prior experience with computing systems

A second class of metaphor problems involved confusing Lisa concepts with other computing concepts, and expectations derived from other systems. L1, for example, balked at “folders,” preferring the more abstract term “files,” which he had had prior experience with in his current system. “If they’d just tell me that we’re using analogies and later will switch to computer language, I’d feel better.”

L2’s puzzlement with the term “click” was exacerbated early on by the fact that he limited his consideration to the keyboard keys. L4 became convinced that he needed a blank disk to store his work on, and that that single problem was the key to his difficulties. Unfortunately, his difficulties had nothing to do with this expectation, presumably grounded in his prior experience with other microcomputer systems.

Other notable concepts in this class were “enter” and “edit” and keyboard function keys. L3, while following a LisaGuide exercise, was disturbed to find that having typed in a string he was not then directed to press the Enter key (something he expected based on prior experience with other systems). Reasoning that he should go ahead anyway, he became disturbed to find that pressing Enter had no apparent consequence. Somewhat ironically, the Lisa designers specifically included an Enter key to be compatible with other product designs.12

L1, L2, and L3 were all surprised to find that the Edit menu contains only operations for cutting, pasting, and copying. The systems they had used before employed the term “edit” to refer to all operations on documents, including document creation. L1, while using LisaGuide, complained that he needed a keyboard function key for Continue, the operation that advanced users through the series of exercise steps (and which in Lisa was a soft-key, a field on the screen that was selected using the mouse).

We even saw evidence that prior knowledge of basic keyboard layout interfered with learning about Lisa. For example, L3 tried to stop LisaGuide by pressing the Shift key in conjunction with the Period after being instructed that the correct combination was Apple plus Period. L5 also had trouble with the Apple plus Period combination. She executed it correctly, but it failed to work because she had never actually loaded LisaGuide. At that point, she found another (irrelevant) Period key on the numeric keypad to use in the combination.

Conceptual distinctions

Several Lisa concepts seemed to cause problems chiefly because they were confused with other Lisa concepts, quite aside from being jargon or obscurely metaphoric. We have already mentioned the confusion between applications (tools) and their parts. We also observed confusion about “view,” “select,” and “open”; “set aside” and “save and put away”; “menu” and “menu bar”; “icon,” “icon ghost,” and “window”; and “duplicate” and “tear off.”

L3 became confused between “opening” and “selecting.” While working with the Clipboard open he went to the File/Print menu: “I’m surprised that I can only select Set Aside, and that I can’t open it.” L3, while using LisaGuide, was directed to open an icon. He ignored the direction because he couldn’t find the choice Open in the menu bar (the choice was in fact listed in the File/Print menu, available from the menu bar). L6, trying LisaCalc, concluded that he needed to have the Calculator open in order to use it, but couldn’t figure out how to use them together.

L6 selected an icon and went to the View menu to “view” the window, confusing the View menu with the File/Print menu. This error quickly became established and he made it repeatedly during both sessions. He would reorganize his icons chronologically, alphabetically, and pictorially, but could not open anything. A variant of the error that developed late in his second session was selecting the Edit pull-down menu for functions like Open and Save and Put Away.

L6 and L3 confused icons with the windows they stood for. L6 picked up a window and moved it to the File/Print menu to save it (instead of selecting the object and then pulling down the File/Print menu to operate on it). L3 picked up a window and moved it over the Wastebasket icon to throw it away. Even later in the second session, after more than four and a half hours of experience, L3 tried to put away an object by picking up the window and superimposing it on another window.

Figure 5. Icon ghosts. The LisaProject Examples icon and the Ad Schedule icon were nonfunctional placeholders for currently open windows. They could not be selected or manipulated.
This image can be zoomedFigure 5. Icon ghosts. The LisaProject Examples icon and the Ad Schedule icon were nonfunctional placeholders for currently open windows. They could not be selected or manipulated.
L2 confused icons with icon ghosts. (When icons were opened into windows, ghost images of them remained as nonfunctional place holders. See Figure 5.) While his Wastebasket was open, he tried to discard a document by dragging its icon to the icon ghost of the Wastebasket. When he released the document icon over the Wastebasket, it returned to the desktop. L3 also had this problem in trying to place document icons into icon ghosts of open folders.

The distinction between Set Aside and Save and Put Away troubled our learners. L1 was disturbed by Save and Put Away because he felt that the two conjoined operations were distinct and should be listed separately. L3 experimented to differentiate Set Aside from Save and Put Away. In the course of this, he tried to Save and Put Away the Clipboard. The Clipboard, however, could not be saved and put away.

L6 decided to Save and Put Away a disk (which could only be Set Aside). He rejected the Set Aside option and switched objects instead, finally saving and putting away a folder.

L5 developed a baroque procedure for saving. She would first select Set Aside from the File/Print menu, then drag the icons from the desktop into the Profile window to save them (which is precisely what the single operation Save and Put Away did).

We also saw some confusion about pointless nondistinctions in the system’s jargon. L6 became confused about the difference between “top of the desktop” and “active window” when there was no difference.

To summarize, the desktop metaphor was far more complicated than it might at first appear. It is quite unclear whether it facilitated learning more than it puzzled and misled our learners. Even aside from the metaphor, the many fine distinctions in the Lisa commands and concepts caused numerous learning difficulties. Surprisingly, in view of the various learner problems we observed with respect to the desktop metaphor and other Lisa concepts, the trade press assessment was entirely positive: “The Lisa’s software commands, such as Store, Save, Put Away and Print, are straightforward.”6

Basic user interaction

Selection and pointing

The most basic interaction channel, particularly for the novice user, was mouse positioning and clicking for selection. This format obviously differs from more traditional menu selection and cursor movement with step-keys. The learners we studied seemed to have problems both in the basics of pointing using the mouse with simple clicking and in more advanced operations involving multiple coordinated mouse clicks.

The mouse

While the mouse device had useful properties, it also posed problems. L1 and L6 felt strongly that the mouse was for 15-year-olds, but not for them. L1 said that the mouse click was not factually obvious enough. L1 and L3 had trouble coordinating cursor position, especially when positioning the cursor to type in text (or to use Space or Backspace).

Interestingly, this problem persisted throughout both sessions. Even after five and one half hours L1 had trouble positioning the cursor to delete text with the Backspace key. L2 pulled down the File/Print menu to open a selected icon, but as he pressed the mouse button and checked in the manual to be sure he was right, his finger slipped and he duplicated the icon instead.

The converse problem, in which a person did not click in a field but selected it anyway, also occurred. While printing, L3 attempted to redundantly specify the print quantity parameter. Since that field had already been selected, his mouse click was taken by the system as a selection of another field, one horizontally across from the print quantity field on the display, namely the Cancel field. Thus, in trying to redundantly specify the print quantity, he actually canceled his print job. He failed to notice any of this, but did notice that the printer did not produce any output. (L3 had a similar experience using the scroll elevators which, once on, continued to cause scrolling even if the cursor was moved out of the elevator “shaft.”)

Double-clicking (a shortcut in which an icon was both selected and opened) also invited clicking errors. L5 routinely clicked too slowly and hence selected but could not open her icons. L2 also had this problem, but later reasoned out how double-clicking ought to work. Unfortunately, he chose to test his hypothesis on a stationery pad icon. Double-clicking a stationery pad selected and then tore off an untitled piece of stationery, instead of selecting and opening. L2 was puzzled by the result of this experiment.

Pressing versus clicking

The system distinguished between pressing and holding down the mouse button and clicking (quickly holding down and then releasing the mouse button). The wording of system prompts always carefully respected this distinction, but our learners nevertheless found it troublesome. After one and one half hours of experience with the Lisa, L1 wondered, “What’s the difference between clicking and pressing?” One difference was that pressing the mouse button on an icon allowed one to drag the icon around the desktop. L5 pressed the mouse button as the second half of a double click and hence ended up reluctantly dragging a highlighted icon around the screen.

A variety of selection errors occurred throughout the sessions. Each of the five learners who managed to get started had trouble with the basic select and open procedure: click an icon to select it, then move the mouse to the File/Print menu and press (not click) the mouse button. Learners initially clicked the mouse in or near the File/Print pull-down menu, instead of merely pressing it. L2 explained that he wanted to, but could not, get the pull-down menu to stay down.

These unnecessary clicks had the side effect of deselecting the originally selected icon and accordingly of dimming (that is, disabling) menu choices associated with the icon. Hence, a learner who clicked instead of pressing in the course of using the File/Print menu to open an icon might well have found the Open choice in the File/Print menu disabled.

This type of error occurred with a variety of operations. In trying to close a window, L1 and L3 miscoordinated the mouse click for selection and hence failed to find the desired Save and Put Away choice in the File/Print menu. Both pointlessly clicked on the dimmed Save and Put Away choice anyway, which had no effect (similar to clicking on an icon ghost), but which elicited no feedback from the system, either. L3 attempted to select an icon and then move it into the Wastebasket. He slipped with the mouse button and deselected the icon before dragging it to the Wastebasket. In the end, the Wastebasket icon was selected (and highlighted) and the original icon was exactly where it had been when he started.

L6 made the same selection error trying to replace text, and succeeded only in replacing a portion of the text he had tried to replace. L1 made an error in trying to rename an icon, then slipped and selected the wrong icon, putting the new name onto another icon entirely. L5 pressed instead of clicking on the Wastebasket when she wished to open that icon. She ended up rather unhappily dragging the icon up to the File/Print menu.

Understanding selection

Some of the learners had trouble with the cause and effect relations of selection through pointing and clicking. L6 expected that selecting a window would cause it to get larger as well as to come to the top of the desktop stack (possibly he confused “icon” with “window”). He tried to select an already-selected window, found that nothing happened, and concluded that selection as described did not work.

L5 deselected a window and commented, “I got rid of part of the menu.” From the system’s standpoint, she had brought a different window to the top of the display stack and thereby occluded part of the original window. Her description did not include any notion of window stack or selection in its cause/effect analysis.

L1 developed the belief that there were two methods for selection. One was to select an icon and then go to the File/Print menu to Open. The second was to drag an icon up to the File/Print menu. Actually, the second method never worked, but a significant element of it was useful, namely, holding down the mouse button. L1’s most typical selection error was to fail to select initially or to deselect along the way to the File/Print menu. Hence, if he focused on the goal of dragging (and its requirement that the mouse button be pressed and held), he was more likely to execute the first selection method successfully – albeit with an incomplete understanding of what he was doing right. Nevertheless, a side effect of his procedure was that he often ended up with icons collected in the upper lefthand corner of the display.

There were a variety of where and when problems with selection. L1 clicked in the wrong window when two windows sat on top of each other. L3 and L5 clicked on icon ghosts, which activated the window within which the ghost icons were located, but had no effect on the ghosts. Moreover, when L3 and L5 subsequently went to the File/Print menu, choices pertaining to the ghost icon were not available. Halfway through the second session, L3 was surprised to find that a window could be activated by a click anywhere within it. He had remembered that activating a window required clicking in the field containing the window’s name. L6 accidentally selected all his icons simultaneously and then could not stop moving them in unison each time he moved the mouse.

People sometimes became slightly superstitious about selection. The menu bar changed, as did the choices in the menus themselves, as the system environment changed. The View menu, for example, initially listed on the menu bar, disappeared when application windows were opened and was replaced by menus specific to the applications. L3, having just opened a LisaWrite window, concluded that he had lost View by sloppy operation of the mouse. He was very careful for a time after that, but View did not reappear.

Throughput

Managing the Lisa desktop was quite different from interacting with more traditional interface designs. For example, the user did not have to specify names for data objects; the system would leave them untitled. L6 was puzzled, but also pleased, that he could have several objects named “untitled,” perhaps based on his experience with other systems. However, L1 named two things “First File,” his own folder and the Profile, and later became confused about what the name referred to.

Typing

Typing in Lisa drew upon a paper page and typewriter metaphor. The display was black-on-white, and at times the metaphor was really quite compelling. However, it broke down somewhat during editing (insertion and replacement of text) and when moving and copying text.

For example, L3 was troubled by the fact that Lisa typed in insert mode, placing new characters between existent characters and adjusting prior material to the right. He initially assumed that the Space Bar would delete by typing blanks over his prior work; however, he found that it inserted blank spaces and merely pushed his prior work aside. L3 seemed to want a typeover capability so much that he invented one. He combined the replace mode, in which one selected a piece of text with the mouse and then typed in replacement text, and the insertion mode, in which one positioned the cursor and then typed. He stated that he could position the cursor with the mouse and type over the existing text (which was wrong).

Lisa’s metaphoric typing page. L1 turned on Center and then had trouble getting the signature line in his letter where he wanted it. The system continued to center everything he typed. L6 selected a portion of text adjusted to flush right. The text became highlighted, but so did the rightmost spaces between lines of text (which concealed invisible formatting characters).

Cut, copy, and paste

Operations that deleted, or temporarily hid, data were always dangerous for our learners. L1 inadvertently selected his entire document and then cut, which deleted all his work up to that point (actually, it was temporarily saved in the Clipboard, but L1 did not know this). He retyped the entire text, but then selected Revert to Previous Version. He thought that this would restore his original text, but in fact it reverted to the version immediately following the cut, again deleting all of his work.

The potential costliness of Cut was exacerbated by an unintuitiveness in the ordering of Cut and Paste, at least for L1. He consistently selected the Paste operation before cutting the text he wanted to paste and before specifying the location. Perhaps he felt timid after his earlier experience with Cut, but the error persisted throughout the six hours. In one case, it led him to paste leftover remnants of another document into his current work.

Toward the end, when he finally got Paste to work, he became snarled in a paste loop, cutting and pasting over and over.

Figure 6. Checking the Clipboard for intermediate results of a cut/copy operation resulted in placing a selected piece of text into the name field of the Clipboard.
This image can be zoomedFigure 6. Checking the Clipboard for intermediate results of a cut/copy operation resulted in placing a selected piece of text into the name field of the Clipboard.
L6 also had trouble coordinating selection of text to be cut/copied. Worried about this, he tried to check the Clipboard for the temporary results. After specifying the text to be cut/copied, he selected the Clipboard (actually he selected the name-field of the Clipboard icon) to check for results. When he then selected Paste, his text was copied not into the Clipboard but onto it, as a new name. A reconstructed example is shown in Figure 6. L6 had accidentally discovered how to rename icons. After this, although the File/Print menu continued to refer to the Clipboard by its original name (Clipboard), it remained permanently relabeled for this learner, who ignored it.

L1 wondered why Cut and Paste was not integrated into a single Move, which would have eliminated the risk of losing data with Cut: “It’s funny that with this mouse capability, you can’t just select and move text instead of this cut and paste.”

The Wastebasket

Several of the learners became quite absorbed with the Wastebasket. L6 duplicated an empty folder in the Wastebasket, which unfortunately led him to believe that certain types of work could be carried on there. Later, he threw away the LisaList Paper and the Calculator icons, although he commented immediately, “It shouldn’t let you do this!”

The same problem occurred with L5, who lost her LisaProject Paper to the Wastebasket. In the midst of working with the application she had decided to clean up some practice documents. Earlier she had inadvertently renamed the LisaProject Paper icon, hence it was accidentally thrown away as a practice document.

L2, while trying to close the Calculator window, wondered whether the Wastebasket might play some role in putting things away. Fortunately (in a sense), he was unable to operate the Wastebasket. He tested the use of the Wastebasket by placing an icon directly into the open Wastebasket window. This would have worked, but he forgot to release the mouse button and hence dragged the icon back out of the window. He concluded that the Wastebasket had to be closed on the desktop in order to function.

Undo

Lisa provided an Undo operation, but it tended not to be available when needed or wanted. For example, L3 in experimenting with the Clipboard made two successive cuts from a document. He then selected Undo and restored the first cut. He selected Undo again, but merely obtained the system message that the previous operation could not be undone. The irony here was that even if the Undo operation had worked it would not have done what L3 wanted. Lisa’s Undo did not stack prior system states, it only retained the prior state at any given point. Hence, it merely alternated the immediately preceding state with the current state.

Figure 7. A typical system response to an Undo request.
This image can be zoomedFigure 7. A typical system response to an Undo request.
A typical system response to an Undo request appears in Figure 7.

Oddly, reviewers found the Undo more impressive than our learners did: “With this command, even the most uncertain users will not hesitate to act in a way they think is appropriate.”14

Printing

Our learners experienced a variety of problems getting output from Lisa. L6 requested a print when the printer was turned off. A rather complicated alert box referred to technical difficulties and advised the user to check cables, ribbons, and paper; to read through section G of the owner’s guide; and to report Difficulty Number 3056. Ironically, the extensive alert box did not say to check the on/off switch. Moreover, when the participant ultimately turned the printer on, he had to reissue his print commands to get output.

Other problems involved ancillary print commands. When L2 requested his first print job, a LisaProject plan, he had to respond to three dialog boxes in succession to get a print. He wondered, “Why are they giving me such a hard time?”

In other cases, learners found print commands irresistible. The only menu choice available in every context from the File/Print menu was Monitor the Printer. Learners tended to select this choice early on, when they had no print jobs to monitor. When L1 went to print, he noticed that Format for Printing appeared prior to Print in the File/Print menu. He did not need to format for printing, but selected it anyway. This error had little consequence, merely delaying him. (In a similar case, when trying to Tear Off Stationery, he selected Duplicate because it too came first in the menu; this latter error had substantial consequences.)

LisaProject

Our original intention was to study LisaProject, perhaps the most innovative and original of the Lisa applications. Our learners seemed utterly unimpressed with LisaProject. Said L6, “I’ve never in my whole life had to do a chart like this or wanted to. I can’t imagine a situation in which I would.” Later, “It’s just flowcharts.”

This perception of limited usefulness might be ascribable to a lack of prior knowledge about PERT charts, upon which LisaProject was based, on the part of our learners. We did not ask them about this.

Aside from the issue of perceived usefulness, there were a variety of problems much in the spirit of selection problems we saw in other contexts. For example, L2 had trouble selecting the title field on his LisaProject Paper. L6 opened the LisaProject master and was told to Tear Off Stationery from LisaProject Paper, but the LisaProject master window had covered up the Profile icons when it opened, including LisaProject Paper.

L5 tried to connect a series of task boxes, but ended up selecting pieces of text within the boxes while positioning the pointer. When she did succeed in connecting boxes, the click establishing each join occasioned a brief flash on the display. The first time, L5 worried that she had lost the box.

L5 connected the task boxes in the wrong order and hence produced a project plan diagram that did not match the example in the manual. To explain this to herself, she pointed out what was in fact an irrelevant typo as the cause of the mismatch. Later, having corrected the typo, she examined the project diagram and found it still mismatched: “Wrong. I must have put another typo in.”

L6 had trouble moving the Start circle because he clicked within the circle (generalizing from other selections) rather than on its boundary, as he should have. L6 began typing in the LisaProject window, but was immediately interrupted by an alert box with the message, “Before you type make a selection.” This puzzled him, but eventually he selected the Start circle and resumed typing not a label, but text. The circle grew until he had typed 50 characters, at which point he was again interrupted with an alert box. By that point, however, the Start circle had covered up most of the window.

L5, as noted earlier, threw away her LisaProject Paper icon. She tried to use LisaDraw Paper instead and rapidly came to the conclusion that she could not substitute. She diagnosed her problem, however, and tested it experimentally by throwing away the LisaDraw Paper icon. (For her second session, we restored the LisaProject Paper, based on our expectation that she would not be able to recover from this error.)

A variety of problems pertained to LisaProject in particular. Both of the learners who managed to work with LisaProject had difficulties setting dates for project plans. They seemed to have little intuitive understanding of what the application was doing. L5 tried to set dates for a single task box. When she finally reached the Set Schedule Dates dialog box, she made the total duration of the project two weeks, the time allotted for the first task. L2 wondered, “What does early start or late start mean? At this point I’m just following the instructions without thinking.” When L2 tried to set dates for his house building project, he became snarled in confusion among “calendar range,” “current date,” and “start date.”

Similar problems arose in the area of resource allocation. L2 created two separate task boxes, in parallel, for a two-man task – a legal but baroque solution. L5 placed a two-man resource on two lines of a single task box, which was more efficient but incorrect. LisaProject interpreted this as two separate resources (and since L5 placed the resource amount parameter next to only one name, her other resource was syntactically incorrect as well). L2 joined three parallel task boxes to two parallel task boxes with six lines (LisaProject expected a milestone circle at junctures like this).

Finally, both L2 and L5 found the application too inflexible. Both noticed that the user could not update the project plan from the resource task representations. L2 also wished to be able to specify variable amounts of resources for task boxes.

Remarks

In our description of professionals learning to use Lisa, we have tried to stay as close to pure reporting as possible. Our goal, to that extent, has been to catalog the problems as we observed them. In this final section, we take up a slightly higher focus, and remark more generally on the design process, on the nature of learning, and on the use of interface metaphors.

Designing for usability is not simple

Our starting point is a paradox: Lisa was deliberately designed to be very easy to learn. The claim was made that people could be working productively after 30 minutes.14 And this was not merely wishful thinking. Research and development work specifically targeted this goal. The desktop metaphor was developed to help make new users “comfortable.” Nevertheless, the 30-minute estimate was off by as much as an order of magnitude for our learners. And the desktop metaphor caused specific and tangling problems for new users. How can this be?

The designers of Lisa, like an increasing number of system designers, reported that they developed the user interface design in parallel with other aspects of the project.12 They ran user tests during the development process. Do we conclude that their tests were misdirected? Inadequate? No. The only reasonable view seems that in spite of pertinent and timely research and development work, the Lisa interface was formidably difficult, even for professionals with some computer experience.

This really should not be surprising. Lisa was a powerful workstation with many diverse capabilities and a detailed interface. Details really matter, because they can tangle into perplexing morasses of confusion for users. The Lisa interface contained many novel elements, so that any amount of testing would from some standpoint have been inadequate. Our view is simply that the job of innovating new interface concepts is an open-ended one. We are confident that the Lisa design process improved the product’s interface, as we are that the interface can be improved further. The question is not “How could Lisa have been so bad?” but “How can we learn from the Lisa experience and go on?”

We do feel that the intensely enthusiastic reception of Lisa by the small-systems community reflected attitudes that ought to be examined. Every review we read proclaimed that Lisa was easy to learn, but none provided any empirical basis for such a claim. Perhaps reviews ought not to be as casual as they sometimes are today. In an otherwise insightful review, Williams14 referred to the following alert box message as “incredibly clear”: “The tasks and milestones cannot be moved off the drawing. Try enlarging your drawing size with the Page Layout menu.” True, this is far from the worst error message ever written (and it was not the worst in Lisa either). But for the user who is not sure what a “milestone” is (they were never labelled as such), or a “task,” this message is quite useless. And as our descriptions show, such troublesome situations did occur for new users.

Focus on critical user tasks – like getting started

Our study suggests some major problem areas in the Lisa design. We would like to underscore three of these.

First, problems associated with initializing the system were potentially disastrous for new users. Indeed, one of our six learners fully realized this potential and another escaped narrowly.

Second, the training available for Lisa was inadequate. The manuals were fat and redundant, and learners had trouble using them. The on-line tutorial seemed inefficient and ineffective; it was frustrating and even insulting for learners.

Third, the system’s interface – mouse pointing and selection, the graphic desktop, etc. – was clearly no design panacea. Selection was difficult to coordinate and desktop icons difficult to understand. Some of our learners found the entire interface concept just plain silly. Moreover, the consistency and integration of the interface often disappeared in all its difficulties. For example, we understand from Don Hatfield that the actions of resizing a window and of resizing a LisaProject task box were designed to be thought of as a single sort of operation that applied across applications. Neither we nor any of our learners noticed this consistency.

Systems can be designed to avoid the serious getting-started problems we observed. One way to organize the design process to achieve this is to analyze the critical user subskills that are prerequisites for accomplishing typical user tasks.15 In Lisa, examples of these critical user subskills would include switching between and integrating different applications. The design of Lisa does suggest that attention was in fact paid to empirically testing and refining the interface for such user subskills, but that less attention was paid to the subskill prerequisites for getting started.

Design training materials for realistic active learning

Getting the system started, as we have seen, was only the first of many challenges for Lisa learners. One key problem with Lisa was the same one we have documented in prior research: Learners do not do what designers want them to; instead they tend to get actively involved and to think and plan and solve problems.16 This thread permeates the description we have given of professionals learning to use Lisa, particularly for our description of the use of the LisaGuide tutorial.

Many times these active learning strategies led to successful outcomes. For example, L2 had trouble understanding the Wastebasket and conceived of several experiments in which he tried to throw away objects, then opened the Wastebasket to check on his success. He noticed that pulling a menu down from the menu bar temporarily halted the printer and tested this several times to be sure he was drawing the right conclusion. He worried about the automatic dimming of the screen, so he tested its connection to mouse movement by experimenting. In each case, causes and effects were systematically correlated as he actively created his understanding of the system. The fact that Lisa made many system functions visible allowed this active learning approach to succeed.

There are of course numerous pitfalls to active learning. For example, one can rummage around in a domain but not get to the important content. L1 and L6 had interactions with the Calculator that exemplify this problem. They were both attracted to it because it seemed familiar and perhaps simpler than LisaProject. However, L1 became absorbed in the fact that some of the buttons didn’t look as much like buttons as did others (such as the MC button). L6 used the Calculator but failed to try any memory functions, commenting, “So that’s all the Calculator can do.” In both cases, the exploratory learning had a less than inspiring outcome: Neither individual mastered the Calculator.

When learners do notice some feature, they seek an explanation for it. They can become frustrated and/or confused if no explanation is obvious, as L5 indicated in her reaction to an inscription she found in the border of a document window: “4692 blocks free out of 9690.” As she said, “It makes me feel that I did something wrong.” Worse, perhaps, they may concoct an inappropriate explanation for what they believe they have observed. L2 got an alert box message: “Before you type make a selection.” The alert box contained a button labelled Cancel. L2 reasoned that this was the selection referred to, and he clicked the Cancel. “I guess they were trying to save me from an error I was about to make; by typing Cancel I averted it.” Cancel was not the selection referred to in the alert box, and his Cancel action did nothing more than dismiss the alert box.

Learners had understandable difficulty in trying to reason out jargon terms (like tool and profile) and to resolve fine distinctions in terminology (Save and Put Away versus Set Aside), in objects (icon ghost versus icon versus window), and in actions (press versus click versus double click). Experimenting with these distinctions can be costly and time consuming, especially when an incorrect alternative (clicking on an icon ghost) evokes no system response or when Undo is unavailable for escape from the consequences of errors.

These troubles must be taken in context, of course; the training approach of Lisa was clearly at or beyond the general state-of-the-art. Nevertheless, alternatives should be considered. LisaGuide was in many ways an awkward compromise between training books and interactive training. The tutorial relied too much on reading from the screen and carrying out seemingly pointless practice exercises for demeaningly cute verbal rewards. Oddly, from this perspective, LisaGuide and the owner’s guide (the actual training manual) were not coordinated. They were two independent, but quite redundant, training approaches.

Lisa’s error shielding (given the potentially distracting and frustrating consequences of errors) was probably a good idea. But one could argue that the straight tutorial approach of LisaGuide was too rigid a vehicle for error-shielded training. One small error can spoil an entire effort. Conversely, when LisaGuide did not shield errors, it often provided too little feedback for successful error diagnosis and recovery.

We have recently studied a “training wheels” approach, in which error shielding was employed in a scaled down, but functional, system. In a training wheels system, new users can take the initiative and learn by doing (which they cannot do with LisaGuide) with the benefit of error shielding.2

Design interface metaphors for mismatches as well as for matches

A concrete metaphor, such as the desktop, can make a system interface easier to learn (or even “comfortable”). At Yorktown, some of our work has been directed at understanding how interface metaphors can impact learning.13,17,18 The upshot of this work, however, is not as simple as suggested in the Lisa literature. L1, for example, did not seem comforted by the familiar term “tool” when he said, “I thought a tool was a floppy.” We have seen many examples (in the present work and in our prior work) of metaphor obstacles to learning (such as the Lisa typing page, which worked very differently from a piece of paper).

We have concluded that metaphors act as conceptual aids as much because they mismatch their targets as because they match. Pressing character keys elicits glowing dots on a TV screen rather than lines of ink on a paper; these are really very different effects. And typing over characters on the screen replaces the prior characters or inserts the new characters, although both outcomes are unpredictable on the basis of literal metaphor projection. Indeed, given a simple view of metaphor, it is remarkable that neither of these metaphor misfits has troubling consequences for learners. In fact, encountering these misfits can afford a concrete opportunity for developing an enhanced understanding of the electronic medium (such as the concept of dynamic storage).

However, for a metaphor mismatch to be an efficacious learning tool, it is critical that it occur in the context of obvious consistency. Learners want and need to solve problems in order to learn actively, but to do this they need to be able to discern the problem. There must be a background of predictability for them to pick up what is novel. This accounts for some of the trouble with LisaProject we observed: The presentation of the application did not make obvious enough what was what, nor did it successfully draw upon what our learners already knew about project planning. Hence, it gave them an inadequate basis from which to proceed.17

The desktop and typing page in Lisa were better examples. As we described above, in these cases the metaphor was obvious; predictability was high, and some mismatches with the metaphor were detected and used to sort out problems during learning. Other mismatches caused problems, however. The notion of stationery pads for folders and the requirement to Tear Off Stationery prior to data entry were mismatches in the desktop metaphor that caused problems, as did invisible format characters and the insert mode in the typing page metaphor. The Lisa literature stressed the deliberate design of the metaphor, but did not indicate how the Lisa designers approached metaphor mismatches in the interface despite their importance.

What can we learn from Lisa?

Lisa’s user interface design was a milestone in human-computer interaction. Indeed, we may only have begun to appreciate the consequences of Lisa’s user interface for design. Nevertheless, present research should make plain the fact that Lisa was not a panacea for user interface design problems. The challenge is to develop a serious understanding of the usability problems and strengths of this new approach, an understanding that can guide the design of even better user interfaces. As our study makes clear, this is not likely to be a simple endeavor. User interface issues are far more subtle than the black-and-white of popular understanding.

We have stressed troublesome aspects of the Lisa interface, in part because other discussions of Lisa focused so exclusively on the many positive aspects of its interface. But our concern is not with merely being balanced. Lisa can have two distinct influences on interface design. First, it can entrain simple “Lisa-mimic” designs, which will duplicate both the advantages and the problems of Lisa (and add little to the state of the art). But second, it can serve as a design lesson, a large-scale interface prototyping experience for “Lisa-based” designs, which will profit from both the advantages and the problems of Lisa. We would like to facilitate the second possibility.

John M. Carroll and Sandra A. Mazur
IBM T. J. Watson Research Center

Acknowledgments

We are grateful to Robert Gordon, Don Hatfield, Clayton Lewis, Mark Martin, and Mary Beth Rosson, and to the reviewers for comments on an earlier version of this paper.

References

1. J. M. Carroll, “The Adventure of Getting to Know a Computer,” Computer, 1982, pp. 49-58.

2. J. M. Carroll and C. Carrithers, “Training Wheels in a User Interface,” CACM, 1984, pp. 800-806.

3. J. M. Carroll et al., “The Minimal Manual,” IBM research report 11637, 1986.

4. J. M. Carroll et al., “Exploring a Word Processor,” Human-Computer Interaction, 1985, pp. 283-307.

5. R. L. Mack, C. H. Lewis, and J. M. Carroll, “Learning to Use Word Processors: Problems and Prospects,” ACM Trans. Office Info. Systems, Vol. 1, No. 3,1983, pp. 254-271.

6. C. Chin, “Lisa Software,” InfoWorld, Jan. 16, 1984, p. 27.

7. E. L. Hutchins, J. D. Hollan, and D. A. Norman, “Direct Manipulation Interfaces,” User Centered System Design: New Perspectives in Human-Computer Interaction, ed. D. A. Norman and S. W. Draper, Hillsdale, NJ, Lawrence Erlbaum Assoc, 1986.

8. B. Shneiderman, “The Future of Interactive Systems and the Emergence of Direct Manipulation,” Behavior and Information Technology, 1982, pp. 237-256.

9. B. Scheiderman, “Direct Manipulation: A Step Beyond Programming Languages,” Computer, Vol. 16, No. 8, pp. 57-69.

10. M. Merkin, “In Love with Lisa,” Creative Computing, Oct. 1983, pp. 12-17.

11. J. L. Ehardt, Apple’s Lisa: A Personal Office System, The Seybold Report on Professional Computing, 1983, pp. 1-26.

12. C. Morgan, G. Williams, and P. Lemmons, An Interview with Wayne Rosing, Bruce Daniels, and Larry Tesler, Byte, Feb. 1983, pp. 90-114.

13. J. M. Carroll and J. C. Thomas, “Metaphor and the Cognitive Representation of Computing Systems,” Trans. Systems, Man, and Cybernetics, Vol. SMC-12, 1982, pp. 107-116.

14. G. Williams, The Lisa Computer System, Byte, Feb. 1983, pp. 33-50.

15. J. M. Carroll and M. B. Rosson, “Usability Specifications as a Tool in Iterative Development,” Advances in Human-Computer Interaction, ed. H. R. Hartson, Norwood, NJ, Ablex, 1985.

16. J. M. Carroll and R. L. Mack, “Learning to Use a Word Processor: By Doing, By Thinking, and By Knowing,” Human Factors of Computing Systems, ed. J. C. Thomas and M. Schneider, Norwood, NJ, Ablex, 1984.

17. J. M. Carroll and R. L. Mack, “Metaphor, Computing Systems, and Active Learning,” Int’l J. Man-Machine Studies, 1985, pp. 39-57.

18. R. L. Mack, “Understanding Text-Editing: Evidence from Predictions and Descriptions Given by Computer-Naive People,” IBM research report RC 10333, 1984.

About the authors

John M. Carroll
This image can be zoomed
John M. Carroll is Manager of Advisory Interfaces at the IBM T. J. Watson Research Center. His research includes the analysis of learning, problem solving, and language capacities that underlie human endeavor and experience. He is on the editorial boards of the Int’l J. of Man-Machine Studies and Behavior and Information Technology.

Carroll holds BA degrees in mathematics and information science from Lehigh University (1972) and a PhD in experimental psychology from Columbia University (1976).


Sandra Mazur
This image can be zoomed
Sandra Mazur is at Micro Control Systems, where she writes user documentation and designs on-line training programs for a PC-based CAD system called CADKEY. Prior to this, she served as a visiting scientist within the Human Factors Group at the IBM T. J. Watson Research Center. Her research there focused on implementing training tools and observing the learning techniques of first-time users.

Mazur holds a BA in communications/speech and hearing sciences from Plattsburgh State University of New York and an MA in communications, computing, and instructional technology from Columbia University.

Readers may write to Carroll at the IBM T. J. Watson Research Center, H1-B16, PO Box 218, Yorktown Heights, NY 10598.

Page added on 22nd January 2005.

Copyright © 2002-2006 Marcin Wichary, unless stated otherwise.