The evaluation of text editors: methodology and empirical results.

ABSTRACT

This paper presents a methodology for evaluating text editors on several dimensions: the time it takes experts to perform basic editing tasks, the time experts spend making and correcting errors, the rate at which novices learn to perform basic editing tasks, and the functionality of editors over more complex tasks. Time, errors, and learning are measured experimentally; functionality is measured analytically; time is also calculated analytically. The methodology has thus far been used to evaluate nine diverse text editors, producing an initial database of performance results. The database is used to tell us not only about the editors but also about the users—the magnitude of individual differences and the factors affecting novice learning.

REFERENCES

• 1: Bates, S. J. User behavior in an interactive computer system. IBM Systems Journal 13 (1974) 1-18.
• 2: Card, S. K., English, W. K,, and Burr, B. J. Evaluation of mouse, ratecontrolled isometric joystick, step keys, and text keys for text selection on a CRT. Ergonomics 21 (1978) 601-613.
• 3: Stuart K. Card , Thomas P. Moran , Allen Newell, The keystroke-level model for user performance time with interactive systems, Communications of the ACM, v.23 n.7, p.396-410, July 1980 [doi>10.1145/358886.358895]
• 4: Stuart K. Card , Allen Newell , Thomas P. Moran, The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 1983
• 5: Dennis E. Egan , Cheryll Bowers , Louis M. Gomez, Learner characteristics that predict success in using a text-editor tutorial, Proceedings of the 1982 conference on Human factors in computing systems, p.337-340, March 15-17, 1982, Gaithersburg, Maryland, United States [doi>10.1145/800049.801806]
• 6: David W. Embley , George Nagy, Behavioral Aspects of Text Editors, ACM Computing Surveys (CSUR), v.13 n.1, p.33-70, March 1981 [doi>10.1145/356835.356838]
• 7: Michael Good, An ease of use evaluation of an integrated document processing system, Proceedings of the 1982 conference on Human factors in computing systems, p.142-147, March 15-17, 1982, Gaithersburg, Maryland, United States [doi>10.1145/800049.801771]
• 8: Norman Meyrowitz , Andries van Dam, Interactive Editing Systems: Part I, ACM Computing Surveys (CSUR), v.14 n.3, p.321-352, Sept. 1982 [doi>10.1145/356887.356889]
• 9: Moran, T. P. The Command Language Grammar: A representation for the user interface of interactive computer systems. Int. Journal of Man-Machine Studies 15, 1 (July 1981) 3-50.
• 10: Riddle, E. A. Comparative Study of Various Text Editors and Formatting Systems. Report AD-A029 050, Air Force Data Services Center, The Pentagon, Washington, D.C., (Aug. 1976).
• 11: Teresa Lynn Roberts, Evaluation of computer text editors, 1980
• 12: Teresa L. Roberts , Thomas P. Moran, Evaluation of text editors, Proceedings of the 1982 conference on Human factors in computing systems, p.136-141, March 15-17, 1982, Gaithersburg, Maryland, United States [doi>10.1145/800049.801770]
• 13: Robertson, C. K., and Akscyn, R. Experimental evaluation of tools for teaching the ZOG frame editor. Computer Science Department, Carnegie-Mellon University, Pittsburgh, PA, (1982).
• 14: The Seybold Report on Office Systems (through 1981 called The Seybold Report on Word Processing). Media, PA.
• 15: The Seybold Report on Word Processing. 4, 4, (April 1981). Issue on Personal Computers: Word Processing Packages.
• 16: Smith, D. C., Irby, C., Kimball, R., Verplank, W., and Harslem, E. Designing the Star user interface. Byte 7, 4 (April 1982) 242-282.
• 17: John Whiteside , Norman Archer , Dennis Wixon , Michael Good, How do people really use text editors?, Proceedings of the SIGOA conference on Office information systems, p.29-40, June 21-23, 1982, Philadelphia, Pennsylvannia, United States
• 18: Augmentation Research Center. NLS-8 Command Summary. Stanford Research Institute, Menlo Park, Calif., (May 1975).
• 19: Augmentation Research Center. NLS-8 Glossary. Stanford Research Institute. Menlo Park, Calif., (July 1975).
• 20: Bolt, Beranek, and Newman, Inc. TENEX Text Editor and Corrector (Manual DEC10-NGZEB-D). Cambridge, Mass., (1973). (Documents TECO.)
• 21: Garcia, K. Xerox Document System Reference Manual. Xerox Office Products Division, Palo Alto, Calif, (1980). (Documents BRAVOX.)
• 22: Lampson, B. Bravo manual. Alto User's Handbook. Xerox Palo Alto Research Center, Palo Alto, Calif., (1979).
• 23: Stallman, R. M. EMACS Manual for ITS Users. AI Lab Memo 554, MIT, Cambridge, Mass., (1980),
• 24: Stanford Center for Information Processing. Wylbur/370 The Stanford Timesharing System Reference Manual, 3rd ed. Stanford University, Stanford, Calif., (1975).
• 25: Tesler, L. The Smalltalk environment. Byte 6, 8, (Aug. 1981) 90-147. (There is no available GYPSY documentation. This paper describes the Smalltalk editor, which is based on many of the same design ideas as GYPSY.)
• 26: Wang Laboratories, Inc. Wang Word Processor Operator's Guide, 3rd release. Lowell, Mass., (1978).
• 27: Xerox Corporation. 8010 Star Information System Reference Guide. Dallas, Texas, (1981). (See also {16}.)

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

Additional Classification:
  I. Computing Methodologies
  I.7 DOCUMENT AND TEXT PROCESSING
      I.7.1 Document and Text Editing
          Subjects: Languages**
      I.7.2 Document Preparation
          Subjects: Languages and systems

General Terms:
Experimentation, Human Factors

Keywords:
ergonomics, human factors, human-computer interaction, human-computer interface, system design, system evaluation, text editing, user model, user performance, user psychology