Finding usability problems through heuristic evaluation

ABSTRACT

Usability specialists were better than non-specialists at performing heuristic evaluation, and “double experts” with specific expertise in the kind of interface being evaluated performed even better. Major usability problems have a higher probability than minor problems of being found in a heuristic evaluation, but more minor problems are found in absolute numbers. Usability heuristics relating to exits and user errors were more difficult to apply than the rest, and additional measures should be taken to find problems relating to these heuristics. Usability problems that relate to missing interface elements that ought to be introduced were more difficult to find by heuristic evaluation in interfaces implemented as paper prototypes but were as easy as other problems to find in running systems.

REFERENCES

• 1: IEEE Software Staff, Interface-Walkthroughs: Efficient Collaborative Testing, IEEE Software, v.8 n.5, p.94-95, September 1991 [doi>10.1109/52.84220]
• 2: John M. Carroll, Infinite detail and emulation in an ontologically minimized HCI, Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, p.321-328, April 01-05, 1990, Seattle, Washington, United States [doi>10.1145/97243.97303]
• 3: John M. Carroll , Wendy A. Kellogg , Mary Beth Rosson, The task-artifact cycle, Designing interaction: psychology at the human-computer interface, Cambridge University Press, New York, NY, 1991
• 4: Heather Desurvire , Debbie Lawrence , Michael Atwood, EMPIRICISM VERSUS JUDGEMENT: COMPARING USER INTERFACE EVALUATION METHODS ON A NEW TELEPHONE-BASED INTERFACE, ACM SIGCHI Bulletin, v.23 n.4, p.58-59, Oct. 1991 [doi>10.1145/126729.1056062]
• 5: Engelbeck, G., and Roberts, T.L. The effect of several voicemenu characteristics on menu selection performance. Behaviour & Information Technology in press.
• 6: Gould, J.D., and Boles, S.J. Speech filingmAn office system for principals. IBM Systems Journal 23, I (1984), 65-81.
• 7: R. Halstead-Nussloch, The design of phone-based interfaces for consumers, Proceedings of the SIGCHI conference on Human factors in computing systems: Wings for the mind, p.347-352, March 1989
• 8: Robin Jeffries , James R. Miller , Cathleen Wharton , Kathy Uyeda, User interface evaluation in the real world: a comparison of four techniques, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.119-124, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108862]
• 9: Claire-Marie Karat , Robert Campbell , Tarra Fiegel, Comparison of empirical testing and walkthrough methods in user interface evaluation, Proceedings of the SIGCHI conference on Human factors in computing systems, p.397-404, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142873]
• 10: Wendy A. Kellogg, Qualitative artifact analysis, Proceedings of the IFIP TC13 Third Interational Conference on Human-Computer Interaction, p.193-198, August 27-31, 1990
• 11: Clayton Lewis , Peter G. Polson , Cathleen Wharton , John Rieman, Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, p.235-242, April 01-05, 1990, Seattle, Washington, United States [doi>10.1145/97243.97279]
• 12: Rolf Molich , Jakob Nielsen, Improving a human-computer dialogue, Communications of the ACM, v.33 n.3, p.338-348, March 1990 [doi>10.1145/77481.77486]
• 13: Jakob Nielsen, Usability engineering at a discount, Proceedings of the third international conference on human-computer interaction on Designing and using human-computer interfaces and knowledge based systems (2nd ed.), p.394-401, September 1989, Boston, Massachusetts, United States
• 14: Jakob Nielsen, Paper versus computer implementations as mockup scenarios for heuristic evaluation, Proceedings of the IFIP TC13 Third Interational Conference on Human-Computer Interaction, p.315-320, August 27-31, 1990
• 15: Nielsen, J. Applying heuristic evaluation to a highly domainspecific interface. Manuscript submitted for publication.
• 16: Jakob Nielsen, Usability Engineering, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1993
• 17: Jakob Nielsen , Rolf Molich, Heuristic evaluation of user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, p.249-256, April 01-05, 1990, Seattle, Washington, United States [doi>10.1145/97243.97281]
• 18: Richard C. Teitelbaum , Richard E. Granda, The effects of positional constancy on searching menus for information, Proceedings of the SIGCHI conference on Human Factors in Computing Systems, p.150-153, December 12-15, 1983, Boston, Massachusetts, United States [doi>10.1145/800045.801600]
• 19: Cathleen Wharton , Janice Bradford , Robin Jeffries , Marita Franzke, Applying cognitive walkthroughs to more complex user interfaces: experiences, issues, and recommendations, Proceedings of the SIGCHI conference on Human factors in computing systems, p.381-388, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142864]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation); Input devices and strategies (e.g., mouse, touchscreen)

General Terms:
Design, Performance

Keywords:
discount usability engineering, heuristic evaluation, interface evaluation, telephone-operated interfaces, usability expertise, usability problems