Usability testing vs. heuristic evaluation: was there a contest?

ABSTRACT

Recent research comparing usability assessment methods has been interpreted by some to imply that usability testing is no longer necessary, because other techniques, such as heuristic evaluation, can find some usability problems more cost-effectively. Such an interpretation grossly overstates the actual results of the studies. In this article, we, as authors of studies that compared inspection methods to usability testing, point out the rather severe limitations to using inspection methods as a substitute for usability testing and argue for a more balanced repertoire of usability assessment techniques.

REFERENCES

• 1: Heather Desurvire , Jim Kondziela , Michael E. Atwood, What is gained and lost when using methods other than empirical testing, Posters and short talks of the 1992 SIGCHI conference on Human factors in computing systems, May 03-07, 1992, Monterey, California [doi>10.1145/1125021.1125115]
• 2: Heather W. Desurvire , Jim M. Kondziela , Michael E. Atwood, What is gained and lost when using evaluation methods other than empirical testing, Proceedings of the conference on People and computers VII, p.89-102, January 1993, York, United Kingdom
• 3: Heather Desurvire , Debbie Lawrence , Michael Atwood, EMPIRICISM VERSUS JUDGEMENT: COMPARING USER INTERFACE EVALUATION METHODS ON A NEW TELEPHONE-BASED INTERFACE, ACM SIGCHI Bulletin, v.23 n.4, p.58-59, Oct. 1991 [doi>10.1145/126729.1056062]
• 4: Robin Jeffries , James R. Miller , Cathleen Wharton , Kathy Uyeda, User interface evaluation in the real world: a comparison of four techniques, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.119-124, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108862]
• 5: Claire-Marie Karat , Robert Campbell , Tarra Fiegel, Comparison of empirical testing and walkthrough methods in user interface evaluation, Proceedings of the SIGCHI conference on Human factors in computing systems, p.397-404, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142873]
• 6: Clayton Lewis , Peter G. Polson , Cathleen Wharton , John Rieman, Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, p.235-242, April 01-05, 1990, Seattle, Washington, United States [doi>10.1145/97243.97279]
• 7: Rolf Molich , Jakob Nielsen, Improving a human-computer dialogue, Communications of the ACM, v.33 n.3, p.338-348, March 1990 [doi>10.1145/77481.77486]
• 8: Jakob Nielsen, Finding usability problems through heuristic evaluation, Proceedings of the SIGCHI conference on Human factors in computing systems, p.373-380, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142834]
• 9: Jakob Nielsen, Usability engineering at a discount, Proceedings of the third international conference on human-computer interaction on Designing and using human-computer interfaces and knowledge based systems (2nd ed.), p.394-401, September 1989, Boston, Massachusetts, United States
• 10: Jakob Nielsen , Rolf Molich, Heuristic evaluation of user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, p.249-256, April 01-05, 1990, Seattle, Washington, United States [doi>10.1145/97243.97281]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)

General Terms:
Human Factors, Management, Performance