When two methods are better than one: combining user study with cognitive modeling

ABSTRACT

We discuss the benefits of combining user studies and cognitive modeling in the context of Firefox tabbed browsing. We studied new users' ability to use tabbed browsing without assistance, and then evaluated alternatives for closing browser tabs to improve the new user experience through user tests and cognitive modeling. In general, our experience highlights the advantages of using user studies and modeling together to do user interface evaluation: user studies provided validation of design intuitions and data to support modeling of user behavior; modeling provided a fast and efficient ability to play "what if" with the design change; the combination of qualitative user test data and quantitative modeling results proved to be a far more convincing package of evidence than either result in isolation, given the variety of perspectives inthe design and development team.

REFERENCES

• 1: The CogTool Project, Tools for Cognitive Performance Modeling for Interactive Devices. http://www.cs.cmu.edu/~bej/cogtool/.
• 2: Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6), 381--391.
• 3: I. Scott MacKenzie , William Buxton, Extending Fitts' law to two-dimensional tasks, Proceedings of the SIGCHI conference on Human factors in computing systems, p.219-226, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142794]
• 4: Mozilla Firefox homepage: http://www.mozilla.com/firefox/.
• 5: Palmer, S. E. (1992). Common region: A new principle of perceptual grouping. Cognitive Psychology, 24, 436--447.
• 6: Salvucci, D. D. (2001). An integrated model of eye movements and visual encoding. Cognitive Systems Research, 1(4), 201--220.
• 7: Alonso Vera , Andrew Howes , Michael McCurdy , Richard L. Lewis, A constraint satisfaction approach to predicting skilled interactive cognition, Proceedings of the SIGCHI conference on Human factors in computing systems, p.121-128, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985708]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Theory and methods

General Terms:
Measurement

Keywords:
performance metrics, usability research, user and cognitive models