Linking surface error characteristics to root problems in user-based evaluation studies

ABSTRACT

This paper reports a study of novice subjects using Word 5.1. for the Macintosh. In particular, user errors were analysed. The intention was to investigate the difficulty, that evaluators have in determining root error causes from surface characteristics. Errors made by subjects were examined and classified in phenotype and genotype categories[3]. An error was classified in a genotype category if it was felt that system had failed to support that particular user mental action. The utility of the analysis that was used for analysing usability errors is then discussed.

REFERENCES

• 1: Carroll, J. M. Koenemann-Belliveau, J. Rosson, M. and Singley, M. (1993), 'Critical Incidents and Critical Threads in Usability Evaluation', in: Proc. People and Computers viii, J. Alty et al (eds.), Cambridge University Press, 279--292
• 2: Ericsson, K. A. Simon, H. A. (1984), 'Protocol Analysis', MIT press
• 3: Hollnagel, E. (1993), 'Human Reliability Analysis: Context and Control', Academic Press
• 4: Hutchins, E. L. Hollan, J. D. Norman, D. A. (1986), 'Direct Manipulation Interfaces'. in: User Centred System Design New Perspectives on Human-Computer Interaction, D. Norman & S. Draper (eds.) Lawrence Erlbaum Associates, New Jersey, 31--62
• 5: Kieras, D. E. and Polson, P. A. (1985), 'An Approach to the Formal Analysis of User Complexity', International Journal of Man Machine Studies, 22, 365--395.
• 6: Lewis, C. (1988), 'Why and How to Learn Why: Analysis-Based Generalisation of Procedures', Cognitive Science, 12, 211--256
• 7: A Monk, Mode errors: a user-centered analysis and some preventative measures using keying-contingent sound, International Journal of Man-Machine Studies, v.24 n.4, p.313-327, April 1986 [doi>10.1016/S0020-7373(86)80049-9]
• 8: Jakob Nielsen , Robert L. Mack, Usability inspection methods, John Wiley & Sons, Inc., New York, NY, 1994
• 9: Norman, D. A. (1988), 'The Psychology of Everyday Things' New York: Basic Books
• 10: Stephen J. Payne, Interface problems and interface resources, Designing interaction: psychology at the human-computer interface, Cambridge University Press, New York, NY, 1991
• 11: James Reason, Framework models of human performance and error: a consumer guide, Tasks, errors, and mental models, Taylor & Francis, Inc., Bristol, PA, 1988
• 12: Antonio Rizzo , Enrica Marchigiani , Alessandro Andreadis, The AVANTI project: prototyping and evaluation with a cognitive walkthrough based on the Norman's model of action, Proceedings of the conference on Designing interactive systems: processes, practices, methods, and techniques, p.305-309, August 18-20, 1997, Amsterdam, The Netherlands [doi>10.1145/263552.263629]
• 13: Ben Shneiderman, Designing the user interface (videotape), University of Maryland at College Park, College Park, MD, 1987
• 14: Mark Springett, The Utility of User Action Models for Direct Manipulation Design, Proceedings of the IFIP TC2/WG2.7 Working Conference on Engineering for Human-Computer Interaction, p.205-223, August 10-12, 1992
• 15: Springett, M. V. (1996), 'User Modelling for Evaluation of Direct Manipulation Interfaces', PhD thesis, City University, UK
• 16: Sutcliffe A. G. (1988), 'Human Computer Interface Design', Macmillan, London
• 17: A. G. Sutcliffe , M. V. Springett, From user's problems to design errors: linking evaluation to improving design practice, Proceedings of the conference on People and computers VII, p.117-134, January 1993, York, United Kingdom
• 18: Peter C. Wright , Andrew F. Monk, A cost-effective evaluation method for use by designers, International Journal of Man-Machine Studies, v.35 n.6, p.891-912, Dec. 1991 [doi>10.1016/S0020-7373(05)80167-1]

INDEX TERMS

Keywords:
action cycle, error studies, phenotypes/genotypes