Model-based evaluation of expert cell phone menu interaction

ABSTRACT

We describe concepts to support the analysis of cell phone menu hierarchies, based on cognitive models of users and easy-to-use optimization techniques. We present an empirical study of user performance on five simple tasks of menu traversal on an example cell phone. Two of the models applied to these tasks, based on GOMS and ACT-R, give good predictions of behavior. We use the empirically supported models to create an effective evaluation and improvement process for menu hierarchies. Our work makes three main contributions: a novel and timely study of a new, very common HCI task; new versions of existing models for accurately predicting performance; and a search procedure to generate menu hierarchies that reduce traversal time, in simulation studies, by about a third.

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., and Quin, Y. 2004. An integrated theory of the mind. Psych. Rev. 111, 4, 1036--1060.
	2	Byrne, M. D. 2001. ACT-R/PM and menu selection: Applying a cognitive architecture to HCI. Int. J. Human-Comput. Stud. 55, 1, 41--84.
	3	DiPrima, D. 2006. Cellular market rising above 2.14 billion subscribers. telecombeat.com (http://www.telecombeat.com/content/view/4689).
	4	Fitts, P. M. 1954. The information capacity of the human motor system in controlling the amplitude of movement. J. Exper. Psych. 47, 381--391.
	5	Gray, W. D., John, B. E., and Atwood, M. E. 1993. Project Ernestine: Validating a GOMS analysis for predicting and explaining real-world task performance. Human-Comput. Interac. 8, 3, 237--309.
	6	Richard W Hamming, Numerical methods for scientists and engineers (2nd ed.), Dover Publications, Inc., New York, NY, 1986
	7	Anthony J. Hornof , David E. Kieras, Cognitive modeling reveals menu search in both random and systematic, Proceedings of the SIGCHI conference on Human factors in computing systems, p.107-114, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258621]
	8	John, B. E. 2003. Information processing and skilled behavior. In HCI Models, Theories, and Frameworks, J. M. Carroll, Ed. Morgan Kaufman, San Francisco, CA.
	9	Bonnie E. John , David E. Kieras, Using GOMS for user interface design and evaluation: which technique?, ACM Transactions on Computer-Human Interaction (TOCHI), v.3 n.4, p.287-319, Dec. 1996 [doi>10.1145/235833.236050]
	10	Bonnie E. John , David E. Kieras, The GOMS family of user interface analysis techniques: comparison and contrast, ACM Transactions on Computer-Human Interaction (TOCHI), v.3 n.4, p.320-351, Dec. 1996 [doi>10.1145/235833.236054]
	11	Bonnie E. John , Konstantine Prevas , Dario D. Salvucci , Ken Koedinger, Predictive human performance modeling made easy, Proceedings of the SIGCHI conference on Human factors in computing systems, p.455-462, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985750]
	12	Kieras, D. 1999. A Guide to GOMS Model Usability Evaluation using GOMSL and GLEAN3. University of Michigan, Ann Arbor, MI.
	13	David Kieras, Model-based evaluation, The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 2002
	14	Kieras, D. and Meyer, D. E. 1997. An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Hum.-Comput. Interact. 12, 4, 391--438.
	15	MacKenzie, I. S. 2003. Motor behavior models for human-computer interaction. In HCI Models, Theories, and Frameworks, J. M. Carroll, Ed. Morgan Kaufman, San Francisco, CA.
	16	H. Albert Napier , Richard R. Batsell , David M. Lane , Norman S. Guadagno, Knowledge of command usage in a spreadsheet program, ACM SIGMIS Database, v.23 n.1, p.13-21, Winter 1992 [doi>10.1145/134347.134353]
	17	Allen Newell, Unified theories of cognition, Harvard University Press, Cambridge, MA, 1990
	18	Sarah Nichols , Frank E. Ritter, A theoretically motivated tool for automatically generating command aliases, Proceedings of the SIGCHI conference on Human factors in computing systems, p.393-400, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223955]
	19	Peter Pirolli, Computational models of information scent-following in a very large browsable text collection, Proceedings of the SIGCHI conference on Human factors in computing systems, p.3-10, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258558]
	20	Pirolli, P. 2003. Exploring and finding information. In HCI Models, Theories, and Frameworks, J. M. Carroll, Ed. Morgan Kaufman, San Francisco, CA.
	21	Pogue, D. 2005. The cellphone that does everything imaginable, at least sort of. The New York Times, May 12, 2005.
	22	Ritter, F. E. 1991. Towards fair comparisons of connectionist algorithms through automatically generated parameter sets. In Proceedings of the 13th Annual Conference of the Cognitive Science Society. 877-881.
	23	Frank E. Ritter , Andrew R. Freed , Onida L. M. Haskett, Discovering user information needs: the case of university department web sites, interactions, v.12 n.5, September + October 2005 [doi>10.1145/1082369.1082385]
	24	Ritter, F. E., Haynes, S. R., Cohen, M., Howes, A., John, B., Best, B., Lebiere, C., Jones, R. M., Crossman, J., Lewis, R. L., St. Amant, R., McBride, S. P., Urbas, L., Leuchter, S., and Vera, A. 2006. High-level behavior representation languages revisited. In Proceedings of ICCM---2006---Seventh International Conference on Cognitive Modeling. D. Fum, F. del Missier and A. Stocco, Eds. Edizioni Goliardiche, Trieste, Italy, 404--407.
	25	Ritter, F. E. and Young, R. M. 2001. Embodied models as simulated users: Introduction to this special issue on using cognitive models to improve interface design. Int. J. Hum.-Comput. Stud. 55, 1, 1--14.
	26	St. Amant, R., Freed, A., and Ritter, F. E. 2004a. Specifying ACT-R models of user interaction with a GOMS language. Cog. Syst. Res. 6, 1, 71--88.
	27	Robert St. Amant , Thomas E. Horton , Frank E. Ritter, Model-based evaluation of cell phone menu interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.343-350, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985736]
	28	Salvucci, D. 2001. Predicting the effects of in-car interface use on driver performance: An integrated model approach. Int. J. Hum.-Comput. Stud. 55, 1, 85--107.
	29	Dario D. Salvucci , Frank J. Lee, Simple cognitive modeling in a complex cognitive architecture, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642658]
	30	Miika Silfverberg , I. Scott MacKenzie , Panu Korhonen, Predicting text entry speed on mobile phones, Proceedings of the SIGCHI conference on Human factors in computing systems, p.9-16, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332044]
	31	Young, R. M. 1998. Rational analysis of exploratory choice. In Rational models of cognition, M. Oaksford & N. Chater, Eds. Oxford University Press, Oxford, UK, 469--500.