Learning and performing by exploration: label quality measured by latent semantic analysis

ABSTRACT

Models of learning and performing by exploration assume that the semantic similarity between task descriptions and labels on display objects (e.g., menus, tool bars) controls in part the users search strategies. Nevertheless, none of the models has an objective way to compute semantic similarity. In this study, Latent Semantic Analysis (LSA) was used to compute semantic similarity between task descriptions and labels in an applications menu system. Participants performed twelve tasks by exploration and they were tested for recall after a l-week delay. When the labels in the menu system were semantically similar to the task descriptions, subjects performed the tasks faster. LSA could be incorporated into any of the current models, and it could be used to automate the evaluation of computer applications for ease of learning and performing by exploration.

REFERENCES

• 1: John Rieman , Susan Davies , D. Charles Hair , Mary Esemplare , Peter Polson , Clayton Lewis, An automated cognitive walkthrough, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.427-428, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108986]
• 2: Peter G. Polson , Clayton Lewis , John Rieman , Cathleen Wharton, Cognitive walkthroughs: a method for theory-based evaluation of user interfaces, International Journal of Man-Machine Studies, v.36 n.5, p.741-773, May 1992 [doi>10.1016/0020-7373(92)90039-N]
• 3: Marita Franzke, Turning research into practice: characteristics of display-based interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.421-428, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223961]
• 4: Landauer, T.K., Foltz, P., and Laham, D. (1998). An Introduction to Latent Semantic Analysis. .Discourse Processes, 24, 259-284.
• 5: Poison, P.G. and Lewis, C.H. (1990). Theory-based design for easily learned interfaces. Human-Computer Interaction, 5(2-3), 191-220.
• 6: Allen Newell, Human Problem Solving, Prentice-Hall, Inc., Upper Saddle River, NJ, 1972
• 7: Engelbeck, G.E. (1986). Exceptions to generalizations: implications for formal models of human-computer interaction. Unpublished masters thesis, University of Colorado, Boulder, CO.
• 8: Muncher, E. (1989). The acquisition of spreadsheet skills. Unpublished masters thesis, University of Colorado, Boulder, CO.
• 9: Muneo Kitajima , Peter G. Polson, LICAI+: a comprehension-based model of learning for display-based human-computer interaction, CHI '97 extended abstracts on Human factors in computing systems: looking to the future, March 22-27, 1997, Atlanta, Georgia [doi>10.1145/1120212.1120420]
• 10: T. K. Landauer , K. M. Galotti , S. Hartwell, Natural command names and initial learning: a study of text-editing terms, Communications of the ACM, v.26 n.7, p.495-503, July 1983 [doi>10.1145/358150.358157]
• 11: Howes, A. and Young, R.M. (1996). Learning consistent, interactive and meaningful device methods" A computational model. Cognitive Science, 20, 30 }{..356.
• 12: John Rieman , Richard M. Young , Andrew Howes, A dual-space model of iteratively deepening exploratory learning, International Journal of Human-Computer Studies, v.44 n.6, p.743-775, June 1996 [doi>10.1006/ijhc.1996.0032]
• 13: John Franklin Rieman, Learning strategies and exploratory behavior of interactive computer users, University of Colorado at Boulder, Boulder, CO, 1994
• 14: Kitajima, M. and Poison, P.G. (1997). A Comprehension-Based Model of Exploration. Human- Computer Interaction, 12, 439-462.
• 15: Kitajima, M., Soto, R., and Poison, P.G. (1998). LICAI+: A Comprehension-Based Model of The: Recall of Action Sequences. in F. Ritter and R.M. Young (Eds.), Proceedings of the Second European Conference on Cognitive Modelling (Nottingham, April 1-4, 1998) (pp. 82-89). Nottingham, UK: Nottingham University Press.
• 16: Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York, NY: Cambridge University Press.
• 17: Mannes, S.M. and Kintsch, W. (1991). Routine Computing Tasks: Planning as Understanding. Cognitive Science, 15, 305-342.
• 18: Deerwester, S., et al. (1990). Indexing by Latent Semantic Analysis. Journal of the American Society For Information Science, 41(6), 391-407.
• 19: Landauer, T.K. and Dumais, S.T. (1997). A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2), 211-240.
• 20: Landauer, T.K. and Dumais, S.T. (1996). How come you know so much? From practical problem to theory. In D. Hermann, et al. (Eds.), Basic and applied memory: Memory in context (pp. 105-126). Mahwah, NJ: Erlbaum.
• 21: Landauer, T.K. and Dumais, S.T. (1994). Latent semantic analysis and the measurement of knowledge. In R.M. Kaplan and J.C. Burstein (Eds.), Educational testing service conference on natural language processing techniques and technology in assessment and education . Princeton, N.J." Educational Testing Service.
• 22: Ericsson, A.K. and Simon, H.A. (1980). Verbal Reports as Data. Psychological Review, 87(3), 215-251.
• 23: Stephen J. Payne, Display-based action at the user interface, International Journal of Man-Machine Studies, v.35 n.3, p.275-289, Sept. 1991 [doi>10.1016/S0020-7373(05)80129-4]
• 24: Nielsen, J. (1992). Applying Heuristic Evaluation to a Highly Domain-Specific User Interface. Technical memorandum. Morristown, NJ: Bellcore.
• 25: Soto, R. (1998). Learning and Performing by Exploration: Label Quality Measured by Latent Semantic Analysis. Unpublished master thesis, University of Colorado, Boulder, CO.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE

General Terms:
Design, Performance, Theory

Keywords:
cognitive models, label-following strategy, latent semantic analysis, learning by exploration, semantic similarity, usability analysis