Learning to predict information needs: context-aware display as a cognitive aid and an assessment tool

ABSTRACT

We discuss the problem of assessing and aiding user performance in dynamic tasks that require rapid selection among multiple information sources. Motivated by research in human sequential learning, we develop a system that learns by observation to predict the information a user desires in different contexts. The model decides when the display should be updated, which is akin to the problem of scene segmentation, and then selects the situationally relevant information display. The model reduces the cognitive burden of selecting situation-relevant displays. We evaluate the system in a tank video game environment and find that the system boosts user performance. The fit of the model to user data provides a quantitative assessment of user behavior, which is useful in assessing individual differences and the progression from novice- to expert-level proficiency. We discuss the relative benefits of adopting a learning approach to predicting information preferences and possible avenues to reduce the negative consequences of automation.

REFERENCES

• 1: ACTON, W., JOHNSON, P., AND GOLDSMITH, T. Structural knowledge assessment: Comparison of referent structures. Journal of Educational Psychology 86 (1994), 303--311.
• 2: David W. Albrecht , Ingrid Zukerman , An E. Nicholson, Bayesian Models for Keyhole Plan Recognition in an Adventure Game, User Modeling and User-Adapted Interaction, v.8 n.1-2, p.5-47, 1998 [doi>10.1023/A:1008238218679]
• 3: Markus Bylund , Fredrik Espinoza, Using Quake III Arena to simulate sensors and actuators when evaluating and testing mobile services, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634210]
• 4: Keith Cheverst , Nigel Davies , Keith Mitchell , Adrian Friday , Christos Efstratiou, Developing a context-aware electronic tourist guide: some issues and experiences, Proceedings of the SIGCHI conference on Human factors in computing systems, p.17-24, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332047]
• 5: CHI, M., FELTOVICH, P., AND GLASER, R. Categorization and representation of physics problems by experts and novices. Cognitive Science 5 (1981), 121--152.
• 6: Axel Cleeremans, Mechanisms of implicit learning: connectionist models of sequence processing, MIT Press, Cambridge, MA, 1993
• 7: DAY, E., ARTHUR, W., AND GETTMAN, D. Knowledge structures and the acquisition of a complex skill. Journal of Applied Psychology 86 (2001), 1022--1033.
• 8: FOGARTY, J., HUDSON, S. E., AND LAI, J. Predictability and accuracy in adaptive user interfaces. In Proceeding of the SIGCHI Conference on Human Factors in Computing Systems (2004), 207--214.
• 9: Krzysztof Z. Gajos , Katherine Everitt , Desney S. Tan , Mary Czerwinski , Daniel S. Weld, Predictability and accuracy in adaptive user interfaces, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357252]
• 10: GOODMAN, B. A., AND LITMAN, D. J. On the interaction between plan recognition and intelligent interfaces. UMUAI 2 (1992), 83--115.
• 11: GRAY, W. Simulated task environments: The role of high-fidelity simulations, scaled worlds, synthetic environments, and laboratory tasks in basic and applied cognitive research. Cognitive Science Quarterly 2 (2002), 205--227.
• 12: GURECKIS, T., AND LOVE, B. C. Transformational vs. statistical complexity in sequential learning. Cognitive Science (in press).
• 13: HORVITZ, E., AND BARRY, M. Display of information for time-critical decision making. In Proc. of Conf. on Uncertainty in AI (1995), pp. 296--305.
• 14: Eric Horvitz , Paul Koch , Johnson Apacible, BusyBody: creating and fielding personalized models of the cost of interruption, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031690]
• 15: JONES, G. J. F., AND BROWN, P. J. Context-Aware Retrieval for Ubiquitous Computing Environments. Springer, 2004.
• 16: Ashish Kapoor , Eric Horvitz, Experience sampling for building predictive user models: a comparative study, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357159]
• 17: KLEIN, G. A. Recognition-primed decisions. In Advances in man-machine research, W. Rouse, Ed. JAI Press, Greenwich, 1989, pp. 47--92.
• 18: Matthias Klusch, Information agent technology for the Internet: a survey, Data & Knowledge Engineering, v.36 n.3, p.337-372, March 2001 [doi>10.1016/S0169-023X(00)00049-5]
• 19: KOHAVI, R. A study of cross-validation and bootstrap for accuracy estimation and model selection. Proceedings of the Fourteenth International Joint Conference on Artificial lntelligence (1995), 1137--1143.
• 20: KRAIGER, K., AND FORD, J.K. ANDSALAS, E. Application of cognitive, skill--based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology 78 (1993), 311--328.
• 21: KURBY, C. A., AND ZACKS, J. M. Segmentation in the perception and memory of events. Trends in Cognitive Sciences 12 (2008), 72--79.
• 22: Jani Mäntyjärvi , Tapio Seppänen, Adapting Applications in Mobile Terminals Using Fuzzy Context Information, Proceedings of the 4th International Symposium on Mobile Human-Computer Interaction, p.95-107, September 18-20, 2002
• 23: MCRAE, K., SPIVEY-KNOWLTON, M. J., AND TANENHAUS, M. K. Modeling the influence of thematic fit (and other constraints) in on-line sentence comprehension. Journal of Memory and Language 38 (1998), 283--312.
• 24: MCTEAR, M. F. User modeling for adaptive computer systems: a survey of recent developments. Artificial Intelligence Review 7 (1993), 157--184.
• 25: MILLER, C., FUNK, H., GOLDMAN, R., MEISNER, J., AND WU, P. Implications of adaptive vs. adaptable UIs on decision making: Why automated adaptiveness is not always the right answer. In Proc. of the 1st Inter. Conf. on Augmented Cognition (2005).
• 26: NEWTON, D. Foundations of attribution: The perception of ongoing behavior. In New Directions in Attribution Research, J. Harvey, Ed. Erlbaum, Mahwah, NJ, 1976.
• 27: RADVANSKY, G. A., AND COPELAND, D. E. Walking through doorways causes forgetting: Situation models and experienced space. Memory&Cognition 34 (2006), 1150--1156.
• 28: RINCK, M., AND WEBER, U. Who when where: An experimental test of the event-indexing model. Memory&Cognition 31 (2003), 1284--1292.
• 29: SAFFRAN, J. R. Statistical language learning: Mechanisms and constraints. Current Directions in Psychological Science 12 (2003), 110--114.
• 30: SARTER, N., MUMAW, R., AND WICKENS, C. Pilots monitoring strategies and performance on automated flight decks: An empirical study combining behavioral and eye-tracking data. Human Factors 49 (2007), 347--357.
• 31: Matthias Schneider-Hufschmidt , Uwe Malinowski , Thomas Kuhme, Adaptive User Interfaces: Principles and Practice, Elsevier Science Inc., New York, NY, 1993
• 32: SHARP, R. Electrophysiological correlates of event segmentation: how does the human mind process ongoing activity. Proceedings of the Annual Meeting of the Cognitive Neuroscience Society 235 (2007).
• 33: SPEER, N., SWALLOW, K., AND ZACKS, K. Activation of human motion processing areas during event perception. Cognitive, Affective&Behavioral Neuroscience 3 (2003), 335--345.
• 34: ST. JOHN, M., SMALLMAN, H. S., AND MANES, D. I. Assisted focus: Heuristic automation for guiding users' attention toward critical information. In Proc. of the 1st Inter. Conf. on Augmented Cognition (2005).
• 35: SWALLOW, K. Perceptual events may be the episodes in episodic memory. Abstr. Psychon. 12 (2007), 25.
• 36: SWALLOW, K., AND ZACKS, J. Hierarchical grouping of events revealed by eye movements. Abstr. Psychon. 9 (2004), 81.
• 37: TANAKA, J. W., AND TAYLOR, M. Object categories and expertise: Is the basic level in the eye of the beholder. Cognitive Psychology 23, 2 (1991), 457--482.
• 38: Jaime Teevan , Susan T. Dumais , Eric Horvitz, Personalizing search via automated analysis of interests and activities, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076111]
• 39: WICKENS, C. D., AND DIXON, S. R. The benefits of imperfect diagnostic automation: A synthesis of the literature. Theoretical Issues in Ergonomic Science 8 (2007), 201--212.
• 40: Weilie Yi , Dana H. Ballard, Behavior Recognition in Human Object Interactions with a Task Model, Proceedings of the IEEE International Conference on Video and Signal Based Surveillance, p.64, November 22-24, 2006 [doi>10.1109/AVSS.2006.28]
• 41: ZACKS, J., SPEER, N., VETTEL, J., AND JACOBY, L. Event understanding and memory in healthy aging and dementia of the alzheimer type. Psychology&Aging 21 (2008), 466--482.
• 42: ZWAAN, R., RADVANSKY, G., HILLIARD, A., AND CURIEL, J. Constructing multidimensional situation models during reading. Scientific Studies of Reading 2 (1998), 199--220.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

General Terms:
Algorithms, Experimentation, Human Factors, Theory

Keywords:
adaptive display, context-aware computing, event segmentation