Interaction in 4-second bursts: the fragmented nature of attentional resources in mobile HCI

ABSTRACT

When on the move, cognitive resources are reserved partly for passively monitoring and reacting to contexts and events, and partly for actively constructing them. The Re-source Competition Framework (RCF), building on the Multiple Resources Theory, explains how psychosocial tasks typical of mobile situations compete for cognitive resources and then suggests that this leads to the depletion of resources for task interaction and eventually results in the breakdown of fluent interaction. RCF predictions were tested in a semi-naturalistic field study measuring attention during the performance of assigned Web search tasks on mobile phone while moving through nine varied but typical urban situations. Notably, we discovered up to eight-fold differentials between micro-level measurements of atten-tional resource fragmentation, for example from spans of over 16 seconds in a laboratory condition dropping to bursts of just a few seconds in difficult mobile situations. By cali-brating perceptual sampling, reducing resources from tasks of secondary importance, and resisting the impulse to switch tasks before finalization, participants compensated for the resource depletion. The findings are compared to previous studies in office contexts. The work is valuable in many areas of HCI dealing with mobility.

REFERENCES

• 1: Baddeley, A. D. and Hitch, G. J. Working memory. In G. Bower (Ed.), The psychology of learning and motivation, Vol. VIII. Academic Press, NY, 1974, 47--89.
• 2: Liam Bannon , Allen Cypher , Steven Greenspan , Melissa L. Monty, Evaluation and analysis of users' activity organization, Proceedings of the SIGCHI conference on Human Factors in Computing Systems, p.54-57, December 12-15, 1983, Boston, Massachusetts, United States [doi>10.1145/800045.801580]
• 3: Victoria Bellotti , Sara Bly, Walking away from the desktop computer: distributed collaboration and mobility in a product design team, Proceedings of the 1996 ACM conference on Computer supported cooperative work, p.209-218, November 16-20, 1996, Boston, Massachusetts, United States [doi>10.1145/240080.240256]
• 4: Stuart K. Card , Allen Newell , Thomas P. Moran, The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 1983
• 5: Cook, T.D. and Campbell, D. Quasi-experimentation: Design and analysis issues for field settings. Rand McNally, 1979.
• 6: Cowan, N. The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24, 1 (2001), 87--185.
• 7: Mary Czerwinski , Eric Horvitz , Susan Wilhite, A diary study of task switching and interruptions, Proceedings of the SIGCHI conference on Human factors in computing systems, p.175-182, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985715]
• 8: Deci, E.L. and Ryan, R.M. Intrinsic motivation and self-determination in human behavior. Plenum Press, New York, 1985.
• 9: Where the action is: the foundations of embodied interaction, MIT Press, Cambridge, MA, 2001
• 10: Ericsson, K. A. and Kintsch, W. Long-term working memory. Psychological Review, 102 (1995), 211--245.
• 11: Victor M. González , Gloria Mark, "Constant, constant, multi-tasking craziness": managing multiple working spheres, Proceedings of the SIGCHI conference on Human factors in computing systems, p.113-120, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985707]
• 12: Goodwin, C. and Duranti, A. Rethinking context: an in-troduction. In A. Duranti & C. Goodwin (Eds.), Rethinking Context: Language as an Interactive Phenomenon. Cambridge Univ. Press, Cambridge, UK, 1992, 1--42.
• 13: Harris, J. and Wilkins, A.J. Remembering to do things: A theoretical framework and illustrative experiment. Human Learning, 1 (1982), 1--14.
• 14: Hutchins, E. Cognition in the Wild. MIT Press, Cronbridge, MA, 1995.
• 15: Kjeldskov J., Skov M. B., Als B. S. and Høegh R. T. Is it worth the hassle? Exploring the added value of evaluating the usability of context-aware mobile systems in the field. In Proc. Mobile HCI 2004, 61--73.
• 16: Steinar Kristoffersen , Fredrik Ljungberg, “Making place” to make IT work: empirical explorations of HCI for mobile CSCW, Proceedings of the international ACM SIGGROUP conference on Supporting group work, p.276-285, November 14-17, 1999, Phoenix, Arizona, United States [doi>10.1145/320297.320330]
• 17: Kulomäki J. and Summala H. Multitasking and Attentional Demands in Pedestrian Behaviour - A Review. Dep. of Psychology, Univ. of Helsinki. 30.6.2004.
• 18: Joanna Lumsden , Stephen Brewster, A paradigm shift: alternative interaction techniques for use with mobile & wearable devices, Proceedings of the 2003 conference of the Centre for Advanced Studies on Collaborative research, p.197-210, October 06-09, 2003, Toronto, Ontario, Canada
• 19: Navon, D. and Gopher, D. On the economy of the human-processing system. Psychological Review, 86 (1979), 214--255.
• 20: Antti Oulasvirta , Antti Salovaara, A cognitive meta-analysis of design approaches to interruptions in intelligent environments, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986012]
• 21: Oulasvirta, A. and Tamminen, S. Temporal tensions in human-computer interaction. CHI'04 Workshop on Temporal Aspects of Work, 2004.
• 22: Oulasvirta, A. Task-processing demands and memory in web interaction: A levels-of-processing approach. Interacting with Computers, 16, 2 (2004), 217--241.
• 23: Oulasvirta, A. Human-computer interaction in mobile context: a cognitive resources perspective. Licentiate Thesis, Cognitive Science, Department of Psychology, University of Helsinki, 2004. Available from http://ethesis.helsinki.fi.
• 24: Sharon Oviatt , Rachel Coulston , Rebecca Lunsford, When do we interact multimodally?: cognitive load and multimodal communication patterns, Proceedings of the 6th international conference on Multimodal interfaces, October 13-15, 2004, State College, PA, USA [doi>10.1145/1027933.1027957]
• 25: Jason Pascoe , Nick Ryan , David Morse, Using while moving: HCI issues in fieldwork environments, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.3, p.417-437, Sept. 2000 [doi>10.1145/355324.355329]
• 26: Harold Pashler, Dual-task interference and elementary mental mechanisms, Attention and performance XIV (silver jubilee volume): synergies in experimental psychology, artificial intelligence, and cognitive neuroscience, MIT Press, Cambridge, MA, 1993
• 27: Mark Perry , Kenton O'hara , Abigail Sellen , Barry Brown , Richard Harper, Dealing with mobility: understanding access anytime, anywhere, ACM Transactions on Computer-Human Interaction (TOCHI), v.8 n.4, p.323-347, December 2001 [doi>10.1145/504704.504707]
• 28: Rogers, Y. New Theoretical approaches for Human-Computer Interaction. Annual Review of Information, Science and Technology, 38 (2004), 87--143.
• 29: Roto, V., Oulasvirta, A., Haikarainen, T., Kuorelahti, J., Lehmuskallio, H. and Nyyssonen, T. Examining mobile phone use in the wild with quasi-experimentation. HIIT Technical Report 2004-1, 2004. www.hiit.fi/publications
• 30: Virpi Roto , Antti Oulasvirta, Need for non-visual feedback with long response times in mobile HCI, Special interest tracks and posters of the 14th international conference on World Wide Web, May 10-14, 2005, Chiba, Japan [doi>10.1145/1062745.1062747]
• 31: Sacks, H., Schegeloff, E.A. and Jefferson, G. A simplest systematics for the organization of turn-taking conversation. Language, 50 (1974), 696--735.
• 32: Nitin Sawhney , Chris Schmandt, Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.3, p.353-383, Sept. 2000 [doi>10.1145/355324.355327]
• 33: Lucy A. Suchman, Plans and situated actions: the problem of human-machine communication, Cambridge University Press, New York, NY, 1987
• 34: Sakari Tamminen , Antti Oulasvirta , Kalle Toiskallio , Anu Kankainen, Understanding mobile contexts, Personal and Ubiquitous Computing, v.8 n.2, p.135-143, May 2004 [doi>10.1007/s00779-004-0263-1]
• 35: Roel Vertegaal, Introduction, Communications of the ACM, v.46 n.3, March 2003 [doi>10.1145/636772.636794]
• 36: Weiser, M. The computer for the twenty-first century. Scientific American, 1991, 94--104.
• 37: Welford, A.T. The psychological refractory period and the timing of high speed performance-A review and a theory. British Journal of Psychology, 43 (1952), 2--19.
• 38: Wickens, C.D. Processing resources in attention. In R. Parasuraman and R. Davies (Eds.), Varieties of Attention. Academic Press, New York, 1984.
• 39: Young, M.S. and Stanton, N.S. Malleable attentional resources theory: a new explanation for the effects of mental underload on performance. Human Factors, 44, 3 (2002), 365--375.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.m Miscellaneous

Keywords:
attention, cognition, context, interruptions, mobile HCI, mobile browsers, multi-modal interfaces, multitasking, semi-naturalistic field study