Understanding and developing models for detecting and differentiating breakpoints during interactive tasks

ABSTRACT

The ability to detect and differentiate breakpoints during task execution is critical for enabling defer-to-breakpoint policies within interruption management. In this work, we examine the feasibility of building statistical models that can detect and differentiate three granularities (types) of perceptually meaningful breakpoints during task execution, without having to recognize the underlying tasks. We collected ecological samples of task execution data, and asked observers to review the interaction in the collected videos and identify any perceived breakpoints and their type. Statistical methods were applied to learn models that map features of the interaction to each type of breakpoint. Results showed that the models were able to detect and differentiate breakpoints with reasonably high accuracy across tasks. Among many uses, our resulting models can enable interruption management systems to better realize defer-to-breakpoint policies for interactive, free-form tasks.

REFERENCES

• 1: Piotr D. Adamczyk , Brian P. Bailey, If not now, when?: the effects of interruption at different moments within task execution, Proceedings of the SIGCHI conference on Human factors in computing systems, p.271-278, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985727]
• 2: Altheide, D.L. Qualitative Media Analysis. Sage, Newbury Park, CA, 1996.
• 3: Ricardo A. Baeza-Yates , Berthier Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 4: Bailey, B.P., P.D. Adamczyk, T.Y. Chang and N.A. Chilson. A Framework for Specifying and Monitoring User Tasks. Journal of Computers in Human Behavior, 22 (4), 2006, 685--708.
• 5: Bailey, B.P. and J.A. Konstan. On the Need for Attention Aware Systems: Measuring Effects of Interruption on Task Performance, Error Rate, and Affective State. Journal of Computers in Human Behavior, 22 (4), 2006, 709--732.
• 6: Stuart K. Card , Allen Newell , Thomas P. Moran, The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 1983
• 7: Czerwinski, M., E. Cutrell and E. Horvitz. Instant Messaging: Effects of Relevance and Timing. People and Computers XIV: Proceedings of HCI, 2000, 71--76.
• 8: Mary Czerwinski , Eric Horvitz , Susan Wilhite, A diary study of task switching and interruptions, Proceedings of the SIGCHI conference on Human factors in computing systems, p.175-182, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985715]
• 9: Anton N. Dragunov , Thomas G. Dietterich , Kevin Johnsrude , Matthew McLaughlin , Lida Li , Jonathan L. Herlocker, TaskTracer: a desktop environment to support multi-tasking knowledge workers, Proceedings of the 10th international conference on Intelligent user interfaces, January 10-13, 2005, San Diego, California, USA [doi>10.1145/1040830.1040855]
• 10: James Fogarty , Scott E. Hudson , Jennifer Lai, Examining the robustness of sensor-based statistical models of human interruptibility, Proceedings of the SIGCHI conference on Human factors in computing systems, p.207-214, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985719]
• 11: James Fogarty , Andrew J. Ko , Htet Htet Aung , Elspeth Golden , Karen P. Tang , Scott E. Hudson, Examining task engagement in sensor-based statistical models of human interruptibility, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055018]
• 12: Victor M. González , Gloria Mark, "Constant, constant, multi-tasking craziness": managing multiple working spheres, Proceedings of the SIGCHI conference on Human factors in computing systems, p.113-120, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985707]
• 13: Mark A. Hall, Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning, Proceedings of the Seventeenth International Conference on Machine Learning, p.359-366, June 29-July 02, 2000
• 14: Hanson, C. and W. Hirst. On the Representation of Events: A Study of Orientation, Recall, and Recognition. Journal of Experimental Psychology: General, 118 (2), 1989, 136--147.
• 15: D. Austin Henderson, Jr. , Stuart Card, Rooms: the use of multiple virtual workspaces to reduce space contention in a window-based graphical user interface, ACM Transactions on Graphics (TOG), v.5 n.3, p.211-243, July 1986 [doi>10.1145/24054.24056]
• 16: Joyce Ho , Stephen S. Intille, Using context-aware computing to reduce the perceived burden of interruptions from mobile devices, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055100]
• 17: Eric Horvitz , Paul Koch , Johnson Apacible, BusyBody: creating and fielding personalized models of the cost of interruption, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031690]
• 18: Shamsi T. Iqbal , Brian P. Bailey, Investigating the effectiveness of mental workload as a predictor of opportune moments for interruption, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1056948]
• 19: Shamsi T. Iqbal , Brian P. Bailey, Leveraging characteristics of task structure to predict the cost of interruption, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124882]
• 20: Bonnie E. John , David E. Kieras, The GOMS family of user interface analysis techniques: comparison and contrast, ACM Transactions on Computer-Human Interaction (TOCHI), v.3 n.4, p.320-351, Dec. 1996 [doi>10.1145/235833.236054]
• 21: Gloria Mark , Victor M. Gonzalez , Justin Harris, No task left behind?: examining the nature of fragmented work, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055017]
• 22: Nair, R., S. Voida and E. Mynatt. Frequency--Based Detection of Task Switches. Proceedings of the 19th British HCI Group Annual Conference, 2005, 94--99.
• 23: Newtson, D. Attribution and the Unit of Perception of Ongoing Behavior. Journal of Personality and Social Psychology, 28 (1), 1973, 28--38.
• 24: Newtson, D. and G. Engquist. The Perceptual Organization of Ongoing Behavior. Journal of Experimental Social Psychology, 12, 1976, 436--450.
• 25: Newtson, D., G. Enquist and J. Bois. The Objective Basis of Behavior Units. Journal of Personality and Social Psychology, 35 (12), 1977, 847--862.
• 26: Rizzolatti, G., L. Fadiga, V. Gallese and L. Fogassi. Premotor Cortex and the Recognition of Motor Actions. Cognitive Brain Research, 3, 1996, 131--141.
• 27: Jianqiang Shen , Lida Li , Thomas G. Dietterich , Jonathan L. Herlocker, A hybrid learning system for recognizing user tasks from desktop activities and email messages, Proceedings of the 11th international conference on Intelligent user interfaces, January 29-February 01, 2006, Sydney, Australia [doi>10.1145/1111449.1111473]
• 28: Zacks, J., B. Tversky and G. Iyer. Perceiving, Remembering, and Communicating Structure in Events. Journal of Experimental Psychology: General, 130 (1), 2001, 29--58.
• 29: Zacks, J.M. and B. Tversky. Event Structure in Perception and Conception. Psychological Bulletin, 127, 2001, 3--21.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human information processing

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

General Terms:
Experimentation, Human Factors

Keywords:
attention, breakpoints, interruption, statistical models