TAPRAV: a tool for exploring workload aligned to models of task execution

ABSTRACT

Existing analysis tools are not sufficient for exploring pupillary response, as the data typically needs to be explored in relation to the corresponding task's execution. To address this need, we have developed an interactive visualization tool called TAPRAV. Key components include (i) a visualization of the pupillary response aligned to the model of task execution, useful for making sense of the overall data set; (ii) an interactive overview+detail metaphor, enabling rapid inspection of details; (iii) synchronization with the video of screen interaction, providing awareness of the state of the task; and (iv) interaction supporting discovery driven analysis.

REFERENCES

• 1: Backs, R. W. and L. C. Walrath. Eye Movement and Pupillary Response Indices of Mental Workload During Visual Search of Symbolic Displays. Applied Ergonomics, 23, 243--254.
• 2: Michelle Q. Wang Baldonado , Allison Woodruff , Allan Kuchinsky, Guidelines for using multiple views in information visualization, Proceedings of the working conference on Advanced visual interfaces, p.110-119, May 2000, Palermo, Italy [doi>10.1145/345513.345271]
• 3: Beatty, J. Task-Evoked Pupillary Responses, Processing Load, and the Structure of Processing Resources. Psychological Bulletin, 91 (2), 276--292.
• 4: Stuart K. Card , Allen Newell , Thomas P. Moran, The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 1983
• 5: Juan Casares , A. Chris Long , Brad Myers , Scott Stevens , Albert Corbett, Simplifying video editing with SILVER, CHI '02 extended abstracts on Human factors in computing systems, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/506443.506539]
• 6: Granholm, E., R. F. Asarnow, A. J. Sarkin and K. L. Dykes. Pupillary Responses Index Cognitive Resource Limitations. Psychophysiology, 33 (4), 457--461.
• 7: Kasper Hornbæk , Benjamin B. Bederson , Catherine Plaisant, Navigation patterns and usability of zoomable user interfaces with and without an overview, ACM Transactions on Computer-Human Interaction (TOCHI), v.9 n.4, p.362-389, December 2002 [doi>10.1145/586081.586086]
• 8: Shamsi T. Iqbal , Piotr D. Adamczyk , Xianjun Sam Zheng , Brian P. Bailey, Towards an index of opportunity: understanding changes in mental workload during task execution, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055016]
• 9: Bonnie John , Alonso Vera , Michael Matessa , Michael Freed , Roger Remington, Automating CPM-GOMS, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503404]
• 10: Kok, A. Event-Related-Potential (ERP) Reflections of Mental Resources: A Review and Synthesis. Biological Psychology, 45, 19--56.
• 11: Jock D. Mackinlay , George G. Robertson , Stuart K. Card, The perspective wall: detail and context smoothly integrated, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.173-176, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108870]
• 12: Marshall, S. P. The Index of Cognitive Activity: Measuring Cognitive Workload. Proc. IEEE Conference on Human Factors and Power Plants, 2002, 7.5--7.9.
• 13: Michael Mills , Jonathan Cohen , Yin Yin Wong, A magnifier tool for video data, Proceedings of the SIGCHI conference on Human factors in computing systems, p.93-98, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142764]
• 14: Miyata, Y. and D. A. Norman. Psychological Issues in Support of Multiple Activities. User Centered System Design: New Perspectives on Human-Computer Interaction, Lawrence Erlbaum Associates, Hillsdale, NJ, 1986, 265--284.
• 15: Pomplun, M. and S. Sunkara. Pupil Dilation as an Indicator of Cognitive Workload in Human-Computer Interaction. Proc. International Conference on HCI, 2003.
• 16: Dennis W. Rowe , John Sibert , Don Irwin, Heart rate variability: indicator of user state as an aid to human-computer interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.480-487, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274709]
• 17: Manojit Sarkar , Marc H. Brown, Graphical fisheye views of graphs, Proceedings of the SIGCHI conference on Human factors in computing systems, p.83-91, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142763]
• 18: Schacter, D. EEG Theta Waves and Psychological Phenomena: A Review and Analysis. Biological Psychology, 5 (1), 47--82.
• 19: Schluroff, M., et al. Pupillary Responses to Syntactic Ambiguity of Sentences. Brain and Language, 27, 322--344.
• 20: Spence, R. and M. Apperley. Database Navigation: An Office Environment for the Professional. BIT, 1 (1), 43--54.
• 21: Chris Stolte , Robert Bosch , Pat Hanrahan , Mendel Rosenblum, Visualizing Application Behavior on Superscalar Processors, Proceedings of the 1999 IEEE Symposium on Information Visualization, p.10, October 24-29, 1999
• 22: Verney, S. P., E. Granholm and D. P. Dionisio. Pupillary Responses and Processing Resources on the Visual Backward Masking Task. Psychophysiology, 38 (1), 76--83.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.6 Methodology and Techniques

General Terms:
Design, Experimentation, Human Factors

Keywords:
mental workload, pupil size, task models, visualization