Finding canonical behaviors in user protocols

ABSTRACT

While the collection of behavioral protocols has been common practice in human-computer interaction research for many years, the analysis of large protocol data sets is often extremely tedious and time-consuming, and automated analysis methods have been slow to develop. This paper proposes an automated method of protocol analysis to find canonical behaviors --- a small subset of protocols that is most representative of the full data set, providing a reasonable "big picture" view of the data with as few protocols as possible. The automated method takes advantage of recent algorithmic developments in computational vision, modifying them to allow for distance measures between behavioral protocols. The paper includes an application of the method to web-browsing protocols, showing how the canonical behaviors found by the method match well to sets of behaviors identified by expert human coders.

REFERENCES

• 1: Michael D. Byrne , John R. Anderson , Scott Douglass , Michael Matessa, Eye tracking the visual search of click-down menus, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.402-409, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303118]
• 2: Stuart K. Card , Allen Newell , Thomas P. Moran, The Psychology of Human-Computer Interaction, L. Erlbaum Associates Inc., Hillsdale, NJ, 1983
• 3: Stuart K. Card , Peter Pirolli , Mija Van Der Wege , Julie B. Morrison , Robert W. Reeder , Pamela K. Schraedley , Jenea Boshart, Information scent as a driver of Web behavior graphs: results of a protocol analysis method for Web usability, Proceedings of the SIGCHI conference on Human factors in computing systems, p.498-505, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365331]
• 4: Ed H. Chi , Adam Rosien , Gesara Supattanasiri , Amanda Williams , Christiaan Royer , Celia Chow , Erica Robles , Brinda Dalal , Julie Chen , Steve Cousins, The bloodhound project: automating discovery of web usability issues using the InfoScentπ simulator, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642699]
• 5: Edward Cutrell , Zhiwei Guan, What are you looking for?: an eye-tracking study of information usage in web search, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240690]
• 6: Trip Denton , Ali Shokoufandeh , John Novatnack , Ko Nishino, Canonical subsets of image features, Computer Vision and Image Understanding, v.112 n.1, p.55-66, October, 2008 [doi>10.1016/j.cviu.2008.06.002]
• 7: Ericsson, K.A.,&Simon, H.A. (1984). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.
• 8: Michael R. Garey , David S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, W. H. Freeman & Co., New York, NY, 1979
• 9: Anthony J. Hornof , Tim Halverson, Cognitive strategies and eye movements for searching hierarchical computer displays, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642656]
• 10: Levenshtein, V. (1966). Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Daklady, 10, 707--710.
• 11: Laila Paganelli , Fabio Paternò, Intelligent analysis of user interactions with web applications, Proceedings of the 7th international conference on Intelligent user interfaces, January 13-16, 2002, San Francisco, California, USA [doi>10.1145/502716.502735]
• 12: Peter Pirolli , Wai-Tat Fu , Robert Reeder , Stuart K. Card, A user-tracing architecture for modeling interaction with the world wide web, Proceedings of the Working Conference on Advanced Visual Interfaces, May 22-24, 2002, Trento, Italy [doi>10.1145/1556262.1556272]
• 13: Frank E. Ritter , Jill H. Larkin, Developing process models as summaries of HCI action sequences, Human-Computer Interaction, v.9 n.4, p.345-383, September 1994 [doi>10.1207/s15327051hci0903&4_4]
• 14: Dario D. Salvucci , John R. Anderson, Automated eye-movement protocol analysis, Human-Computer Interaction, v.16 n.1, p.39-86, March 2001 [doi>10.1207/S15327051HCI1601_2]
• 15: Penelope Sanderson , Jay Scott , Tom Johnston , John Mainzer , Larry Watanabe , Jeff James, MacSHAPA and the enterprise of exploratory sequential data analysis (ESDA), International Journal of Human-Computer Studies, v.41 n.5, p.633-681, Nov. 1994 [doi>10.1006/ijhc.1994.1077]
• 16: John B. Smith , Dana Kay Smith , Eileen Kupstas, Automated protocol analysis, Human-Computer Interaction, v.8 n.2, p.101-145, June 1993 [doi>10.1207/s15327051hci0802_2]
• 17: Taatgen, N. A.,&Lee, F. J. (2003). Production compilation: A simple mechanism to model complex skill acquisition. Human Factors, 45, 61--76.
• 18: Toh, K.C., Todd, M.J.,&Tutuncu, R. (1999). SDPT3: a Matlab package for semidefinite programming. Optimization Methods and Software, 11, 545--581.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

General Terms:
Algorithms, Human Factors

Keywords:
protocol analysis, sequential data analysis, web browsing