Another person's eye gaze as a cue in solving programming problems

ABSTRACT

Expertise in computer programming can often be difficult to transfer verbally. Moreover, technical training and communication occur more and more between people who are located at a distance. We tested the hypothesis that seeing one person's visual focus of attention (represented as an eyegaze cursor) while debugging software (displayed as text on a screen) can be helpful to another person doing the same task. In an experiment, a group of professional programmers searched for bugs in small Java programs while wearing an unobtrusive head-mounted eye tracker. Later, a second set of programmers searched for bugs in the same programs. For half of the bugs, the second set of programmers first viewed a recording of an eyegaze cursor from one of the first programmers displayed over the (indistinct) screen of code, and for the other half they did not. The second set of programmers found the bugs more quickly after viewing the eye gaze of the first programmers, suggesting that another person's eye gaze, produced instrumentally (as opposed to intentionally, like pointing with a mouse), can be a useful cue in problem solving. This finding supports the potential of eye gaze as a valuable cue for collaborative interaction in a visuo-spatial task conducted at a distance.

REFERENCES

• 1: Ballard, D. H., Hayhoe, M. M., Li, F., and Whitehead, S. D. (1992). Hand-eye coordination during sequential tasks. Philosophical Transactions of the Royal Society of London, B(337), 331--339.
• 2: Ballard, D. H., Hayhoe, M. M., and Pelz, J. B. (1995). Memory representations in natural tasks. Journal of Cognitive Neuroscience, 7(1), 66--80.
• 3: Brennan, S. E. (1990). Seeking and providing evidence for mutual understanding. Unpublished doctoral dissertation, Stanford University.
• 4: Brennan, S. E. (In press). How conversation is shaped by visual and spoken evidence. In J. Trueswell and M. Tanenhaus (Eds.), World Situated Language Use: Psycholinguistic, Linguistic and Computational Perspectives on Bridging the Product and Action Traditions. Cambridge, MA: MIT Press.
• 5: Brennan, S. E. (1995). Centering attention in discourse. Language and Cognitive Processes, 10, 137--167.
• 6: Brennan, S. E. and Lockridge, C. B. Monitoring an addressee's visual attention: Effects of visual co-presence on referring in conversation. Unpublished manuscript.
• 7: Clark, H.H. and Brennan, S.E. (1991). Grounding in communication. In L.B. Resnick, J.M. Levine and S.D. Teasley (Eds.) Perspectives on socially shared cognition (pp. 127--149). Washington, DC: American Psychological Association.
• 8: Doherty-Sneddon, G., Anderson, A., O'Malley, C., Langton, S., Garrod, S., and Bruce, V. (1997). Face-to-face and video-mediated communication: A comparison of dialogue structure and task performance. Journal of Experimental Psychology: Applied, 3, 105-125.
• 9: Paul Dourish , Annette Adler , Victoria Bellotti , Austin Henderson, Your place or mine? Learning from long-term use of audio-video communication, Computer Supported Cooperative Work, v.5 n.1, p.33-62, 1996
• 10: Vikki Fix , Susan Wiedenbeck , Jean Scholtz, Mental representations of programs by novices and experts, Proceedings of the SIGCHI conference on Human factors in computing systems, p.74-79, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169088]
• 11: Grice, H. P. (1975). Logic and conversation (from the William James lectures, Harvard University, 1967). In P. Cole and J. Morgan (Eds.), Syntax and semantics 3: Speech acts. (pp. 41--58). New York: Academic Press.
• 12: Barbara J. Grosz , Scott Weinstein , Aravind K. Joshi, Centering: a framework for modeling the local coherence of discourse, Computational Linguistics, v.21 n.2, p.203-225, June 1995
• 13: Barbara J. Grosz , Candace L. Sidner, Attention, intentions, and the structure of discourse, Computational Linguistics, v.12 n.3, p.175-204, July-September 1986
• 14: L. Gugerty , G. Olson, Debugging by skilled and novice programmers, Proceedings of the SIGCHI conference on Human factors in computing systems, p.171-174, April 13-17, 1986, Boston, Massachusetts, United States
• 15: Carl Gutwin , Saul Greenberg , Mark Roseman, Supporting awareness of others in groupware, Conference companion on Human factors in computing systems: common ground, p.205, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/257089.257282]
• 16: Hanna, J. E, and Brennan, S. E. (2003). Eye gaze has immediate effects on reference resolution in conversation. Abstracts of the Psychonomic Society, 44th Annual Meeting (p. 124), Vancouver, Canada.
• 17: Hanna, J.E. and Brennan, S. E. (2004). Using a Speaker's Eye gaze During Comprehension: A Cue Both Rapid and Flexible. Abstract, 17th Annual CUNY Conference on Human Sentence Processing, College Park, MD.
• 18: Jon Hindmarsh , Mike Fraser , Christian Heath , Steve Benford , Chris Greenhalgh, Fragmented interaction: establishing mutual orientation in virtual environments, Proceedings of the 1998 ACM conference on Computer supported cooperative work, p.217-226, November 14-18, 1998, Seattle, Washington, United States [doi>10.1145/289444.289496]
• 19: Ellen A. Isaacs , John C. Tang , Trevor Morris, Piazza: a desktop environment supporting impromptu and planned interactions, Proceedings of the 1996 ACM conference on Computer supported cooperative work, p.315-324, November 16-20, 1996, Boston, Massachusetts, United States [doi>10.1145/240080.240316]
• 20: Hiroshi Ishii , Minoru Kobayashi , Jonathan Grudin, Integration of inter-personal space and shared workspace: ClearBoard design and experiments, Proceedings of the 1992 ACM conference on Computer-supported cooperative work, p.33-42, November 01-04, 1992, Toronto, Ontario, Canada [doi>10.1145/143457.143459]
• 21: Robert J. K. Jacob, Eye tracking in advanced interface design, Virtual environments and advanced interface design, Oxford University Press, Inc., New York, NY, 1995
• 22: Karsenty, L. (1999). Cooperative work and shared visual context: An empirical study of comprehension problems in side-by-side and remote help dialogues. Human-Computer Interaction, 14, 283-316.
• 23: Jürgen Koenemann , Scott P. Robertson, Expert problem solving strategies for program comprehension, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.125-130, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108863]
• 24: Kraut, R. E., Fussell, S. R., Brennan, S. E., and Siegel, J. (2002). Understanding effects of proximity on collaboration : Implications for technologies to support remote collaborative work. In P. Hinds and S. Kiesler, Distributed work (pp. 137--162). Cambridge, MA: MIT Press.
• 25: Bonnie A. Nardi , Heinrich Schwarz , Allan Kuchinsky , Robert Leichner , Steve Whittaker , Robert Sclabassi, Turning away from talking heads: the use of video-as-data in neurosurgery, Proceedings of the SIGCHI conference on Human factors in computing systems, p.327-334, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169261]
• 26: Vennila Ramalingam , Susan Wiedenbeck, An empirical study of novice program comprehension in the imperative and object-oriented styles, Papers presented at the seventh workshop on Empirical studies of programmers, p.124-139, October 1997, Alexandria, Virginia, United States [doi>10.1145/266399.266411]
• 27: Sellen, A. J. (1995). Remote conversations: The effects of mediating talk with technology. Human-Computer Interaction, 10, 401--444.
• 28: Elliot Soloway , Kate Ehrlich , Jeffrey Bonar, Tapping into tacit programming knowledge, Proceedings of the 1982 conference on Human factors in computing systems, p.52-57, March 15-17, 1982, Gaithersburg, Maryland, United States [doi>10.1145/800049.801754]
• 29: J. C. Spohrer , E. Soloway, Alternatives to construct-based programming misconceptions, Proceedings of the SIGCHI conference on Human factors in computing systems, p.183-191, April 13-17, 1986, Boston, Massachusetts, United States
• 30: James C. Spohrer , Elliot Soloway, Novice mistakes: are the folk wisdoms correct?, Communications of the ACM, v.29 n.7, p.624-632, July 1986 [doi>10.1145/6138.6145]
• 31: James C. Spohrer , Elliot Soloway , Edgar Pope, Where the bugs are, Proceedings of the SIGCHI conference on Human factors in computing systems, p.47-53, April 1985, San Francisco, California, United States
• 32: Tang, J. and Isaacs, E. (1993). Why do users like video? Studies of multimedia-supported collaboration. Computer Supported Cooperative Work, 11, 163--196.
• 33: Mark Weiser, Programmers use slices when debugging, Communications of the ACM, v.25 n.7, p.446-452, July 1982 [doi>10.1145/358557.358577]
• 34: Steve Whittaker, Rethinking video as a technology for interpersonal communications: theory and design implications, International Journal of Human-Computer Studies, v.42 n.5, p.501-529, May 1995 [doi>10.1006/ijhc.1995.1022]
• 35: Steve Whittaker , Erik Geelhoed , Elizabeth Robinson, Shared workspaces: how do they work and when are they useful?, International Journal of Man-Machine Studies, v.39 n.5, p.813-842, Nov. 1993 [doi>10.1006/imms.1993.1085]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces

General Terms:
Experimentation, Human Factors

Keywords:
debugging, eye tracking, gaze-based & attentional interfaces, mediated communication, programming, visual co-presence