Gaze alignment of interlocutors in conversational dialogues

ABSTRACT

In the area of Psycholinguistics, eye-tracking has been a successful and valuable tool for the investigation of on-line processes of language comprehension and language production (e.g., [Griffin and Bock 2000], [Tanenhaus et al. 1995]). However, the application of eye-tracking to the investigation of mechanisms underlying more naturalistic language use, e.g. dialogue, has so far been limited to the examination of eye-movements of either the speaker or the listener in isolation (e.g., [Brown-Schmidt et al. 2005]; [Richardson and Dale 2004]). Even offline dialogue experiments investigating, e.g. priming effects, usually involve only one "real" participant while their interlocutor is a confederate of the experimenter. In order to test predictions coming from dialogue models (e.g., [Pickering and Garrod 2004]) and in order to provide the kinds of evidence necessary for their further development, experimental methods that directly examine behaviour of participants actually engaged in a conversation are needed. Additionally, eye-tracking measures established in psycholinguistic monologue research need to be compared with their dialogue processing counterparts. Furthermore, new measures describing the relation between speaker and listener eye-movements in communication are needed, as they can give rise to the language mechanisms underlying conversational interaction.

REFERENCES

• 1: Brown-Schmidt, S., Campana, E., and Tanenhaus, M. K. 2005. Real-time reference resolution by nave participants during a task-based unscripted conversation. In World-situated language processing: Bridging the language as product and language as action traditions, J. Trueswell and M. Tanenhaus, Eds. MIT Press, Cambridge, MA, 153--171.
• 2: Griffin, Z. M., and Bock, K. 2000. What the eyes say about speaking. Psychological Sciences 11, 274--279.
• 3: Pickering, M. J., and Garrod, S. 2004. Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences 27, 1--57.
• 4: Richardson, D. C., and Dale, R. 2004. Looking to understand: The coupling between speakers and listeners eye movements and its relationship to discourse comprehension. In Proceedings of the Twenty-sixth Annual Conference of the Cognitive Science Society, Erlbaum, Mahwah, NJ.
• 5: Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., and Sedivy, J. 1995. Integration of visual and linguistic information during spoken language comprehension. Science 168, 1632--1634.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.6 Segmentation
      I.4.7 Feature Measurement
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology

General Terms:
Algorithms, Design, Theory