Computational mechanisms for gaze direction in interactive visual environments

ABSTRACT

Next-generation immersive virtual environments and video games will require virtual agents with human-like visual attention and gaze behaviors. A critical step is to devise efficient visual processing heuristics to select locations that would attract human gaze in complex dynamic environments. One promising approach to designing such heuristics draws on ideas from computational neuroscience. We compared several such heuristics with eye movement recordings from five observers playing video games, and found that heuristics which detect outliers from the global distribution of visual features were better predictors of human gaze than were purely local heuristics. Heuristics sensitive to dynamic events performed best overall. Further, heuristic prediction power differed more between games than between different human observers. Our findings suggest simple neurally-inspired algorithmic methods to predict where humans look while playing video games.

REFERENCES

• 1: Bailenson, J., and Yee, N. 2005. Digital chameleons: Automatic assimilation of nonverbal gestures in immersive virtual environments. Psychological Science in press.
• 2: Finney, S. A. 2001. Real-time data collection, in linux: A case study. Behavior Research Methods, Instruments, and Computers 33, 167--173.
• 3: Mary M. Hayhoe , Dana H. Ballard , Jochen Triesch , Hiroyuki Shinoda , Pilar Aivar , Brian Sullivan, Vision in natural and virtual environments, Proceedings of the 2002 symposium on Eye tracking research & applications, March 25-27, 2002, New Orleans, Louisiana [doi>10.1145/507072.507074]
• 4: Hayhoe, M., Shrivastava, A., Mruczek, R., and Pelz, J. 2003. Visual memory and motor planning in a natural task. Journal of Vision 3, 1, 49--63.
• 5: Henderson, J. M., and Hollingworth, A. 1999. High-level scene perception. Annual Review of Psychology 50, 243--271.
• 6: Laurent Itti , Christof Koch , Ernst Niebur, A Model of Saliency-Based Visual Attention for Rapid Scene Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.20 n.11, p.1254-1259, November 1998 [doi>10.1109/34.730558]
• 7: Itti, L., Dhavale, N., and Pighin, F. 2003. Realistic avatar eye and head animation using a neurobiological model of visual attention. In Proceedings of SPIE 48th annual international symposium on optical science and technology, 64--78.
• 8: Land, M., and Hayhoe, M. 2001. In what ways do eye movements contribute to everyday activities? Vision Research 41, 25--26, 3559--3565.
• 9: Parkhurst, D., Law, K., and Niebur, E. 2002. Modeling the role of salience in the allocation of overt visual attention. Vision Research 42, 1, 107--123.
• 10: Christopher Peters , Carol O' Sullivan, Bottom-Up Visual Attention for Virtual Human Animation, Proceedings of the 16th International Conference on Computer Animation and Social Agents (CASA 2003), p.111, May 08-09, 2003
• 11: Peters, R., Iyer, A., Itti, L., and Koch, C. 2005, Components of bottom-up gaze allocation in natural images. Vision Research 45, 18, 2397--2416.
• 12: Claudio M. Privitera , Lawrence W. Stark, Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.9, p.970-982, September 2000 [doi>10.1109/34.877520]
• 13: Reinagel, P., and Zador, A. 1999. Natural scene statistics at the centre of gaze. Network-Computation in Neural Systems 10, 4 (November), 341--350.
• 14: Rensink, R. A. 2000. The dynamic representation of scenes. Visual Cognition 7, 17--742.
• 15: M. Sodhi , B. Reimer , J. L. Cohen , E. Vastenburg , R. Kaars , S. Kirschenbaum, On-road driver eye movement tracking using head-mounted devices, Proceedings of the 2002 symposium on Eye tracking research & applications, March 25-27, 2002, New Orleans, Louisiana [doi>10.1145/507072.507086]
• 16: Stampe, D. M. 1993. Heuristic filtering and reliable calibration methods for video based pupil tracking systems. Behavior Research Methods, Instruments, and Computers 25, 2, 137--142.
• 17: Terzopoulos, D., and Rabie, T. F. 1997. Animat vision: Active vision in artificial animals. Videre: Journal of Computer Vision Research 1, 1, 2--19.
• 18: Wolfe, J., and Horowitz, T. 2004. What attributes guide the deployment of visual attention and how do they do it? Nature Reviews Neuroscience 5, 6 (June), 495--501.
• 19: Yarbus, A. 1967. Eye movements during perception of complex objects. In Eye Movements and Vision, L. Riggs, Ed. Plenum Press, New York, NY.
• 20: C. Zetzsche , K. Schill , H. Deubel , G. Krieger , E. Umkehrer , S. Beinlich, Investigation of a sensorimotor system for saccadic scene analysis: an integrated approach, Proceedings of the fifth international conference on simulation of adaptive behavior on From animals to animats 5, p.120-126, September 1998, Univ. of Zurich, Zurich, Switzerland

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Software psychology

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Perceptual reasoning
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation
          Subjects: Viewing algorithms

Keywords:
active vision, computational modeling, eye movements, immersive environments, video games, visual attention