Video game design using an eye-movement-dependent model of visual attention

ABSTRACT

Eye movements can be used to infer the allocation of covert attention. In this article, we propose to model the allocation of attention in a task-dependent manner based on different eye movement conditions, specifically fixation and pursuit. We show that the image complexity at eye fixation points during fixation, and the pursuit direction during pursuit are significant factors in attention allocation. Results of the study are applied to the design of an interactive computer game. Real-time eye movement information is taken as one of inputs for the game. The utility of such eye information for controlling game difficulty is shown.

REFERENCES

• 1: Dye, M. and Bavelier, D. 2004. Playing video games enhances visual attention in children. J. Vis. 4, 11, Abstract 40, 40a.
• 2: Pedro F. Felzenszwalb , Daniel P. Huttenlocher, Efficient Graph-Based Image Segmentation, International Journal of Computer Vision, v.59 n.2, p.167-181, September 2004 [doi>10.1023/B:VISI.0000022288.19776.77]
• 3: Green, C. and Bavelier, D. 2003. Action video game modifies visual selective attention. Nature 423, 534--537.
• 4: Green, C. and Bavelier, D. 2007. Action-video-game experience alters the spatial resolution of vision. Psychol. Sci. 18, 1, 88--94.
• 5: Guo, K., Mahmoodi, S., Robertson, R., and Young, M. 2006. Longer fixation duration while viewing face images. Exp. Brain Res. 171, 1, 91--98.
• 6: Henderson, J. and Hollingworth, A. 1998. Eye movements during scene viewing: an overview. In Eye Guidance in Reading and Scene Perception, G. Underwood, Ed. 269--283.
• 7: Henderson, J., Weeks, P., and Hollingworth, A. 1999. The effects of semantic consistency on eye movements during complex scene viewing. J. Exp. Psychol.: Hum. Perc. Perf. 25, 210--228.
• 8: Itti, L. and Koch, C. 2000. A saliency-based search mechanism for overt and covert shift of visual attention. Vision Res. 40, 10--12, 1489--1506.
• 9: Jie, L. and Clark, J. 2005. Microsaccadic eye movements during ocular pursuit. Vision Sciences Society Annual Meeting (VSS) 5, 8, Abstract, 697a.
• 10: Kayser, C., Nielsen, K., and Logothetis, N. 2006. Fixations in natural scenes: interaction of image structure and image content. Vision Res. 46, 16, 2535--2545.
• 11: Land, M. and Hayhoe, M. 2001. In what ways do eye movements contribute to everyday activities? Vision Res. 41, 3559--3565.
• 12: Loftus, G. 1985. Picture perception: effects of luminance on available information and information-extraction rate. J. Exp. Psychol. Gen 114, 3, 342--356.
• 13: Loftus, G., Kaufman, L., Nishimoto, T., and Ruthruff, E. 1992. Effects of visual degradation on eye-fixation durations, perceptual processing, and long-term visual memory. In Eye Movements and Visual Cognition: Scene Perception and Reading, K. Reyner, Ed. 203--226.
• 14: Mannan, S., Ruddock, K., and Wooding, D. 1996. The relationship between the locations of spatial features and those fixations made during visual examination of briefly presented images. Spatial Vision 10, 3, 165--188.
• 15: Mannan, S., Ruddock, K., and Wooding, D. 1997. Fixation sequences made during visual examination of briefly presented 2d images. Spatial Vision 11, 2, 157--178.
• 16: Oliva, A., Torralba, A., Castelhano, M., and Henderson, J. 2003. Top-down control of visual attention in object detection. In Proceedings of the IEEE International Conference on Image Processing. IEEE Computer Society Press, Los Alamitos, CA, 253--256.
• 17: Parkhurst, D. and Niebur, E. 2003. Scene content selected by active vision. Spatial Vis. 16, 2, 125--154.
• 18: Pomplun, M. 2006. Saccadic selectivity in complex visual search displays. Vision Res. 46(12), 1886--1900.
• 19: Raj, R., Geisler, W., Frazor, R., and Bovik, A. 2005. Contrast statistics for foveated visual systems: Fixation selection by minimizing contrast entropy. J. Opt. Soc. Am. A. 22, 10, 2039--2049.
• 20: Rajashekar, U., Cormack, L., and Bovik, A. 2003. Image features that draw fixations. In Proceedings of the IEEE International Conference on Image Processing. IEEE Computer Society Press, Los Alamitos, CA, 313--316.
• 21: Rayner, K. 1998. Eye movements in reading and information processing: 20 years of research. Psychol. Bull. 124, 372--422.
• 22: Reichle, E., Pollatsek, A., Fisher, D., and Rayner, K. 1998. Toward a model of eye movement control in reading. Psychol. Rev. 105, 125--157.
• 23: Rueda, M., Rothbart, M., McCandliss, B., Saccomanno, L., and Posner, M. 2005. From the cover: Training, maturation, and genetic influences on the development of executive attention. PNAS 102, 14931--14936.
• 24: Magy Seif El-Nasr , Chinmay Rao, Visually directing user's attention in interactive 3D environments, ACM SIGGRAPH 2004 Posters, August 08-12, 2004, Los Angeles, California [doi>10.1145/1186415.1186510]
• 25: Magy Seif El-Nasr , Su Yan, Visual attention in 3D video games, Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology, June 14-16, 2006, Hollywood, California [doi>10.1145/1178823.1178849]
• 26: Rainer Stiefelhagen , Jie Zhu, Head orientation and gaze direction in meetings, CHI '02 extended abstracts on Human factors in computing systems, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/506443.506634]
• 27: Torralba, A., Oliva, A., Castelhano, M., and Henderson, J. 2006. Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search. Psychol. Rev. 113, 766--786.
• 28: Turano, K. A., Geruschat, D., and Baker, F. 2003. Oculomotor strategies for the direction of gaze tested with a real-world activity. Vision Res. 43, 333--346.
• 29: Van Donkelaar, P. and Drew, A. 2002. The allocation of attention during smooth pursuit eye movements. Prog. Brain. Res. 140, 267--277.
• 30: Vinje, W. and Gallant, J. 2000. Sparse coding and decorrelationin primary visual cortex during natural vision. Science 287, 1273V1276.
• 31: Wolfe, J., Cave, K., and Franzel, S. 1989. Guided search: An alternative to the feature integration model for visual search. J. Exp. Psychol.: Hum. Perc. Perf. 15, 419--433.
• 32: Yarbus, A. 1967. Eye Movements and Vision. Plenum Press, New York.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human information processing
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

  J. Computer Applications
  J.4 SOCIAL AND BEHAVIORAL SCIENCES
      Subjects: Psychology

General Terms:
Design, Human Factors

Keywords:
Entertainment, HCI, eye movements, eye tracking, video games, visual attention