Face-tracking as an augmented input in video games: enhancing presence, role-playing and control

ABSTRACT

Motion-detection only games have inherent limitations on game experience in that the systems cannot identify the player's existence and identity. A way of improvement is by introducing information such as a player's face or head into the system. We designed and implemented two game prototypes that apply real-time face position information as intrinsic elements of gameplay to enhance game experience. The first prototype augmented a typical motion-detection-based game. Face information was designed to enhance the sense of presence and role-playing. In the second prototype, face tracking is applied as a new axis of control in a First Person Shooter (FPS) game.Although Face detection and tracking technology has started utilizing in game scenarios, there was little systematic research on how user experience is leveraged by applying face information to video games. The results of our user tests on comparing camera-based video games with and without face tracking demonstrated that using face position information can effectively enhance presence and role-playing. In addition, an intuitive control that augmented by face-tracking in the FPS game also got positive feedbacks from the test.

REFERENCES

• 1: Eyetoy games developed by Sega. http://www.sega.com/gamesite/segasuperstars/.
• 2: GestureTek, http://www.gesturetek.com/.
• 3: Sony Eye-toy, http://www.eyetoy.com/.
• 4: A. Elgammal, R. Duraiswami, D. Harwood, and L. S. Davis, Background and foreground modeling using nonparametric kernel density estimation for visual surveillance, Proceedings of the IEEE, vol. 90,2002, 1151--1163.
• 5: B.G., & Singer, M.J., Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and Virtual Environments 7 (1998), 225 -- 240.
• 6: Chris Crawford, The Art of Computer Game Design, Osborne/McGraw-Hill, Berkeley, CA, 1984
• 7: Dorin Comaniciu , Visvanathan Ramesh , Peter Meer, Kernel-Based Object Tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.5, p.564-575, May 2003 [doi>10.1109/TPAMI.2003.1195991]
• 8: Dmitry O. Gorodnichy, On Importance of Nose for Face Tracking, Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, p.188, May 20-21, 2002
• 9: Gary R. Bradski, Computer Vision Face Tracking for Use in a Perceptual User Interface, Intel Technology Journal, 1998.
• 10: Heather Desurvire , Martin Caplan , Jozsef A. Toth, Using heuristics to evaluate the playability of games, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986102]
• 11: Herman D'Hooge and Michael Goldsmith, Game Design Principles for the Intel® Play™ Me2Cam Virtual Game System, Intel Technology Journal, 2001. http://www.intel.com/technology/itj/archive/2001.htm.
• 12: Myron W. Krueger , Thomas Gionfriddo , Katrin Hinrichsen, VIDEOPLACE—an artificial reality, Proceedings of the SIGCHI conference on Human factors in computing systems, p.35-40, April 1985, San Francisco, California, United States
• 13: Lessiter, J., Freeman, J., Keogh, E., and Davidoff, J. A Cross-Media Presence Questionnaire: The ITC-Sense of Presence Inventory. Presence: Teleoperators & Virtual Environments, 10, (2001), 282--298.
• 14: Lombard, M. and Ditton, T. At the heart of it all: The concept of presence. Journal of Computer Mediated Communication, 1997.
• 15: Marian Stewart Bartlett, Gwen Littlewort, Ian Fasel, Javier R. Movellan, Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction. Conference on Computer Vision and Pattern Recognition Workshop - Volume 5, 2003, 53.
• 16: Michael Isard , Andrew Blake, C$ONDENSATION$—Conditional Density Propagation forVisual Tracking, International Journal of Computer Vision, v.29 n.1, p.5-28, Aug. 1998 [doi>10.1023/A:1008078328650]
• 17: Niklas Ravaja , Mikko Salminen , Jussi Holopainen , Timo Saari , Jari Laarni , Aki Järvinen, Emotional response patterns and sense of presence during video games: potential criterion variables for game design, Proceedings of the third Nordic conference on Human-computer interaction, p.339-347, October 23-27, 2004, Tampere, Finland [doi>10.1145/1028014.1028068]
• 18: Pugnetti, L., Meehan, M., and Mendozzi, L. Psychophysiological correlates of virtual reality: A review. Presence: Teleoperators & Virtual Environments, 10, (2001), 384--400.
• 19: Paul Viola , Michael J. Jones, Robust Real-Time Face Detection, International Journal of Computer Vision, v.57 n.2, p.137-154, May 2004 [doi>10.1023/B:VISI.0000013087.49260.fb]
• 20: Rong Xiao , Long Zhu , Hong-Jiang Zhang, Boosting Chain Learning for Object Detection, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.709, October 13-16, 2003
• 21: Terhi Säilä, Beyond Role and Play Tools, Toys and Theory for Harnessing the Imagination, http://www.ropecon.fi/brap/.
• 22: Pattie Maes , Trevor Darrell , Bruce Blumberg , Alex Pentland, The ALIVE system: wireless, full-body interaction with autonomous agents, Multimedia Systems, v.5 n.2, p.105-112, March 1997 [doi>10.1007/s005300050046]
• 23: W. Grimson and C. Stauffer, Adaptive background mixture models for real-time tracking, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, 1999, 22--29.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.m Miscellaneous

Keywords:
camera-based games, face tracking, first person shooter (FPS), game control, motion detection, presence, role-playing