O' game, can you feel my frustration?: improving user's gaming experience via stresscam

ABSTRACT

One of the major challenges of video game design is to have appropriate difficulty levels for users in order to maximize the entertainment value of the game. Game players may lose interests if a game is either too easy or too difficult. This paper presents a novel methodology to improve user's experience in computer games by automatically adjusting the level of the game difficulty. The difficulty level is computed from measurements of the facial physiology of the players at a distance. The measurements are based on the assumption that the players' performance during the game-playing session alters blood flow in the supraorbital region, which is an indirect measurement of increased mental activities. This alters heat dissipation, which can be monitored in a contact-free manner through a thermal imaging-based stress monitoring and analysis system, known as StressCam.

In this work, we investigated on two primary objectives: (1) the feasibility of utilizing the facial physiology in automatically adjusting the difficulty level of the game and (2) the capability of the automatic difficulty level adjustment in improving game players' experience. We employed and extended a XNA video game for this study, and performed an in-depth, comparative usability evaluation on it. Our results show that the automatic difficulty adjustable system successfully maintains game players' interests and substantially outperforms traditional fixed-difficulty mode games. Although a number of issues of this preliminary study remain to be investigated further, this research opens a new direction that utilizes non-contact stress measurements for monitoring and further enhancing a variety of user-centric, interactive entertainment activities.

REFERENCES

• 1: Anderson, C.A. and Dill, K.E. Video games and aggressive thoughts, feelings, and behavior in the laboratory and in life. Journal of Personality and Social Psychology, 78, (2000), 772--790.
• 2: Anderson, C.A. and Bushman, B.J. Effects of violent video games on aggressive behavior, aggressive cognition, aggressive affect, physiological arousal, and prosocial behavior: A meta-analytic review of the scientific literature. Psychological Science, 12, 5 (2001), 353--359.
• 3: Anderson, C.A. An update on the effects of playing violent video games. Journal of Adolescence, 27, 1 (2004), 113--122.
• 4: Sherry, J.L. The effects of violent video games on aggression: A meta-analysis. Human Communication Research, 27, 3(2001), 409--431.
• 5: Ballard, M.E. and Weist, J.R. Mortal Kombat: The effects of violent video game play on males' hostility and cardiovascular responding. Journal of Applied Social Psychology, 26, (1996), 717--730.
• 6: Carnagey, N.L. and Anderson, C.A. The effects of reward and punishment in violent video games on aggressive affect, cognition, and behavior. Psychological Science, 16, 11 (2005), 882--889.
• 7: Polle T. Zellweger , Niels Olof Bouvin , Henning Jehøj , Jock D. Mackinlay, Fluid annotations in an open world, Proceedings of the 12th ACM conference on Hypertext and Hypermedia, August 14-18, 2001, Århus, none, Denmark [doi>10.1145/504216.504224]
• 8: Scott, D. The effect of video games on feelings of aggression. The Journal of Psychology, 129, (1995), 121--132.
• 9: Niklas Ravaja , Mikko Salminen , Jussi Holopainen , Timo Saari , Jari Laarni , Aki Järvinen, Emotional response patterns and sense of presence during video games: potential criterion variables for game design, Proceedings of the third Nordic conference on Human-computer interaction, p.339-347, October 23-27, 2004, Tampere, Finland [doi>10.1145/1028014.1028068]
• 10: Niklas Ravaja , Timo Saari , Marko Turpeinen , Jari Laarni , Mikko Salminen , Matias Kivikangas, Spatial Presence and Emotions during Video Game Playing: Does It Matter with Whom You Play?, Presence: Teleoperators and Virtual Environments, v.15 n.4, p.381-392, August 2006 [doi>10.1162/pres.15.4.381]
• 11: Nosu, K., Kurokawa, T.A., Horita, H., Ohhazama, Y., and Takeda, K. Real Time Emotion-Diagnosis of Video Game Players from Their Facial Expressions and Its Applications to Voice Feed-Backing to Game Players. Proceedings of the 6th International Conference on Machine Learning and Cybernetics, (2007), 2208--2212.
• 12: Schneider, E.F., Lang, A., Shin, M. Bradley, S.D. Death with a story: How story impacts emotional, motivational, and physiological responses to first-person shooter video games. Human Communication Research, 30, 3 (2004), 361--375.
• 13: Ville Nenonen , Aleksi Lindblad , Ville Häkkinen , Toni Laitinen , Mikko Jouhtio , Perttu Hämäläinen, Using heart rate to control an interactive game, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240752]
• 14: Jun H. Kim , Daniel V. Gunn , Eric Schuh , Bruce Phillips , Randy J. Pagulayan , Dennis Wixon, Tracking real-time user experience (TRUE): a comprehensive instrumentation solution for complex systems, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357126]
• 15: Tao Lin , Masaki Omata , Wanhua Hu , Atsumi Imamiya, Do physiological data relate to traditional usability indexes?, Proceedings of the 17th Australia conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future, November 21-25, 2005, Canberra, Australia
• 16: Lazzaro, N. Why We Play Games: Four Keys to Move Emotion Without Story. XEODesign, 5273 College Ave. Suite 201, Oakland, California, 94618, http://www.xeodesign.com.
• 17: Colin Puri , Leslie Olson , Ioannis Pavlidis , James Levine , Justin Starren, StressCam: non-contact measurement of users' emotional states through thermal imaging, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1057007]
• 18: Zhen Zhu , Panagiotis Tsiamyrtzis , Ioannis Pavlidis, The Segmentation of the Supraorbital Vessels in Thermal Imagery, Proceedings of the 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance, p.237-244, September 01-03, 2008 [doi>10.1109/AVSS.2008.36]
• 19: I. Pavlidis , J. Dowdall , N. Sun , C. Puri , J. Fei , M. Garbey, Interacting with human physiology, Computer Vision and Image Understanding, v.108 n.1-2, p.150-170, October, 2007 [doi>10.1016/j.cviu.2006.11.018]
• 20: Palvidis, I. and Levine, J. Thermal image analysis for polygraph testing. IEEE Engineering in Medicine and Biology Magazine, 21, 6 (2002), 56--64.
• 21: Jonathan Dowdall , Ioannis T. Pavlidis , Panagiotis Tsiamyrtzis, Coalitional tracking, Computer Vision and Image Understanding, v.106 n.2-3, p.205-219, May, 2007 [doi>10.1016/j.cviu.2006.08.011]
• 22: P. Tsiamyrtzis , J. Dowdall , D. Shastri , I. T. Pavlidis , M. G. Frank , P. Ekman, Imaging Facial Physiology for the Detection of Deceit, International Journal of Computer Vision, v.71 n.2, p.197-214, February 2007 [doi>10.1007/s11263-006-6106-y]
• 23: Yamakoshi, T., Yamakoshi, K., Tanaka, S., Nogawa, M., Shibata, M., Sawada, Y., Rolfe, P., Hirose, Y. A Preliminary Study on Driver's Stress Index Using a New Method Based on Differential Skin Temperature Measurement. In Proc of IEEE Conf Eng Med Biol Soc. (2007), 722--725.
• 24: Yamaguchi, M., Wakasugi, J., Sakakima, J. Evaluation of Driver Stress using Biomarker in Motor-vehicle Driving Simulator. In Proc IEEE Conf. Eng Med Biol Soc. (2006), 1834--1837.
• 25: Healey, J.A., Picard, R.W. Detecting stress during real-world driving tasks using physiological sensors. Intelligent Transportation Systems, IEEE Transactions on. June (2005), 156--166. Volume: 6, Issue: 2
• 26: FLIR Systems, 70 Castilian Dr., Goleta, California 93117, http://www.flir.com.
• 27: Quickset International, 3650 Woodhead Drive, Northbrook, IL 60062, http://www.quickset.com
• 28: SBIR Inc, 30 South Calle Cesar Chavez, Santa Barbara, CA 9310, http://www.sbir.com
• 29: Moxham, B. J., Krish C., Berkovitz B., Alusi G., and Cheeseman T., Interactive head&neck (CD--ROM), Primal Pictures, December 2002.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation); Input devices and strategies (e.g., mouse, touchscreen); Prototyping

General Terms:
Design, Experimentation, Human Factors, Performance

Keywords:
game difficulty adjustment, human-computer interaction, stress monitoring, thermal imaging, video games