A continuous and objective evaluation of emotional experience with interactive play environments

ABSTRACT

Researchers are using emerging technologies to develop novel play environments, while established computer and console game markets continue to grow rapidly. Even so, evaluating the success of interactive play environments is still an open research challenge. Both subjective and objective techniques fall short due to limited evaluative bandwidth; there remains no corollary in play environments to task performance with productivity systems. This paper presents a method of modeling user emotional state, based on a user's physiology, for users interacting with play technologies. Modeled emotions are powerful because they capture usability and playability through metrics relevant to ludic experience; account for user emotion; are quantitative and objective; and are represented continuously over a session. Furthermore, our modeled emotions show the same trends as reported emotions for fun, boredom, and excitement; however, the modeled emotions revealed differences between three play conditions, while the differences between the subjective reports failed to reach significance.

REFERENCES

• 1: Interactive digital software association: www.idsa.com.
• 2: Staffan Björk , Jussi Holopainen , Peter Ljungstrand , Regan Mandryk, Special Issue on Ubiquitous Games, Personal and Ubiquitous Computing, v.6 n.5-6, p.358-361, December 2002 [doi>10.1007/s007790200040]
• 3: Boucsein, W. Electrodermal activity. Plenum Press, New York, 1992.
• 4: Cacioppo, J.T., Berntson, G.G., Larsen, J.T., Poehlmann, K.M., and Ito, T.A. The psychophysiology of emotion, in Handbook of emotions, (M. Lewis and J.M. Haviland-Jones, eds.). The Guilford Press: New York, (2000).
• 5: Cacioppo, J.T., and Tassinary, L.G. Inferring psychological significance from physiological signals. American Psychologist, 45,1, (1990), 16--28.
• 6: Daniel Chen , Roel Vertegaal, Using mental load for managing interruptions in physiologically attentive user interfaces, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986103]
• 7: Heather Desurvire , Martin Caplan , Jozsef A. Toth, Using heuristics to evaluate the playability of games, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986102]
• 8: Ekman, P. Basic emotions, in Handbook of cognition and emotion, (T. Dalgleish and M. Power, eds.). John Wiley & Sons, Ltd.: Sussex, (1999).
• 9: Ekman, P., Levenson, R.W., and Friesen, W.V. Autonomic nervous system activity distinguishes among emotions. Science, 221,4616, (1983), 1208--1210.
• 10: Carolanne Fisher , Penelope Sanderson, Exploratory sequential data analysis: exploring continuous observational data, interactions, v.3 n.2, p.25-34, March 1996 [doi>10.1145/227181.227185]
• 11: Fulton, B., and Medlock, M. Beyond focus groups: Getting more useful feedback from consumers. In Proc. Game Dev. Conf., (2003).
• 12: Lang, P.J. The emotion probe. American Psychologist, 50,5, (1995), 372--385.
• 13: Lang, P.J., Greenwald, M.K., Bradley, M.M., and Hamm, A.O. Looking at pictures: Affective, facial, visceral, and behavioural reactions. Psychophysiology, 30,(1993), 261--273.
• 14: Lazzaro, N. Why we play games: 4 keys to more emotion. In Proc. Game Dev. Conf., (2004).
• 15: Levenson, R.W. Autonomic nervous system differences among emotions. American Psychological Society, 3,1, (1992), 23--27.
• 16: Carsten Magerkurth , Adrian David Cheok , Regan L. Mandryk , Trond Nilsen, Pervasive games: bringing computer entertainment back to the real world, Computers in Entertainment (CIE), v.3 n.3, July 2005 [doi>10.1145/1077246.1077257]
• 17: Mandryk, R.L., and Atkins, M.S. (2006). A fuzzy physiological approach for continuously modeling emotional experience during interaction with computer games, Technical Report: Simon Fraser University, Burnaby, BC.
• 18: Regan L. Mandryk , Kori M. Inkpen, Physiological indicators for the evaluation of co-located collaborative play, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031625]
• 19: Mandryk, R.L., Inkpen, K., and Calvert, T.W. Using psychophysiological techniques to measure user experience with entertainment technologies. Behaviour and Information Technology (Special Issue on User Experience), 25,2, (2006), 141--158.
• 20: Marshall, C., and Rossman, G.B. Designing qualitative research. Sage Publications, Thousand Oaks, 1999.
• 21: Jakob Nielsen, Evaluating the thinking-aloud technique for use by computer scientists, Advances in human-computer interaction (vol. 3), Ablex Publishing Corp., Norwood, NJ, 1993
• 22: Don Norman, Emotion & design: attractive things work better, interactions, v.9 n.4, p.36-42, July 2002 [doi>10.1145/543434.543435]
• 23: Randy J. Pagulayan , Kevin Keeker , Dennis Wixon , Ramon L. Romero , Thomas Fuller, User-centered design in games, The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 2002
• 24: Papillo, J.F., and Shapiro, D. The cardiovascular system, in Principles of psychophysiology: Physical, social, and inferential elements, (L.G. Tassinary, ed. Cambridge University Press: Cambridge, (1990), 456--512.
• 25: Partala, T., and Surakka, V. The effects of affective interventions in human-computer interaction. Interacting with Computers, 16,(2004), 295--309.
• 26: Rosalind W. Picard, Affective computing, MIT Press, Cambridge, MA, 1997
• 27: Dennis W. Rowe , John Sibert , Don Irwin, Heart rate variability: indicator of user state as an aid to human-computer interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.480-487, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274709]
• 28: Russell, J.A., Weiss, A., and Mendelsohn, G.A. Affect grid: A single-item scale of pleasure and arousal. Journal of Personality and Social Psychology, 57,3, (1989), 493--502.
• 29: Scheirer, J., Fernandez, R., Klein, J., and Picard, R. Frustrating the user on purpose: A step toward building an affective computer. Interacting with Computers, 14,2, (2002), 93--118.
• 30: Stern, R.M., Ray, W.J., and Quigley, K.S. Psychophysiological recording. Oxford University Press, New York, 2001.
• 31: Penelope Sweetser , Peta Wyeth, GameFlow: a model for evaluating player enjoyment in games, Computers in Entertainment (CIE), v.3 n.3, July 2005 [doi>10.1145/1077246.1077253]
• 32: Kim J. Vicente , D. Craig Thornton , Neville Moray, Spectral analysis of sinus arrhythmia: a measure of mental effort, Human Factors, v.29 n.2, p.171-182, April 1987
• 33: R. D. Ward , P. H. Marsden, Physiological responses to different WEB page designs, International Journal of Human-Computer Studies, v.59 n.1-2, p.199-212, July 2003 [doi>10.1016/S1071-5819(03)00019-3]
• 34: Wilson, G.M., and Sasse, M.A. Do users always know what's good for them? Utilizing physiological responses to assess media quality. In Proc. HCI 2000, Springer (2000), 327--339.
• 35: Wilson, G.M., and Sasse, M.A. Investigating the impact of audio degradations on users: Subjective vs. Objective assessment methods. In Proc. OZCHI 2000, (2000), 135--142.
• 36: Winton, W., Putnam, L., and Krauss, R. Facial and autonomic manifestations of the dimensional structure of emotion. Journal of Experimental Social Psychology, 20,(1984), 195--216.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Keywords:
EMG, GSR, HR, emotion, evaluation methodology, fun, fuzzy logic, games, physiology, play