The soundtrack of your mind: mind music - adaptive audio for game characters

ABSTRACT

In this paper we describe an experimental application for individualized adaptive music for games. Expression of emotion is crucial for increasing believability. Since a fundamental aspect of music is it's ability to express emotions research into the area of believable agents can benefit from exploring how music can be used. In our experiment we use an affective model that can be integrated to player characters. Music is composed to reflect the affective processes of mood, emotion, and sentiment. The composition takes results of empirical studies regarding the influence of different factors in musical structure on perceived musical expression into account. The musical output from the test application varies in harmony and time signature along a matrix of moods, moods that change depending on what emotions are activated during game play.

REFERENCES

• 1: Oz project, 1989 - 2002. Oz Project publications can be found at http://www.cs.cmu.edu/afs/cs.cmu.edu/project/oz/web. URL verified March 7 2006.
• 2: Nice project, 2002 - 2005. NICE Project publication can be found at http://www.niceproject.com/publications. URL verified March 7 2006.
• 3: Star wars galaxies - an empire divided, 2003. Platform: PC. LucasArts http://starwarsgalaxies.station.sony.com.
• 4: World of warcraft, 2004. Platform: PC. http://www.worldofwarcraft.com.
• 5: Integrated project on pervasive gaming (iperg), 2004 - 2008. http://www.pervasive-gaming.org, URL verified March 13 2006.
• 6: J. R. Anderson. A spreading activation theory of memory. Journal of Verbal Learning and Verbal Behaviour, (22):261--295, 1983.
• 7: J. Bates. The role of emotions in believable agents. Technical Report CMU-CS-94-136, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, April 1994.
• 8: A. Cohen. Music, Mind, and Science, chapter The functions of music in multimedia: A cognitive approach, pages 53--69. Seoul National University Press, Seoul, Korea, 1999.
• 9: A. J. Cohen. Music and Emotion - Theory and Research, chapter Music as a Source of Emotion in Film, pages 249--272. Series in Affective Science. Oxford University Press, New York, 2001.
• 10: A. M. Collins and E. F. Loftus. A spreading activation theory of semantic processing. Psychological Rewview, (82):407--428, 1975.
• 11: S. Davies. Music and Emotion - Theory and Research, chapter Philosophical Perspectives on Music's Expressiveness, pages 23--44. Series in Affective Science. Oxford University Press, New York, 2001.
• 12: A. Egges, S. Kshirsagar, and N. Magnenat-Thalmann. Lecture Notes in Computer Science, volume 2773, chapter A Model for Personality and Emotion Simulation, pages 453--461. Springer Berlin / Heidelberg, 2003.
• 13: Arjan Egges , Sumedha Kshirsagar , Nadia Magnenat-Thalmann, Generic personality and emotion simulation for conversational agents: Research Articles, Computer Animation and Virtual Worlds, v.15 n.1, p.1-13, March 2004 [doi>10.1002/cav.v15:1]
• 14: P. Ekman. The Nature of Emotion, chapter All emotions are basic, pages 15 - 19. Oxford University Press, 1994.
• 15: M. El Jed, N. Pallamin, J. Dugdale, and B. Pavard. Modelling character emotion in an interactive virtual environment. In proceedings of AISB 2004 Symposium: Motion, Emotion and Cognition, Leeds, UK, 29 March 1 April 2004.
• 16: N. Frijda. The Nature of Emotion, chapter Varieties of Affect: Emotions and Episodes, Moods and Sentiments. Oxford University Press, 1994.
• 17: A. Gabrielsson and E. Lindström. Music and Emotion - Theory and Research, chapter The Influence of Musical Structure on Emotional Expression, pages 223--248. Series in Affective Science. Oxford University Press, New York, 2001.
• 18: Jan Berg , Johnny Wingstedt, Relations between selected musical parameters and expressed emotions: extending the potential of computer entertainment, Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology, p.164-171, June 15-17, 2005, Valencia, Spain [doi>10.1145/1178477.1178499]
• 19: P. N. Jusling and J. A. Sloboda, editors. Music and Emotion - Theory and Research. Series in Affective Science. Oxford University Press, New York, 2001.
• 20: Steven R. Livingstone , Andrew R. Brown, Dynamic response: real-time adaptation for music emotion, Proceedings of the second Australasian conference on Interactive entertainment, p.105-111, November 23-25, 2005, Sydney, Australia
• 21: R. McCrae and P. Costa. Validation of the five-factor model of personality across instruments and observers. Journal of Personality and Social Psychology, (52):81--90, 1987.
• 22: L. Meyer. Emotion and Meaning in Music. The University of Chicago Press, 1956.
• 23: Microsoft. Directmusic.
• 24: Marvin Minsky, The society of mind, Simon & Schuster, Inc., New York, NY, 1986
• 25: Dave Moffat, Personality Parameters and Programs, Creating Personalities for Synthetic Actors, Towards Autonomous Personality Agents, p.120-165, January 1997
• 26: D. L. Nathanson. Shame and pride: affect, sex and the birth of the self. W. W. Norton & Company, 1992.
• 27: M. Pignatiello, C. Camp, and L. Rasar. Musical mood induction: An alternative to the velten technique. Journal of Abnormal Pychology, (95):295--297, 1986.
• 28: M. R. Quillian. Semantic Information Processing, chapter Semantic Memory, pages 216--260. MIT Press, Cambridge, 1968.
• 29: J. Russell. A circumplex model of affect. Journal of Personality and Social Psychology, (39):345--356, 1980.
• 30: Robyn Taylor , Daniel Torres , Pierre Boulanger, Using music to interact with a virtual character, Proceedings of the 2005 conference on New interfaces for musical expression, May 26-28, 2005, Vancouver, Canada
• 31: S. Tomkins. Affect/imagery/consciousness. Vol. 1: The positive affects., volume 1. Springer, New York, 1962.
• 32: S. Tomkins. Affect/imagery/consciousness. Vol. 2: The negative affects., volume 2. Springer, New York, 1963.
• 33: Johnny Wingstedt , Mats Liljedahl , Stefan Lindberg , Jan Berg, REMUPP: an interactive tool for investigating musical properties and relations, Proceedings of the 2005 conference on New interfaces for musical expression, May 26-28, 2005, Vancouver, Canada
• 34: G. Withmore. Design with music in mind: A guide to adaptive audio for game designers. Gamasutra, May 29 2003. URL verified March 6 2006.

INDEX TERMS

Primary Classification:
  J. Computer Applications
  J.5 ARTS AND HUMANITIES
      Subjects: Performing arts (e.g., dance, music)

Keywords:
adaptive audio, believable agents, game, game design, music, role playing games