Social active listening and making of expressive music: the interactive piece the bow is bent and drawn

ABSTRACT

This paper discusses the concepts, the interaction paradigms, the system design and implementation that have been developed for the interactive dance and music performance The Bow is bent and drawn (composer Nicola Ferrari), presented at Casa Paganini, Genova, Italy, in occasion of the opening concert of the 8^th Intl. Conference on New Interfaces for Musical Expression (NIME08), June 4, 2008. The Bow is bent and drawn grounds its bases on current research at Casa Paganini -- InfoMus Lab on social active listening of sound and music content and on analysis and processing of expressiveness in human full-body movement and gesture. In particular, The Bow is bent and drawn exploits our recent system Mappe per Affetti Erranti (literally Maps for Wandering Affects), which enables a novel paradigm for social active experience and dynamic molding of expressive content of a music piece. In Mappe per Affetti Erranti multiple users can physically navigate a polyphonic music piece and can intervene in real-time on the expressive content music performance conveys through their full-body movement and gesture. The research topics addressed in this paper are currently investigated in the EUICT Project SAME (Sound and Music for Everyone, Everyday, Everywhere, Every Way, www.sameproject.eu).

REFERENCES

• 1: Robert Rowe, Interactive music systems: machine listening and composing, MIT Press, Cambridge, MA, 1992
• 2: Camurri, A., Canepa, C., Coletta, P., Mazzarino, B., and Volpe, G. 2008 Mappe per Affetti Erranti: a Multimodal System for Social Active Listening and Expressive Performance. In Proceedings of the 8^th Intl. Conference on New Interfaces for Musical Expression (Genova, Italy, June 2008). NIME08.
• 3: Antonio Camurri , Corrado Canepa , Gualtiero Volpe, Active listening to a virtual orchestra through an expressive gestural interface: the orchestra explorer, Proceedings of the 7th international conference on New interfaces for musical expression, June 06-10, 2007, New York, New York [doi>10.1145/1279740.1279748]
• 4: Russell, J. A. 1980. A circumplex model of affect, Journal of Personality and Social Psychology, 39 (1980), 1161--1178.
• 5: Tellegen, A., Watson, D., and Clark, L. A. 1999. On the dimensional and hierarchical structure of affect. Psychological Science, 10, 4 (Jul 1999), 297--303.
• 6: Juslin, P. N. 2000. Cue utilization in communication of emotion in music performance: relating performance to perception. Journal of Experimental Psychology: Human Perception and Performance, 26, 6 (2000), 1797--1813.
• 7: Sergio Canazza , Giovanni De Poli , Carlo Drioli , Antonio Rodà , Alvise Vidolin, Audio Morphing Different Expressive Intentions for Multimedia Systems, IEEE MultiMedia, v.7 n.3, p.79-83, July 2000
• 8: Vines, B. W., Krumhansl, C. L., Wanderley, M. M., Ioana, M. D., and Levitin, D. J. 2005. Dimensions of Emotion in Expressive Musical Performance. Ann. N. Y. Acad. Sci., 1060 (2005), 462--466.
• 9: Antonio Camurri , Ingrid Lagerlöf , Gualtiero Volpe, Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques, International Journal of Human-Computer Studies, v.59 n.1-2, p.213-225, July 2003 [doi>10.1016/S1071-5819(03)00050-8]
• 10: Laban, R., and Lawrence, F. C. 1947. Effort. Macdonald & Evans Ltd., London.
• 11: Laban, R. 1963. Modern Educational Dance. Macdonald & Evans Ltd., London.
• 12: Boone, R. T., and Cunningham, J. G. 1998. Children's decoding of emotion in expressive body movement: The development of cue attunement, Developmental Psychology, 34 (1998), 1007--1016.
• 13: Kilian, J. 2001. Simple Image Analysis By Moments. Open Computer Vision (OpenCV) Library documentation.
• 14: Antonio Camurri , Gualtiero Volpe , Giovanni De Poli , Marc Leman, Communicating Expressiveness and Affect in Multimodal Interactive Systems, IEEE MultiMedia, v.12 n.1, p.43-53, January 2005 [doi>10.1109/MMUL.2005.2]
• 15: Camurri, A., Mazzarino, B., Ricchetti, M., Timmers, R., and Volpe, G. 2994. Multimodal analysis of expressive gesture in music and dance performances. In Gesture-based Communication in Human-Computer Interaction, A. Camurri, and G. Volpe Eds, LNAI 2915, Springer Verlag, 20--39.
• 16: Wallbott, H. G. 1998. Bodily expression of emotion, European Journal of Social Psychology, 28 (1998), 879--896.
• 17: Argyle, M., 1980. Bodily Communication, Methuen & Co Ltd, London.
• 18: De Meijer, M. 1989. The contribution of general features of body movement to the attribution of emotions, Journal of Nonverbal Behavior, 13 (1989), 247--268.
• 19: Camurri, A., Coletta, P., Demurtas, M., Peri, M., Ricci, A., Sagoleo, R., Simonetti, M., Varni, G., and Volpe, G. 2007. A Platform for Real-Time Multimodal Processing. In Proceedings International Conference Sound and Music Computing 2007 (Lefkada, Greece, July 2007), SMC2007.
• 20: Ginevra Castellano , Roberto Bresin , Antonio Camurri , Gualtiero Volpe, User-Centered Control of Audio and Visual Expressive Feedback by Full-Body Movements, Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction, September 12-14, 2007, Lisbon, Portugal [doi>10.1007/978-3-540-74889-2_44]
• 21: Anders Friberg, pDM: An Expressive Sequencer with Real-Time Control of the KTH Music-Performance Rules, Computer Music Journal, v.30 n.1, p.37-48, March 2006 [doi>10.1162/014892606776021308]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.5 Sound and Music Computing
          Subjects: Systems

General Terms:
Algorithms, Design, Human Factors, Measurement

Keywords:
active listening of music, expressive interfaces, full-body motion