Eavesdropping: audience interaction in networked audio performance

ABSTRACT

Eavesdropping is an internet-based, interactive audio system that explores network mediated, musical performance in shared public spaces. The project aims to develop an environment which increases audience interaction and connectedness in a localized, computer-controlled performance. The system is a client-server architecture made of three components: (1) an audio preparation interface, (2) an interactive performance interface, and (3) a machine learning-based conductor. An artificial conductor mixes an acoustic ecology based on mood data entered by participants while learning from their feedback. Technicalities and early evaluation are presented.

REFERENCES

• 1: Paul Dourish , Sara Bly, Portholes: supporting awareness in a distributed work group, Proceedings of the SIGCHI conference on Human factors in computing systems, p.541-547, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142982]
• 2: Panos Markopoulos , Natalia Romero , Joy van Baren , Wijnand IJsselsteijn , Boris de Ruyter , Babak Farshchian, Keeping in touch with the family: home and away with the ASTRA awareness system, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986062]
• 3: Truax, B. Acoustic communication. Ablex Publishing, London, 2001.
• 4: Truax, B. Handbook for acoustic ecology. Cambridge Street Publishing, London, 1999.
• 5: Gaver, W.W., What in the world do we hear? An ecological approach to auditory event perception. Ecological Psychology 5,1 (1993). 1--29.
• 6: Ron Wakkary , Marek Hatala , Ying Jiang , Milena Droumeva , Malahat Hosseini, Making sense of group interaction in an ambient intelligent environment for physical play, Proceedings of the 2nd international conference on Tangible and embedded interaction, February 18-20, 2008, Bonn, Germany [doi>10.1145/1347390.1347431]
• 7: Vickers, P. and Hogg, B., "Sonification abstraite/sonification concrete: An 'aesthetic perspective space' for classifying auditory displays in the ars musica domain," presented at International Conference on Auditory Display (ICAD2006), London, UK, 2006.
• 8: Ralph Schroeder , Ann-Sofie Axelsson, Avatars at Work and Play: Collaboration and Interaction in Shared Virtual Environments (Computer Supported Cooperative Work), Springer-Verlag New York, Inc., Secaucus, NJ, 2006
• 9: Laura Dabbish , Robert Kraut, Coordinating communication: awareness displays and interruption, CHI '03 extended abstracts on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/765891.765991]
• 10: James L. Alty, Can we use music in computer-human communication?, Proceedings of the HCI'95 conference on People and computers X, p.409-423, April 1996, Huddersfield, United Kingdom
• 11: R.E. Thayer. 1989. The Biopsychology of Mood and Arousal. Oxford University Press.
• 12: Liu, D., Lu, L. and Zhang, H. Automatic mood detection from acoustic music data. In Proceedings of the International Symposium on Music Information Retrieval (ISMIR'03) (Baltimore, MD, USA, October 26-30 2003).
• 13: Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
• 14: Stuart J. Russell , Peter Norvig, Artificial intelligence: a modern approach, Prentice-Hall, Inc., Upper Saddle River, NJ, 1995
• 15: B. LaBelle. 2006. Background Noise: Perspectives on Sound Art. New York and London: Continuum International Publishing Group.
• 16: Thomas Erickson , Wendy A. Kellogg, Social translucence: an approach to designing systems that support social processes, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.1, p.59-83, March 2000 [doi>10.1145/344949.345004]
• 17: Manovich, L. The anti-sublime in new media. Chair Et Metal/Metal and Flesh (Fall, 2002),
• 18: Whitelaw, M. Hearing pure data: Aesthetics and ideals of data-sound, In Unsorted: Thoughts on the Information Arts: An A to Z for Sonic Acts X Altena,A., Ed.
• 19: Kivy. P. 1989. Sound Sentiment: An Essay on the Musical Emotions. Philadelphia, PA: Temple University Press.
• 20: Jan Berg , Johnny Wingstedt, Relations between selected musical parameters and expressed emotions: extending the potential of computer entertainment, Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology, p.164-171, June 15-17, 2005, Valencia, Spain [doi>10.1145/1178477.1178499]
• 21: Scherer, K.R. and Zentner, M.R., Emotional effects of music: Production rules. In Music and Emotion: Theory and Research. Juslin, P.N. and Sloboda, J.A., Eds. Oxford University Press, Oxford, England, 2001, 361--392.
• 22: Weinberg, G. The aesthetics, history and future challenges of interconnected music networks. In Proceedings of the International Computer Music Association Conference (Goteborg, Sweden, Sept. 2002). 2002, 349--356.
• 23: Bischoff, J., Gold, R. and Horton, J. Music for an interactive network of microcomputers. Computer Music Journal 2, 3 (Dec, 1978), 24--29.
• 24: Gresham-Lancaster, S. The aesthetics and history of the hub: The effects of changing technology on network computer music. Leonardo Music Journal 8 (1998), 39--44.
• 25: Barbosa, A. Displaced soundscapes: A survey of network systems for music and sonic art creation. Leonardo Music Journal 13 (2003), 53--59.
• 26: Brown, C. Talking drum: A local area network music installation. Leonardo Music Journal 9 (1999), 23--28.
• 27: van Baren, J., IJsselsteijn, W.A., Romero, N., Markopoulos, P. and de Ruyter, B., Affective benefits in communication: The development and field--testing of a new questionnaire measure. In PRESENCE 2003 (Aalborg, Denmark, October 2003). 200.

INDEX TERMS

Primary Classification:
  J. Computer Applications
  J.5 ARTS AND HUMANITIES
      Subjects: Performing arts (e.g., dance, music)

General Terms:
Algorithms, Human Factors, Performance

Keywords:
acoustic ecology, artificial intelligence, auditory display, computer music, net art, reinforcement learning