Ambient Display using Musical Effects

ABSTRACT

The paper presents a novel approach to the peripheral display of information by applying audio effects to an arbitrary selection of music. We examine a specific instance: the communication of information about human affect, and construct a functioning prototype which captures behavioral activity level from the face and maps it to musical effects. Several audio effects are empirically evaluated as to their suitability for ambient display. We report measurements of the ambience, perceived affect, and pleasure of these effects. The findings support the hypothesis that musical effects are a promising method for ambient informational display.

REFERENCES

• 1: Elizabeth D. Mynatt , Maribeth Back , Roy Want , Michael Baer , Jason B. Ellis, Designing audio aura, Proceedings of the SIGCHI conference on Human factors in computing systems, p.566-573, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274720]
• 2: Kilander, F. and Lonnqvist, P. A whisper in the woods - an ambient soundscape for peripheral awareness of remote processes. International Conference on Auditory Display. (2002).
• 3: Andreas Butz , Ralf Jung, Seamless user notification in ambient soundscapes, Proceedings of the 10th international conference on Intelligent user interfaces, January 10-13, 2005, San Diego, California, USA [doi>10.1145/1040830.1040914]
• 4: Russell, J.A. Core affect and the psychological construction of emotion. Psychological Review 110, 1 (2003), 145--172.
• 5: Jennifer Mankoff , Anind K. Dey , Gary Hsieh , Julie Kientz , Scott Lederer , Morgan Ames, Heuristic evaluation of ambient displays, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642642]
• 6: Mathias Funk , Kazuhiro Kuwabara , Michael J. Lyons, Sonification of facial actions for musical expression, Proceedings of the 2005 conference on New interfaces for musical expression, May 26-28, 2005, Vancouver, Canada
• 7: Max/MSP. Cycling 74. http://www.cycling74.com
• 8: Jehan, T. Event-Synchronous Music Analysis/Synthesis Proceedings, International Conference on Digital Audio Effects (2004).

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Auditory (non-speech) feedback

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Video analysis

General Terms:
Design, Experimentation, Human Factors, Theory

Keywords:
affective computing, ambient display, musical interface