Performance visualization: a new challenge to music through visualization

ABSTRACT

This paper describes performance visualization with its purposes, significance, and our prototype system. Performance visualization provides us with a new user interface for music systems, a non-subjective explanation and judgment for musical expression, and a visual aid to understand, analyze, and compare performances and their musical structures. It will also introduce a more challenging research on designing a musical data mining interface not by tags nor by contents but by the mood. Other challenges include finding qualitative characteristics of music with analysis methods for information visualization and synesthesia between visualized and sonified information. We will describe which information should be shown on the visualized figures for expressive performance. Then we will show our prototype system to visualize expressive performance consisting of several instruments.

REFERENCES

• 1: R. B. Dannenberg. Music representation issues, techniques, and systems. Computer Music Journal, 17(3):20--30, 1993.
• 2: Jonathan Foote, Visualizing music and audio using self-similarity, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.77-80, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319472]
• 3: Rumi Hiraga, Case study: a look of performance expression, Proceedings of the conference on Visualization '02, October 27-November 01, 2002, Boston, Massachusetts
• 4: R. Hiraga, S. Igarashi, and Y. Matsuura. Visualized music expression in an object-oriented environment. In Proc. of ICMC, pages 483--486, 1996.
• 5: R. Hiraga, F. Watanabe, and I. Fujishiro. Music learning through visualization. In Proc. of WEB Delivering Music, 2002.
• 6: F. Lerdahl and R. Jackendoff. A Generative Theory of Tonal Music. The MIT Press, 1996.
• 7: R. Miyazaki and I. Fujishiro. Interactive poster: 3d visualization of midi dataset. In DVD of IEEE Visualization, 2002.
• 8: E. Narmour. The Analysis and Cognition of Melodic Complexity: The Implication-Realization Model. The University of Chicago Press, 1992.
• 9: RENCON. http://shouchan.ei.tuat.ac.jp/-rencon/en/index.html.
• 10: George G. Robertson , Jock D. Mackinlay , Stuart K. Card, Cone Trees: animated 3D visualizations of hierarchical information, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.189-194, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108883]
• 11: Ben Shneiderman, Designing the User Interface: Strategies for Effective Human-Computer Interaction, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1997
• 12: Sean M. Smith , Glen N. Williams, A visualization of music, Proceedings of the 8th conference on Visualization '97, p.499-ff., October 18-24, 1997, Phoenix, Arizona, United States

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.5 Sound and Music Computing

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.8 Applications

Keywords:
performance visualization