Visualization of music performance as an aid to listener's comprehension

ABSTRACT

We present a new method for visualizing musical expressions with a special focus on the three major elements of tempo change, dynamics change, and articulation. We have represented tempo change as a horizontal interval delimited by vertical lines, while dynamics change and articulation within the interval are represented by the height and width of a bar, respectively. Then we grouped local expression into several groups by k-means clustering based on the values of the elements. The resulting groups represented the emotional expression in a performance that is controlled by the rhythmic and melodic structure, which controls the gray scale of the graphical components. We ran a pilot experiment to test the effectiveness of our method using two matching tasks and a questionnaire. In the first task, we used the same section of music, played by two different interpretations, while in the second task, two different sections of a performance were used. The results of the test seem to support the present approach, although there is still room for further improvement that will reflect the subtleties in performance.

REFERENCES

• 1: Roberto Bresin , Anders Friberg, Emotional Coloring of Computer-Controlled Music Performances, Computer Music Journal, v.24 n.4, p.44-63, December 2000 [doi>10.1162/014892600559515]
• 2: S. Dixon, W. Goebl, and G. Widmer. The performance worm: Real time visualization of expression based on langner's tempo-laudness animation. In Proc. of ICMC, pages 361--364. ICMA, 2002.
• 3: Rumi Hiraga, Case study: a look of performance expression, Proceedings of the conference on Visualization '02, October 27-November 01, 2002, Boston, Massachusetts
• 4: R. Hiraga, S. Igarashi, and Y. Matsuura. Visualized music expression in an object-oriented environment. In Proc. of ICMC, pages 483--486. ICMA, 1996.
• 5: Rumi Hiraga , Reiko Mizaki , Issei Fujishiro, Performance visualization: a new challenge to music through visualization, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641054]
• 6: Tapas Kanungo , David M. Mount , Nathan S. Netanyahu , Christine Piatko , Ruth Silverman , Angela Y. Wu, The analysis of a simple k-means clustering algorithm, Proceedings of the sixteenth annual symposium on Computational geometry, p.100-109, June 12-14, 2000, Clear Water Bay, Kowloon, Hong Kong [doi>10.1145/336154.336189]
• 7: F. Lerdahl and R. Jackendoff. A Generative Theory of Tonal Music. The MIT Press, 1996.
• 8: S. Müller and G. Mazzola. The extraction of expressive shaping in performance. Computer Music Journal, 27(1):47--58, 2003.
• 9: E. Narmour. The Analysis and Cognition of Melodic Complexity: The Implication-Realization Model. The University of Chicago Press, 1992.
• 10: D. Oppenheim. Compositional tools for adding expression to music. In Proc. of ICMC, pages 223--226. ICMA, 1992.
• 11: A. Watanabe and I. Fujishiro. tutti: A 3d interactive interface for browsing and editing sound data. In Proc. of The 9th Workshop on Interactive System and Software. Japan Society for Software Science and Technology, 2001.
• 12: F. Watanabe, R. Hiraga, and I. Fujishiro. Brass: Visualizing scores for assiting music learning. In Proc. of ICMC, pages 107--114. International Computer Music Association, 2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

Additional Classification:
  J. Computer Applications
  J.5 ARTS AND HUMANITIES