Interactive multi-scale visualizations of tonal evolution in MuSA.RT Opus 2

ABSTRACT

The purpose of this article is to describe MuSA.RT Opus 2, an interactive system for tonal visualization of music at multiple scales, and to present examples of the types of musical features and attributes that can be abstracted and visualized by the system. MuSA.RT aims to create an environment by which musical performances can be mapped in real-time to a concrete and visual metaphor for tonal space, wherein we can see the establishment and evolution of the tonal context. In this environment, expert musicians will be able to see the tonal structures of what they play, initiated listeners will be able to visually follow the structures that they hear, and novices can learn to hear the structures that they see. MuSA.RT is both an interactive art installation that can convert musical performances to mathematically elegant graphics and a scientific tool for visualizing the inner workings of tonal induction and tracking algorithms. In this article we describe the mapping strategies for transforming a MIDI stream into tonal structures in 3D space, and our solution for overcoming the challenge of real-time concurrent processing of data streams; we will also give examples and present case studies of visual mappings of music by Pachelbel, Bach, and Barber.

REFERENCES

• 1: Chew, E. 2000. Towards a mathematical model of tonality. Ph.D. dissertation, Massachusetts Institute of Technology, Cambridge, MA.
• 2: Chew, E. 2005/06. Slicing it all ways: Mathematical models for induction, approximation and segmentation using the spiral array. INFORMS J. Computing. To appear.
• 3: Chew, E. 2005. Thinking out of the grid and inside the spiral - Geometric interpretations of and comparisons with the spiral array model. Computing in Musicology 15 (2005). Special issue on harmonic and rhythmic analysis. To appear.
• 4: Elaine Chew , Yun-Ching Chen, Real-Time Pitch Spelling Using the Spiral Array, Computer Music Journal, v.29 n.2, p.61-76, June 2005 [doi>10.1162/0148926054094378]
• 5: Chew, E. And Chen, Y.-C. 2003b. Determining context-defining windows: Pitch spelling using the spiral array. In Proceedings of the 4th International Conference for Music Information Retrieval (ISMIR, Baltimore, MD, Oct. 26-30, 2003).
• 6: Chew, E. And Chen, Y.-C. 2003a. Mapping MIDI to the spiral array: Disambiguating pitch spellings. In Computational Modeling and Problem Solving in the Networked World, Proceedings of the 8th INFORMS Computer Society Conference (ICS2003, Chandler, AZ, Jan 8-10, 2003). Kluwer Academic. 259-275.
• 7: Elaine Chew , Alexandre R.J. Francois, MuSA.RT: music on the spiral array. real-time, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957106]
• 8: Alexandre R. J. Francois, A Hybrid Architectural Style for Distributed Parallel Processing of Generic Data Streams, Proceedings of the 26th International Conference on Software Engineering, p.367-376, May 23-28, 2004
• 9: Francois, A. R. J. 2002. Components for immersion. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME '02, Lausanne, Switzerland, Aug. 2002).
• 10: Francois, A. R. J. 2001. Modular flow scheduling framework (MFSM): mfsm.sourceforge.net.
• 11: Krumhansl, C. L. 1990. Cognitive Foundations of Musical Pitch. Oxford University Press. Oxford, England.
• 12: Sapp, C. S. 2001. Harmonic visualizations of tonal music." In Proceedings of the International Computer Music Conference (2001). 423-430. Tonal Landscape Gallery. ccrma.stanford.edu/~craig/keyscape.
• 13: Toiviainen, P. And Krumhansl, C. L. 2003. Measuring and modeling real-time responses to music: The dynamics of tonality induction. Perception 32, 6 (2003), 741-766.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.5 Sound and Music Computing

General Terms:
Algorithms, Design, Experimentation, Human Factors, Theory

Keywords:
computational music cognition, geometric models, harmonic analysis, music visualization, tonal analysis, tonality models