MuSA.RT: music on the spiral array. real-time

ABSTRACT

We present MuSA.RT, Opus 1, a multimodal interactive system for music analysis and visualization using the Spiral Array model. Real-time MIDI input from a live performance is processed, analyzed and mapped to the 3D model, revealing tonal structures such as pitches, chords and keys. A user can concurrently navigate through the Spiral Array space using a gamepad or set the camera control to automatic pilot. The interaction among and concurrent processing of the different data streams is made possible through the Modular Flow Scheduling Middleware.

REFERENCES

• 1: E. Chew. Towards a Mathematical Model of Tonality. Ph.d. thesis, Massachusetts Institute of Technology, Cambridge, MA, 2000.
• 2: E. Chew. Modeling tonality: Applications to music cognition. In Proc. 23rd Annual Meeting of the Cognitive Science Society, Edinburgh, Scotland, 2001.
• 3: E. Chew and Y.-C. Chen. Mapping midi to the spiral array: Disambiguating pitch spellings. In Proc. 8th INFORMS Computer Society Conference, pages 259--275, Chandler, AZ, 2003.
• 4: A. R. J. Francois. Software Architecture for Computer Vision. In G. Medioni and S. Kang, editors, Emerging Topics in Computer Vision. Prentice Hall, 2004.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.5 Sound and Music Computing
          Subjects: Systems

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.5 Implementation
          Subjects: Interactive systems

  J. Computer Applications
  J.5 ARTS AND HUMANITIES
      Subjects: Performing arts (e.g., dance, music)

General Terms:
Algorithms, Design, Experimentation, Human Factors, Theory

Keywords:
MFSM, SAI, music analysis, music visualization, spiral array, system implementation