An immersive, multi-user, musical stage environment

ABSTRACT

A multi-user, polyphonic sensor stage environment that maps position and gestures of up to four performers to the pitch and articulation of distinct notes is presented. The design seeks to provide multiple players on a stage with the feeling of a traditional acoustic instrument by giving them complete control over the instrument's expressive parameters and a clear causal connection between their actions and the resulting sound. The positions of the performers are determined by a custom ultrasonic tracking system, while hand motions are measured by custom-made gloves containing accelerometer units. Furthermore, juggling clubs are illuminated dynamically to make complex juggling patterns more apparent. The system is currently on tour with the Flying Karamazov Brothers juggling troupe.

REFERENCES

• 1: David Bianciardi and Ryan Ulyate. Interactive Dance Club, 1998. ACM SIGGRAPH.
• 2: Ben Denckla and Patrick Pelletier. The technical documentation for Rogus McBogus, a MIDI library, 1996. Available at http://www.media.mit.edu/hyperins/rogus/home.html.
• 3: Eric Foxlin , Michael Harrington , George Pfeifer, Constellation: a wide-range wireless motion-tracking system for augmented reality and virtual set applications, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.371-378, July 1998 [doi>10.1145/280814.280937]
• 4: Greg Welch , Gary Bishop , Leandra Vicci , Stephen Brumback , Kurtis Keller , D'nardo Colucci, The HiBall Tracker: high-performance wide-area tracking for virtual and augmented environments, Proceedings of the ACM symposium on Virtual reality software and technology, p.1-ff., December 20-22, 1999, London, United Kingdom [doi>10.1145/323663.323664]
• 5: Ralf Gelhaar. Sound=space: an interactive musical environement. Contemporary Music Review, 6(1):59-72, 1991.
• 6: Neil Gershenfeld, The nature of mathematical modeling, Cambridge University Press, New York, NY, 1999
• 7: O. Omojola , E. Rehmi Post , M. D. Hancher , Y. Maguire , R. Pappu , B. Schoner , P. R. Russo , R. Fletcher , N. Gershenfeld, An installation of interactive furniture, IBM Systems Journal, v.39 n.3-4, p.861-879, July 2000
• 8: Joseph A. Paradiso , Kai Yuh Hsiao , Ari Benbasat, Interfacing to the foot: apparatus and applications, CHI '00 extended abstracts on Human factors in computing systems, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/633292.633389]
• 9: J. Paradiso and F. Sparacino. Optical tracking for music and dance performance. In A. Gruen and H. Kahmen, editors, Optical 3-D measurement techniques IV, pages 11-18. Herbert Wichmann Verlag, Heidelberg, 1997.
• 10: Joseph A. Paradiso, Electronic music: new ways to play, IEEE Spectrum, v.34 n.12, p.18-30, Dec., 1997 [doi>10.1109/6.642965]
• 11: Joseph A. Paradiso. The brain opera technology: New instruments and gestural sensors for musical interaction and performance. Journal of New Music Research, 28(2):130-149, 1999.
• 12: Michel Waisvisz. The hands. In Proceedings International Computer Music Conference, pages 313-318, 1985.
• 13: M. Wanderly and M. Battier, editors. Trends in Gestural Control of Music. Editions IRCAM, Paris, 2000.
• 14: Greg Welch , Gary Bishop, SCAAT: incremental tracking with incomplete information, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.333-344, August 1997 [doi>10.1145/258734.258876]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING

Additional Classification:
  B. Hardware
  B.4 INPUT/OUTPUT AND DATA COMMUNICATIONS

  J. Computer Applications
  J.5 ARTS AND HUMANITIES
      Subjects: Performing arts (e.g., dance, music)

General Terms:
Experimentation, Performance

Keywords:
HCI, applications, object tracking, spatialized sound, user interface design