An interface for melody input

ABSTRACT

We present a software system, called Tunserver, which recognizes a musical tune whistled by the user, finds it in a database, and returns its name, composer, and other information. Such a service is useful for track retrieval at radio stations, music stores, etc., and is also a step toward the long-term goal of communicating with a computer much like one would with a human being. Tuneserver is implemented as a public Java-based WWW service with a database of approximately 10,000 motifs. Tune recognition is based on a highly error-resistant encoding, proposed by Parsons, that uses only the direction of the melody, ignoring the size of intervals as well as rhythm. We present the design and implementation of the tune recognition core, outline the design of the Web service, and describe the results obtained in an empirical evaluation of the new interface, including the derivation of suitable system parameters, resulting performance figures, and an error analysis.

REFERENCES

• 1: BARLOW,H.AND MORGENSTERN, S. 1948. A Directory of Musical Themes. Crown Publishers, New York, NY.
• 2: BENOIT, C., PELACHAUD,C.,AND SUHM, B. 1999. Multimodal speech systems. In D. Gibbons, R. Moore, and R. Winski Eds., Handbook of Standards and Resources in Spoken Language Systems, Volume supplement. Berlin: Mouton de Gruyter. http://werner.ira.uka.de/ }} bsuhm/.
• 3: Jan O. Borchers, WorldBeat: designing a baton-based interface for an interactive music exhibit, Proceedings of the SIGCHI conference on Human factors in computing systems, p.131-138, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258632]
• 4: COLE, R. A., MARIANI, J., USZKOREIT, H., ZAENEN, A., AND ZUE, V. 1995. Survey of the State of the Art in Human Language Technology. Center for Spoken Language Understanding CSLU, Carnegie Mellon University, Pittsburgh, PA.
• 5: DELLAERT, F., POLZIN,T.,AND WAIBEL, A. 1996. Recognizing emotion in speech. In Proc. Intl. Conf. on Spoken Language Processing (Philadelphia, Oct. 1996), 1970-1973.
• 6: Asif Ghias , Jonathan Logan , David Chamberlin , Brian C. Smith, Query by humming: musical information retrieval in an audio database, Proceedings of the third ACM international conference on Multimedia, p.231-236, November 05-09, 1995, San Francisco, California, United States [doi>10.1145/217279.215273]
• 7: Robert J. K. Jacob, Human-computer interaction: input devices, ACM Computing Surveys (CSUR), v.28 n.1, p.177-179, March 1996 [doi>10.1145/234313.234387]
• 8: S. Manke , M. Finke , A. Waibel, NPen/sup ++/: a writer independent, large vocabulary on-line cursive handwriting recognition system, Proceedings of the Third International Conference on Document Analysis and Recognition (Volume 1), p.403, August 14-15, 1995
• 9: MCNAB,R.J.,SMITH, L. A., BAINBRIDGE,D.,AND WITTEN, I. H. 1997. The New Zealand digital library MELody inDEX. D-Lib magazine 3, 5 (May). www.dlib.org.
• 10: Brad Myers , Jim Hollan , Isabel Cruz , Steve Bryson , Dick Bulterman , Tiziana Catarci , Wayne Citrin , Ephraim Glinert , Jonathan Grudin , Yannis Ioannidis, Strategic directions in human-computer interaction, ACM Computing Surveys (CSUR), v.28 n.4, p.794-809, Dec. 1996 [doi>10.1145/242223.246855]
• 11: PARSONS, D. 1975. The Directory of Tunes and Musical Themes. Spencer Brown.
• 12: Raj Reddy, Grand challenges in AI, ACM Computing Surveys (CSUR), v.27 n.3, p.301-303, Sept. 1995 [doi>10.1145/212094.212097]
• 13: STIEFELHAGEN, R., YANG,J.,AND WAIBEL, A. 1997. A model-based gaze tracking system. Intl. Journal of Artificial Intelligence Tools 6, 2, 193-209.
• 14: David G. Stork , Marcus E. Hennecke, Invited Speech: "Speechreading: An Overview of Image Processing, Feature Extraction, Sensory Intergration and Pattern Recognition Techiques, Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition (FG '96), p.2, October 14-16, 1996
• 15: UKKONEN, E. 1985. Finding approximate patterns in strings. Journal of Algorithms 6, 132-137.
• 16: Ian H. Witten , Craig Nevill-Manning , Rodger McNab , Sally Jo Cunningham, A public library based on full-text retrieval, Communications of the ACM, v.41 n.4, p.71-75, April 1998 [doi>10.1145/273035.273057]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)

Additional Classification:
  E. Data
  E.4 CODING AND INFORMATION THEORY
      Subjects: Error control codes

  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human information processing
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Selection process

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
  I.5 PATTERN RECOGNITION
      I.5.4 Applications
          Subjects: Signal processing

  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY

General Terms:
Algorithms, Human Factors, Performance

Keywords:
input mode, melody, motif, recognition, theme, tune