Music structure based vector space retrieval

ABSTRACT

This paper proposes a novel framework for music content indexing and retrieval. The music structure information, i.e., timing, harmony and music region content, is represented by the layers of the music structure pyramid. We begin by extracting this layered structure information. We analyze the rhythm of the music and then segment the signal proportional to the inter-beat intervals. Thus, the timing information is incorporated in the segmentation process, which we call Beat Space Segmentation. To describe Harmony Events, we propose a two-layer hierarchical approach to model the music chords. We also model the progression of instrumental and vocal content as Acoustic Events. After information extraction, we propose a vector space modeling approach which uses these events as the indexing terms. In query-by-example music retrieval, a query is represented by a vector of the statistics of the n-gram events. We then propose two effective retrieval models, a hard-indexing scheme and a soft-indexing scheme. Experiments show that the vector space modeling is effective in representing the layered music information, achieving 82.5% top-5 retrieval accuracy using 15-sec music clips as the queries. The soft-indexing outperforms hard-indexing in general.

REFERENCES

• 1: Adam Berenzweig , Beth Logan , Daniel P. W. Ellis , Brian P. W. Whitman, A Large-Scale Evaluation of Acoustic and Subjective Music-Similarity Measures, Computer Music Journal, v.28 n.2, p.63-76, June 2004 [doi>10.1162/014892604323112257]
• 2: Cavnar, W.B., and Trenkle, J.M. N-Gram-Based Text Categorization. In Proc. of 3rd Annual Symposium on Document Analysis and Information Retrieval, 1994.
• 3: Wei Chai , Barry Vercoe, Structural analysis of musical signals for indexing and thumbnailing, Proceedings of the 3rd ACM/IEEE-CS joint conference on Digital libraries, May 27-31, 2003, Houston, Texas
• 4: John R. Deller, Jr. , John G. Proakis , John H. Hansen, Discrete Time Processing of Speech Signals, Prentice Hall PTR, Upper Saddle River, NJ, 1993
• 5: Shyamala Doraisamy , Stefan Rüger, Robust Polyphonic Music Retrieval with N-grams, Journal of Intelligent Information Systems, v.21 n.1, p.53-70, July 2003 [doi>10.1023/A:1023553801115]
• 6: Stephen Downie , Michael Nelson, Evaluation of a simple and effective music information retrieval method, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.73-80, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345551]
• 7: Duxburg. C, Sandler. M., and Davies. M. A Hybrid Approach to Musical Note Onset Detection. In Proc. Int. Conf. DAFx. Hamburg, Germany, Sept, 2002.
• 8: Jonathan Foote, Visualizing music and audio using self-similarity, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.77-80, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319472]
• 9: Fujishima, T. Real Time Chord Recognition of Musical Sound: A System Using Lisp Music. In Proc. ICMC, Oct. 1999.
• 10: Asif Ghias , Jonathan Logan , David Chamberlin , Brian C. Smith, Query by humming: musical information retrieval in an audio database, Proceedings of the third ACM international conference on Multimedia, p.231-236, November 05-09, 1995, San Francisco, California, United States [doi>10.1145/217279.215273]
• 11: Goldstein, J. L. An Optimum Processor Theory for the Central Formation of the Pitch of Complex Tones. In JASA, Vol. 54, 1973.
• 12: Kageyama, T., Mochizuki, K., and Takashima, Y. Melody Retrieval with Humming. In Proc. ICMC, Sept, 1993.
• 13: Lemström, K., and Laine, P. Music Information Retrieval using Musical Parameters. In Proc. of the ICMC, Oct, 1998.
• 14: Bin Ma , Haizhou Li, A phonotactic-semantic paradigm for automatic spoken document classification, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076098]
• 15: Namunu C. Maddage , Changsheng Xu , Mohan S. Kankanhalli , Xi Shao, Content-based music structure analysis with applications to music semantics understanding, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027549]
• 16: Rodger J. McNab , Lloyd A. Smith , Ian H. Witten , Clare L. Henderson , Sally Jo Cunningham, Towards the digital music library: tune retrieval from acoustic input, Proceedings of the first ACM international conference on Digital libraries, p.11-18, March 20-23, 1996, Bethesda, Maryland, United States [doi>10.1145/226931.226934]
• 17: Massimo Melucci , Nicola Orio, Musical information retrieval using melodic surface, Proceedings of the fourth ACM conference on Digital libraries, p.152-160, August 11-14, 1999, Berkeley, California, United States [doi>10.1145/313238.313293]
• 18: Pickens, J. A Survey of Feature Selection Techniques for Music Information Retrieval. Technical report, Center of Intelligent Information Retrieval, Dept. of Computer Science, University of Massachusetts, 2001.
• 19: Pickens, J. and Iliopoulos, C. Markov Random Fields and Maximum Entropy Modeling for Music Information Retrieval. In Proc. of ISMIR, Sept, 2005.
• 20: Salton, G. The SMART retrieval system. Prentice-Hall, Englewood Cliffs, NJ, 1971.
• 21: Shih, H.-H., Narayanan, S. S., and Kuo, C.-C. J. An HMM-Based Approach to Humming Transcription. In Proc. of ICME, Aug, 2002.
• 22: Song, J., Bae, S. Y., and Yoon, K. Mid-Level Music Melody Representation of Polyphonic Audio for Query-by-Humming System. In Proc. of ISMIR, Oct, 2002.
• 23: Terhardt, E. Pitch, Consonance and Harmony. In JASA, Vol. 55, No. 5, 1974.
• 24: Typke, R., Wiering, F., and Veltkamp, R. A Survey of Music Information Retrieval Systems. In Proc. of the ISMIR, Sept. 2005.
• 25: Alexandra L. Uitdenbogerd , Justin Zobel, An architecture for effective music information retrieval, Journal of the American Society for Information Science and Technology, v.55 n.12, p.1053-1057, October 2004 [doi>10.1002/asi.20057]
• 26: Ward, W. Subjective Music Pitch. In JASA, Vol. 26, 195

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms, Experimentation, Performance

Keywords:
n-gram, acoustic event, beat space segmentation, harmony event, music structure, vector space modeling