Content-based organization and visualization of music archives

ABSTRACT

With Islands of Music we present a system which facilitates exploration of music libraries without requiring manual genre classification. Given pieces of music in raw audio format we estimate their perceived sound similarities based on psychoacoustic models. Subsequently, the pieces are organized on a 2-dimensional map so that similar pieces are located close to each other. A visualization using a metaphor of geographic maps provides an intuitive interface where islands resemble genres or styles of music. We demonstrate the approach using a collection of 359 pieces of music.

REFERENCES

• 1: David Bainbridge , Craig G. Nevill-Manning , Ian H. Witten , Lloyd A. Smith , Rodger J. McNab, Towards a digital library of popular music, Proceedings of the fourth ACM conference on Digital libraries, p.161-169, August 11-14, 1999, Berkeley, California, United States [doi>10.1145/313238.313295]
• 2: W. P. Birmingham, R. B. Dannenberg, G. H. Wakefield, M. Bartsch, D. Bykowski, D. Mazzoni, C. Meek, M. Mellody, and W. Rand. MUSART: Music retrieval via aural queries. In Int. Symposium on Music Information Retrieval (ISMIR), 2001.
• 3: Christopher M. Bishop , Markus Svensén , Christopher K. I. Williams, GTM: the generative topographic mapping, Neural Computation, v.10 n.1, p.215-234, Jan. 1, 1998 [doi>10.1162/089976698300017953]
• 4: R. Bladon. Modeling the judgment of vowel quality differences. Journal of the Acoustical Society of America, 69:1414--1422, 1981.
• 5: R. B. Dannenberg, B. Thom, and D. Watson. A machine learning approach to musical style recognition. In Proc. Int. Computer Music Conf. (ICMC), pages 344--347, Thessaloniki, GR, 1997.
• 6: M. Dittenbach, D. Merkl, and A. Rauber. The Growing Hierarchical Self-Organizing Map. In Proc. Int. Joint Conf. on Neural Networks (IJCNN), volume VI, pages 15--19, Como, Italy, 2000. IEEE Computer Society.
• 7: H. Fastl. Fluctuation strength and temporal masking patterns of amplitude-modulated broad-band noise. Hearing Research, 8:59--69, 1982.
• 8: B. Feiten and S. Günzel. Automatic Indexing of a Sound Database Using Self-organizing Neural Nets. Computer Music Journal, 18(3):53--65, 1994.
• 9: Jonathan Foote, An overview of audio information retrieval, Multimedia Systems, v.7 n.1, p.2-10, Jan. 1999 [doi>10.1007/s005300050106]
• 10: Asif Ghias , Jonathan Logan , David Chamberlin , Brian C. Smith, Query by humming: musical information retrieval in an audio database, Proceedings of the third ACM international conference on Multimedia, p.231-236, November 05-09, 1995, San Francisco, California, United States [doi>10.1145/217279.215273]
• 11: H. Hotelling. Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24:417--441 and 498--520, 1933.
• 12: Teuvo Kohonen , T. Kohonen , M. R. Schroeder , T. S. Huang, Self-Organizing Maps, Springer-Verlag New York, Inc., Secaucus, NJ, 2001
• 13: T. Kohonen, S. Kaski, K. Lagus, J. Salojärvi, J. Honkela, V. Paatero, and A. Saarela. Self-Organization of a Massive Text Document Collection. In Kohonen Maps, pages 171--182. Elsevier, Amsterdam, 1999.
• 14: Naoko Kosugi , Yuichi Nishihara , Tetsuo Sakata , Masashi Yamamuro , Kazuhiko Kushima, A practical query-by-humming system for a large music database, Proceedings of the eighth ACM international conference on Multimedia, p.333-342, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354520]
• 15: J. B. Kruskal and M. Wish. Multidimensional Scaling. Number 07-011 in Paper Series on Quantitative Applications in the Social Sciences. Sage Publications, Newbury Park, CA, 1978.
• 16: K. Lagus and S. Kaski. Keyword selection method for characterizing text document maps. In Proc. Int. Conf. on Artificial Neural Networks (ICANN), volume 1, pages 371--376, London, 1999. IEE.
• 17: Mingchun Liu , Chunru Wan, A Study on Content-Based Classification and Retrieval of Audio Database, Proceedings of the International Database Engineering & Applications Symposium, p.339-345, July 16-18, 2001
• 18: D. Merkl and A. Rauber. Document classification with unsupervised neural networks. In F. Crestani and G. Pasi, editors, Soft Computing in Information Retrieval, pages 102--121. Physica Verlag, 2000.
• 19: F. Pachet and D. Cazaly. A taxonomy of musical genres. In Proc. Content-Based Multimedia Information Access (RIAO), Paris, France, 2000.
• 20: E. Pampalk. Islands of Music: Analysis, Organization, and Visualization of Music Archives. Master's thesis, Vienna University of Technology, 2001. http://www.oefai.at/~elias/music/thesis.html.
• 21: Elias Pampalk , Andreas Rauber , Dieter Merkl, Using Smoothed Data Histograms for Cluster Visualization in Self-Organizing Maps, Proceedings of the International Conference on Artificial Neural Networks, p.871-876, August 28-30, 2002
• 22: A. Rauber. LabelSOM: On the Labeling of Self-Organizing Maps. In Proc. Int. Joint Conf. on Neural Networks (IJCNN), Washington, DC, 1999.
• 23: Andreas Rauber , Markus Frühwirth, Automatically Analyzing and Organizing Music Archives, Proceedings of the 5th European Conference on Research and Advanced Technology for Digital Libraries, p.402-414, September 04-09, 2001
• 24: Andreas Rauber , Dieter Merkl, SOMLib: a digital library system based on neural networks, Proceedings of the fourth ACM conference on Digital libraries, p.240-241, August 11-14, 1999, Berkeley, California, United States [doi>10.1145/313238.313412]
• 25: A. Rauber, E. Pampalk, and D. Merkl. Using psycho-acoustic models and self-organizing maps to create a hierarchical structuring of music by sound similarities. In Proc. Int. Symposium on Music Information Retrieval (ISMIR), Paris, France, 2002.
• 26: Pierre-Yves Rolland , Gailius Raškinis , Jean-Gabriel Ganascia, Musical content-based retrieval: an overview of the Melodiscov approach and system, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.81-84, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319473]
• 27: J. W. Sammon. A nonlinear mapping for data structure analysis. IEEE Transactions on Computers, 18:401--409, 1969.
• 28: Eric D. Scheirer , Barry L. Vercoe, Music-listening systems, 2000
• 29: M. R. Schröder, B. S. Atal, and J. L. Hall. Optimizing digital speech coders by exploiting masking properties of the human ear. Journal of the Acoustical Society of America, 66:1647--1652, 1979.
• 30: G. Tzanetakis and P. Cook. Musical genre classification of audio signals. IEEE Transactions on Speech and Audio Processing, 2002. To appear.
• 31: G. Tzanetakis, G. Essl, and P. Cook. Automatic musical genre classification of audio signals. In Proc. Int. Symposium on Music Information Retrieval (ISMIR), 2001.
• 32: A. Ultsch and H. P. Siemon. Kohonen's Self-Organizing Feature Maps for Exploratory Data Analysis. In Proc. Int. Neural Network Conf. (INNC), pages 305--308, Dordrecht, Netherlands, 1990. Kluwer.
• 33: Erling Wold , Thom Blum , Douglas Keislar , James Wheaton, Content-Based Classification, Search, and Retrieval of Audio, IEEE MultiMedia, v.3 n.3, p.27-36, September 1996 [doi>10.1109/93.556537]
• 34: E. Zwicker and H. Fastl. Psychoacoustics, Facts and Models, volume 22 of Springer Series of Information Sciences. Springer, Berlin, 2nd updated edition, 1999.

INDEX TERMS

Keywords:
clustering, content-based music retrieval, feature extraction, genre, rhythm, self-organizing map, user interface