A novel framework for efficient automated singer identification in large music databases

ABSTRACT

Over the past decade, there has been explosive growth in the availability of multimedia data, particularly image, video, and music. Because of this, content-based music retrieval has attracted attention from the multimedia database and information retrieval communities. Content-based music retrieval requires us to be able to automatically identify particular characteristics of music data. One such characteristic, useful in a range of applications, is the identification of the singer in a musical piece. Unfortunately, existing approaches to this problem suffer from either low accuracy or poor scalability. In this article, we propose a novel scheme, called Hybrid Singer Identifier (HSI), for efficient automated singer recognition. HSI uses multiple low-level features extracted from both vocal and nonvocal music segments to enhance the identification process; it achieves this via a hybrid architecture that builds profiles of individual singer characteristics based on statistical mixture models. An extensive experimental study on a large music database demonstrates the superiority of our method over state-of-the-art approaches in terms of effectiveness, efficiency, scalability, and robustness.

REFERENCES

• 1: Bartsch, M. and Wakefield, G. 2004. Singing voice identification using spectral envelop estimation. IEEE Trans. Speech Aud. Process. 12, 100--109.
• 2: Claudio Becchetti , Lucio Prina Ricotti, Speech recognition: theory and C++ implementation, John Wiley & Sons, Inc., New York, NY, 1999
• 3: Berenzweig, A., Ellis, D. P. W., and Lawrence, S. 2002. Using voice segments to improve artist classification of music. In Proceedings of the AES 22nd International Conference on Virtual, Synthetic and Entertainment Audio. 119--122.
• 4: Adam Berenzweig , Beth Logan , Daniel P. W. Ellis , Brian P. W. Whitman, A Large-Scale Evaluation of Acoustic and Subjective Music-Similarity Measures, Computer Music Journal, v.28 n.2, p.63-76, June 2004 [doi>10.1162/014892604323112257]
• 5: Berenzweig, A. L. and Ellis, D. P. W. 2001. Locating singing voice segments within music signals. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. 119--122.
• 6: A. Blum, Learning boolean functions in an infinite attribute space, Proceedings of the twenty-second annual ACM symposium on Theory of computing, p.64-72, May 13-17, 1990, Baltimore, Maryland, United States [doi>10.1145/100216.100224]
• 7: Chad Carson , Serge Belongie , Hayit Greenspan , Jitendra Malik, Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.8, p.1026-1038, August 2002 [doi>10.1109/TPAMI.2002.1023800]
• 8: Chang, C.-C. and Lin, C.-J. 2001. LIBSVM: A library for support vector machines. http://www.csie.ntu.edu.tw/~cjlin/libsvm.
• 9: Michael Collins , Robert E. Schapire , Yoram Singer, Logistic Regression, AdaBoost and Bregman Distances, Proceedings of the Thirteenth Annual Conference on Computational Learning Theory, p.158-169, June 28-July 01, 2000
• 10: J. Stephen Downie , Andreas F. Ehmann , Xiao Hu, Music-to-knowledge (M2K): a prototyping and evaluation environment for music digital library research, Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2005, Denver, CO, USA [doi>10.1145/1065385.1065479]
• 11: Downie, J. S. 2006. The Music Information Retrieval Evaluation Exchange (MIREX). D-Lib Mag. 12, 12 (Dec.)
• 12: Downie, J. S., West, K., Ehmann, A., and Vincent, E. 2005b. The 2005 Music Information Retrieval Evaluation Exchange (MIREX 2005) preliminary overview. In Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR). 320--323.
• 13: Robert F. Easley , John G. Michel , Sarv Devaraj, The MP3 open standard and the music industry's response to Internet piracy, Communications of the ACM, v.46 n.11, p.90-96, November 2003 [doi>10.1145/948383.948386]
• 14: Yoav Freund , Robert E. Schapire, A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Computer and System Sciences, v.55 n.1, p.119-139, Aug. 1997 [doi>10.1006/jcss.1997.1504]
• 15: Hayit Greenspan , Jacob Goldberger , Lenny Ridel, A continuous probabilistic framework for image matching, Computer Vision and Image Understanding, v.84 n.3, p.384-406, December 2001 [doi>10.1006/cviu.2001.0946]
• 16: Hastie, T., Tibshirani, R., and Friedman, J. 2001. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Verlag, Berlin, Germany.
• 17: ISMIR. 2004. The Fifth International Conference on Music Information Retrieval. http://ismir2004.ismir.net/index.html.
• 18: Jordan, M. I. 1995. Why the logistic function? a tutorial discussion on probabilities and neural networks. Tech. rep. 9503. MIT, Cambridge, MA.
• 19: Kim, Y. E. and Whitman, B. 2002. Singer identification in popular music recordings using voice coding features. In Proceedings of the 3rd International Conference Music on Information Retrieval (ISMIR). 164--169.
• 20: Kim, Y. E., Williamson, D., and Pilli, S. 2006. Towards quantifying the album effect in artist identification. In Proceedings of the 7th International Conference Music Information Retrieval (ISMIR'06). 393--394.
• 21: The Internet is changing the music industry, Communications of the ACM, v.44 n.8, p.62-68, Aug. 2001 [doi>10.1145/381641.381658]
• 22: Lebanon, G. and Lafferty, J. 2001. Boosting and maximum likelihood for exponential model and Bregman distances. In Advances in Neural Information Processing Systems 14 (Proceedings of NIPS). 110--121.
• 23: Tao Li , Mitsunori Ogihara, Music artist style identification by semi-supervised learning from both lyrics and content, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027612]
• 24: Tao Li , Mitsunori Ogihara , Qi Li, A comparative study on content-based music genre classification, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860487]
• 25: Chih-Chin Liu , Chuan-Sung Huang, A singer identification technique for content-based classification of MP3 music objects, Proceedings of the eleventh international conference on Information and knowledge management, November 04-09, 2002, McLean, Virginia, USA [doi>10.1145/584792.584864]
• 26: Livshin, A. and Rodet, X. 2004. Musical instrument identification in continuous recordings. In Proceedings of the 7th International Conference on Digital Audio Effects (DAFx). 222--227.
• 27: Lu, L., Zhang, H., and Li, S. Z. 2003. Content-based audio classification and segmentation by using support vector machines. Multimed. Syst. 8, 6, 482--492.
• 28: MIREX. 2005. Artist identification contest track. http://www.music-ir.org/evaluation/mirex-results/audio-artist/index.html.
• 29: MIREX. 2007. Artist identification contest track. http://www.music-ir.org/mirex2007/index.php/AudioArtistIdentificationResults.
• 30: François Pachet, Content management for electronic music distribution, Communications of the ACM, v.46 n.4, p.71-75, April 2003 [doi>10.1145/641205.641207]
• 31: Bryan Pardo, Introduction, Communications of the ACM, v.49 n.8, August 2006 [doi>10.1145/1145287.1145309]
• 32: Alberto Pinto , Goffredo Haus, A novel XML music information retrieval method using graph invariants, ACM Transactions on Information Systems (TOIS), v.25 n.4, p.19-es, October 2007 [doi>10.1145/1281485.1281490]
• 33: Lawrence Rabiner , Biing-Hwang Juang, Fundamentals of speech recognition, Prentice-Hall, Inc., Upper Saddle River, NJ, 1993
• 34: Rabiner, L. and Schafer, R. 1978. Digital Processing of Speech Signals. Prentice-Hall, Englewood Cliffs, NJ.
• 35: Rissanen, J. 1978. Modeling by shortest data description. Automatica 14, 465--471.
• 36: Jialie Shen , John Shepherd , Bin Cui , Kian-Lee Tan, HSI: A Novel Framework for Efficient Automated Singer Identification in Large Music Database, Proceedings of the 22nd International Conference on Data Engineering, p.169, April 03-07, 2006 [doi>10.1109/ICDE.2006.79]
• 37: Tolonen, T. and Karjalainen, M. 2000. A computationally efficient multipitch analysis model. IEEE Trans. Speech Aud. Process. 8, 4, 708--716.
• 38: Tsai, W. H. and Wang, H. M. 2006. Automatic singer recognition of popular music recordings via estimation and modeling of solo vocal signals. IEEE Trans. Speech Aud. Process. 14, 1, 330--341.
• 39: Tsai, W. H., Wang, H. M., Rodgers, D., Cheng, S. S., and Yu, H. M. 2003. Blind clustering of popular music recordings based on singer voice characteristics. In Proceedings of the 4th international Conference on Music Information Retrieval (ISMIR). 167--173.
• 40: Vapnik, V. 1998. Statistical Learning Theory. John Wiley & Sons. New York, NY.
• 41: Whitman, B., Flake, G., and Lawrence, S. 2001. Artist detection in music with Minnowmatch. In Proceedings of the IEEE Workshop on Neural Networks for Signal Processing. 559--568.
• 42: Xu, C. S., Maddage, N., and Shao, X. 2005. Automatic music classification and summarization. IEEE Trans. Speech Aud. Process. 13, 3, 441--450.
• 43: Tong Zhang, Automatic singer identification, Proceedings of the 2003 International Conference on Multimedia and Expo, p.33-36, July 06-09, 2003

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software
          Subjects: Performance evaluation (efficiency and effectiveness)
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.5 Sound and Music Computing
          Subjects: Modeling

  J. Computer Applications
  J.5 ARTS AND HUMANITIES
      Subjects: Performing arts (e.g., dance, music)

General Terms:
Algorithms, Experimentation, Performance

Keywords:
EM algorithm, Gaussian mixture models, Music retrieval, classification, evaluation, singer identification, statistical modeling