How do you feel about "dancing queen"?: deriving mood & theme annotations from user tags

ABSTRACT

Web 2.0 enables information sharing, collaboration among users and most notably supports active participation and creativity of the users. As a result, a huge amount of manually created metadata describing all kinds of resources is now available. Such semantically rich user generated annotations are especially valuable for digital libraries covering multimedia resources such as music, where these metadata enable retrieval relying not only on content-based (low level) features, but also on the textual descriptions represented by tags. However, if we analyze the annotations users generate for music tracks, we find them heavily biased towards genre. Previous work investigating the types of user provided annotations for music tracks showed that the types of tags which would be really beneficial for supporting retrieval - usage (theme) and opinion (mood) tags - are often neglected by users in the annotation rocess. In this paper we address exactly this problem: in order to support users in tagging and to fill these gaps in the tag space, we develop algorithms for recommending mood and theme annotations. Our methods exploit the available user annotations, the lyrics of music tracks, as well as combinations of both. We also compare the results for our recommended mood / theme annotations against genre and style recommendations - a much easier and already studied task. Besides evaluating against an expert (AllMusic.com) ground truth, we evaluate the quality of our recommended tags through a Facebook-based user study. Our results are very promising both in comparison to experts as well as users and provide interesting insights into possible extensions for music tagging systems to support music search.

REFERENCES

• 1: M. Aurnhammer, P. Hanappe, and L. Steels. Integrating collaborative tagging and emergent semantics for image retrieval. In WWW Collaborative Web Tagging Workshop, 2006.
• 2: Kerstin Bischoff , Claudiu S. Firan , Wolfgang Nejdl , Raluca Paiu, Can all tags be used for search?, Proceeding of the 17th ACM conference on Information and knowledge management, October 26-30, 2008, Napa Valley, California, USA [doi>10.1145/1458082.1458112]
• 3: Kerstin Bischoff , Claudiu S. Firan , Raluca Paiu, Deriving music theme annotations from user tags, Proceedings of the 18th international conference on World wide web, April 20-24, 2009, Madrid, Spain [doi>10.1145/1526709.1526924]
• 4: A. Byde, H. Wan, and S. Cayzer. Personalized tag recommendations via tagging and content-based similarity metrics. In ICWSM, 2007.
• 5: Yazhong Feng , Yueting Zhuang , Yunhe Pan, Popular music retrieval by detecting mood, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860508]
• 6: Claudiu S. Firan , Wolfgang Nejdl , Raluca Paiu, The Benefit of Using Tag-Based Profiles, Proceedings of the 2007 Latin American Web Conference, p.32-41, October 31-November 02, 2007 [doi>10.1109/LA-WEB.2007.24]
• 7: Harry Halpin , Valentin Robu , Hana Shepherd, The complex dynamics of collaborative tagging, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242602]
• 8: Paul Heymann , Daniel Ramage , Hector Garcia-Molina, Social tag prediction, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore [doi>10.1145/1390334.1390425]
• 9: X. Hu and J. S. Downie. Exploring mood metadata: Relationships with genre, artist and usage metadata. In ISMIR, 2007.
• 10: Tao Li , Mitsunori Ogihara , Qi Li, A comparative study on content-based music genre classification, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860487]
• 11: Xin Li , Lei Guo , Yihong Eric Zhao, Tag-based social interest discovery, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367589]
• 12: D. Liu, L. Lu, and H.-J. Zhang. Automatic mood detection from acoustic music data. In ISMIR, 2003.
• 13: Gilad Mishne, AutoTag: a collaborative approach to automated tag assignment for weblog posts, Proceedings of the 15th international conference on World Wide Web, May 23-26, 2006, Edinburgh, Scotland [doi>10.1145/1135777.1135961]
• 14: Mor Naaman , Rahul Nair, ZoneTag's Collaborative Tag Suggestions: What is This Person Doing in My Phone?, IEEE MultiMedia, v.15 n.3, p.34-40, July 2008 [doi>10.1109/MMUL.2008.69]
• 15: Tye Rattenbury , Nathaniel Good , Mor Naaman, Towards automatic extraction of event and place semantics from flickr tags, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277762]
• 16: Shilad Sen , Shyong K. Lam , Al Mamunur Rashid , Dan Cosley , Dan Frankowski , Jeremy Osterhouse , F. Maxwell Harper , John Riedl, tagging, communities, vocabulary, evolution, Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, November 04-08, 2006, Banff, Alberta, Canada [doi>10.1145/1180875.1180904]
• 17: P. Shaver, J. Schwartz, D. Kirson, and C. O'Connor. Emotion knowledge: Further exploration of a prototype approach. Journal of Personality and Social Psychology, 52(6):1061--1086, 1987.
• 18: Börkur Sigurbjörnsson , Roelof van Zwol, Flickr tag recommendation based on collective knowledge, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367542]
• 19: J. Skowronek, M. McKinney, and S. van de Par. A demonstrator for automatic music mood estimation. In ISMIR, 2007.
• 20: S. Sood, S. Owsley, K. Hammond, and L. Birnbaum. Tagassist: Automatic tag suggestion for blog posts. In ICWSM, 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.2 Information Storage

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Experimentation, Human Factors, Management, Reliability

Keywords:
collaborative tagging, high-level music descriptors, metadata enrichment, mood and theme tag recommendation