Learning to tag

ABSTRACT

Social tagging provides valuable and crucial information for large-scale web image retrieval. It is ontology-free and easy to obtain; however, irrelevant tags frequently appear, and users typically will not tag all semantic objects in the image, which is also called semantic loss. To avoid noises and compensate for the semantic loss, tag recommendation is proposed in literature. However, current recommendation simply ranks the related tags based on the single modality of tag co-occurrence on the whole dataset, which ignores other modalities, such as visual correlation. This paper proposes a multi-modality recommendation based on both tag and visual correlation, and formulates the tag recommendation as a learning problem. Each modality is used to generate a ranking feature, and Rankboost algorithm is applied to learn an optimal combination of these ranking features from different modalities. Experiments on Flickr data demonstrate the effectiveness of this learning-based multi-modality recommendation strategy.

REFERENCES

• 1: E. Akbas and F. Yarman Vural. Automatic image annotation by ensemble of visual descriptors. CVPR'07., June 2007.
• 2: Morgan Ames , Mor Naaman, Why we tag: motivations for annotation in mobile and online media, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240772]
• 3: Jaume Amores , Nicu Sebe , Petia Radeva, Context-Based Object-Class Recognition and Retrieval by Generalized Correlograms, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.29 n.10, p.1818-1833, October 2007 [doi>10.1109/TPAMI.2007.1098]
• 4: Ziv Bar-Yossef , Maxim Gurevich, Random sampling from a search engine's index, Proceedings of the 15th international conference on World Wide Web, May 23-26, 2006, Edinburgh, Scotland [doi>10.1145/1135777.1135833]
• 5: Jim Blythe , Yolanda Gil, Incremental formalization of document annotations through ontology-based paraphrasing, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988734]
• 6: Susanne Boll , Philipp Sandhaus , Ansgar Scherp , Utz Westermann, Semantics, content, and structure of many for the creation of personal photo albums, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291385]
• 7: Yoav Freund , Raj D. Iyer , Robert E. Schapire , Yoram Singer, An Efficient Boosting Algorithm for Combining Preferences, Proceedings of the Fifteenth International Conference on Machine Learning, p.170-178, July 24-27, 1998
• 8: Georgia Koloniari , Yannis Petrakis , Evaggelia Pitoura , Thodoris Tsotsos, Query workload-aware overlay construction using histograms, Proceedings of the 14th ACM international conference on Information and knowledge management, October 31-November 05, 2005, Bremen, Germany [doi>10.1145/1099554.1099717]
• 9: Haoyuan Li , Yi Wang , Dong Zhang , Ming Zhang , Edward Y. Chang, Pfp: parallel fp-growth for query recommendation, Proceedings of the 2008 ACM conference on Recommender systems, October 23-25, 2008, Lausanne, Switzerland [doi>10.1145/1454008.1454027]
• 10: Xirong Li , Cees G.M. Snoek , Marcel Worring, Learning tag relevance by neighbor voting for social image retrieval, Proceeding of the 1st ACM international conference on Multimedia information retrieval, October 30-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1460096.1460126]
• 11: Jing Liu , Bin Wang , Mingjing Li , Zhiwei Li , Weiying Ma , Hanqing Lu , Songde Ma, Dual cross-media relevance model for image annotation, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291380]
• 12: J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman. Supervised dictionary learning, 2008.
• 13: Mor Naaman , Rahul Nair, ZoneTag's Collaborative Tag Suggestions: What is This Person Doing in My Phone?, IEEE MultiMedia, v.15 n.3, p.34-40, July 2008 [doi>10.1109/MMUL.2008.69]
• 14: Guo-Jun Qi , Xian-Sheng Hua , Yong Rui , Jinhui Tang , Tao Mei , Hong-Jiang Zhang, Correlative multi-label video annotation, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291245]
• 15: Yan Qi , K. Selçuk Candan , Junichi Tatemura , Songting Chen , Fenglin Liao, Supporting OLAP operations over imperfectly integrated taxonomies, Proceedings of the 2008 ACM SIGMOD international conference on Management of data, June 09-12, 2008, Vancouver, Canada [doi>10.1145/1376616.1376703]
• 16: Xiaoguang Rui , Mingjing Li , Zhiwei Li , Wei-Ying Ma , Nenghai Yu, Bipartite graph reinforcement model for web image annotation, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291378]
• 17: Shilad Sen , Shyong K. Lam , Al Mamunur Rashid , Dan Cosley , Dan Frankowski , Jeremy Osterhouse , F. Maxwell Harper , John Riedl, tagging, communities, vocabulary, evolution, Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, November 04-08, 2006, Banff, Alberta, Canada [doi>10.1145/1180875.1180904]
• 18: Börkur Sigurbjörnsson , Roelof van Zwol, Flickr tag recommendation based on collective knowledge, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367542]
• 19: C. G. M. Snoek, B. Huurnink, L. Hollink, M. D. Rijke, G. Schreiber, and M. Worring. Adding semantics to detectors for video retrieval. IEEE Transactions on Multimedia, 9, 2007.
• 20: C. Wang, F. Jing, L. Zhang, and H.-J. Zhang. Content-based image annotation refinement. Proceedings of CVPR 07, 2007.
• 21: Lei Wu , Xian-Sheng Hua , Nenghai Yu , Wei-Ying Ma , Shipeng Li, Flickr distance, Proceeding of the 16th ACM international conference on Multimedia, October 26-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1459359.1459364]
• 22: Lei Wu , Mingjing Li , Zhiwei Li , Wei-Ying Ma , Nenghai Yu, Visual language modeling for image classification, Proceedings of the international workshop on Workshop on multimedia information retrieval, September 24-29, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290082.1290101]
• 23: Rong Yan , Alexander Hauprmann, Query expansion using probabilistic local feedback with application to multimedia retrieval, Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, November 06-10, 2007, Lisbon, Portugal [doi>10.1145/1321440.1321492]
• 24: Jie Yu , Jaume Amores , Nicu Sebe , Petia Radeva , Qi Tian, Distance Learning for Similarity Estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.30 n.3, p.451-462, March 2008 [doi>10.1109/TPAMI.2007.70714]
• 25: Y.-T. Zheng, S.-Y. Neo, T.-S. Chua, and Q. Tian. Visual synset: towards a higher-level visual representation. In Proceedings of CVPR'08, 2008.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Image databases

General Terms:
Algorithms, Experimentation, Theory

Keywords:
learning to tag, multi-modality rankboost, social tagging, tag recommendation