Flickr distance

ABSTRACT

This paper presents Flickr distance, which is a novel measurement of the relationship between semantic concepts (objects, scenes) in visual domain. For each concept, a collection of images are obtained from Flickr, based on which the improved latent topic based visual language model is built to capture the visual characteristic of this concept. Then Flickr distance between different concepts is measured by the square root of Jensen-Shannon (JS) divergence between the corresponding visual language models. Comparing with WordNet, Flickr distance is able to handle far more concepts existing on the Web, and it can scale up with the increase of concept vocabularies. Comparing with Google distance, which is generated in textual domain, Flickr distance is more precise for visual domain concepts, as it captures the visual relationship between the concepts instead of their co-occurrence in text search results. Besides, unlike Google distance, Flickr distance satisfies triangular inequality, which makes it a more reasonable distance metric. Both subjective user study and objective evaluation show that Flickr distance is more coherent to human perception than Google distance. We also design several application scenarios, such as concept clustering and image annotation, to demonstrate the effectiveness of this proposed distance in image related applications.

REFERENCES

• 1: Wasfi Al-Khatib , Y. Francis Day , Arif Ghafoor , P. Bruce Berra, Semantic Modeling and Knowledge Representation in Multimedia Databases, IEEE Transactions on Knowledge and Data Engineering, v.11 n.1, p.64-80, January 1999 [doi>10.1109/69.755616]
• 2: S. Borgatti. Netdraw. http://www.analytictech.com/Netdraw/netdraw.htm, 2008.
• 3: Shih-Fu Chang , Dan Ellis , Wei Jiang , Keansub Lee , Akira Yanagawa , Alexander C. Loui , Jiebo Luo, Large-scale multimodal semantic concept detection for consumer video, Proceedings of the international workshop on Workshop on multimedia information retrieval, September 24-29, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290082.1290118]
• 4: Yixin Chen , James Z. Wang, Image Categorization by Learning and Reasoning with Regions, The Journal of Machine Learning Research, 5, p.913-939, 12/1/2004
• 5: Rudi L. Cilibrasi , Paul M. B. Vitanyi, The Google Similarity Distance, IEEE Transactions on Knowledge and Data Engineering, v.19 n.3, p.370-383, March 2007 [doi>10.1109/TKDE.2007.48]
• 6: Ritendra Datta , Dhiraj Joshi , Jia Li , James Z. Wang, Image retrieval: Ideas, influences, and trends of the new age, ACM Computing Surveys (CSUR), v.40 n.2, p.1-60, April 2008 [doi>10.1145/1348246.1348248]
• 7: Fei-Fei Li , Pietro Perona, A Bayesian Hierarchical Model for Learning Natural Scene Categories, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2, p.524-531, June 20-26, 2005 [doi>10.1109/CVPR.2005.16]
• 8: Thomas Hofmann, Probabilistic latent semantic indexing, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.50-57, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312649]
• 9: T. S. Huang, C. K. Dagli, S. Rajaram, E. Y. Chang, M. I. Mandel, G. E. Poliner, and D. P. W. Ellis. Active learning for interactive multimedia retrieval. In Proc. of the IEEE, 2008.
• 10: J. Jeon , V. Lavrenko , R. Manmatha, Automatic image annotation and retrieval using cross-media relevance models, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860459]
• 11: V. Lavrenko, R. Manmatha, and J. Jeon. A model for learning the semantics of pictures. In Proc. of NIPS'03., 2003.
• 12: Douglas B. Lenat, CYC: a large-scale investment in knowledge infrastructure, Communications of the ACM, v.38 n.11, p.33-38, Nov. 1995 [doi>10.1145/219717.219745]
• 13: Liza Leslie , Tat-Seng Chua , Jain Ramesh, Annotation of paintings with high-level semantic concepts using transductive inference and ontology-based concept disambiguation, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291335]
• 14: Huiying Liu , Shuqiang Jiang , Qingming Huang , Changsheng Xu , Wen Gao, Region-based visual attention analysis with its application in image browsing on small displays, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291298]
• 15: Jing Liu , Bin Wang , Mingjing Li , Zhiwei Li , Weiying Ma , Hanqing Lu , Songde Ma, Dual cross-media relevance model for image annotation, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291380]
• 16: G. A. Miller and et.al. Wordnet, a lexical database for the english language. Cognition Science Lab, Princeton University, 1995.
• 17: Apostol (Paul) Natsev , Alexander Haubold , Jelena Tešić , Lexing Xie , Rong Yan, Semantic concept-based query expansion and re-ranking for multimedia retrieval, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291448]
• 18: Guo-Jun Qi , Xian-Sheng Hua , Yong Rui , Jinhui Tang , Tao Mei , Hong-Jiang Zhang, Correlative multi-label video annotation, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291245]
• 19: B. Wang, Z. Li, M. Li, and W.-Y. Ma. Large-scale duplicate detection for web image search. In Proc. of IEEE International Conference on Multimedia & Expo (ICME'06), 2006.
• 20: C. Wang, F. Jing, L. Zhang, and H.-J. Zhang. Content-based image annotation refinement. 2007.
• 21: Lei Wu , Mingjing Li , Zhiwei Li , Wei-Ying Ma , Nenghai Yu, Visual language modeling for image classification, Proceedings of the international workshop on Workshop on multimedia information retrieval, September 24-29, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290082.1290101]
• 22: L. Wu, J. Liu, M. Li, and N. Yu. Query oriented subspace shifting for near-duplicate image detection. In Proc. of IEEE International Conference on Multimedia & Expo (ICME'08), 2008.
• 23: J. Yu and Q. Tian. Semantic subspace projection and its application in image retrieval. IEEE Transactions on Circuits and Systems for Video Technology (CSVT), pages 544--548, 2008.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Abstracting methods

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Perceptual reasoning

General Terms:
Algorithms, Experimentation, Theory

Keywords:
concept relationship, flickr distance, similarity measurement, tagnet, visual concept net, visual distance