Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

Photo community sites such as Flickr and Picasa Web Album host a massive amount of personal photos with millions of new photos uploaded every month. These photos constitute an overwhelming source of images that require effective management. There is an increasingly imperative need for semantic annotation of these web images. This paper addresses the problem by considering two kinds of annotation: semantic annotation and geographic annotation. Both are useful for image search and retrieval and for facilitating communities and social networks. This paper proposes a novel method of Logistic Canonical Correlation Regression (LCCR) for the annotation task. This model exploits the canonical correlation between heterogeneous features and an annotation lexicon of interest, and builds a generalized annotation engine based on canonical correlations in order to produce enhanced annotation for web images. We validate the effectiveness of our algorithm using a dataset of over 380,000 images tagged with GPS coordinates.

REFERENCES

• 1: Flickr APIs. http://www.flickr.com/services/api/.
• 2: Liangliang Cao , Jiebo Luo , Thomas S. Huang, Annotating photo collections by label propagation according to multiple similarity cues, Proceeding of the 16th ACM international conference on Multimedia, October 26-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1459359.1459376]
• 3: Yizong Cheng, Mean Shift, Mode Seeking, and Clustering, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.17 n.8, p.790-799, August 1995 [doi>10.1109/34.400568]
• 4: Dorin Comaniciu , Peter Meer, Mean Shift Analysis and Applications, Proceedings of the International Conference on Computer Vision-Volume 2, p.1197, September 20-25, 1999
• 5: David J. Crandall , Lars Backstrom , Daniel Huttenlocher , Jon Kleinberg, Mapping the world's photos, Proceedings of the 18th international conference on World wide web, April 20-24, 2009, Madrid, Spain [doi>10.1145/1526709.1526812]
• 6: S. Deerwester, S. Dumais, G. Furnas, T. Landauer, and R. Harshman. Indexing by latent semantic analysis. Journal of the American society for information science, 41(6):391--407, 1990.
• 7: Y. Freund and R. E. Schapire. Experiments with a new boosting algorithm. ICML, pages 148--156, 1996.
• 8: Yun Fu , Liangliang Cao , Guodong Guo , Thomas S. Huang, Multiple feature fusion by subspace learning, Proceedings of the 2008 international conference on Content-based image and video retrieval, July 07-09, 2008, Niagara Falls, Canada [doi>10.1145/1386352.1386373]
• 9: David R. Hardoon , Sandor R. Szedmak , John R. Shawe-taylor, Canonical Correlation Analysis: An Overview with Application to Learning Methods, Neural Computation, v.16 n.12, p.2639-2664, December 2004 [doi>10.1162/0899766042321814]
• 10: J. Hays and A. A. Efros. Im2gps: estimating geographic information from a single image. In IEEE Conference on Computer Vision and Pattern Recognition, 2008.
• 11: G. Holmes, A. Donkin, and I. Witten. Weka: A machine learning workbench. Intelligent Information Systems, pages 357--361, 1994.
• 12: H. Hotelling. Relations between two sets of variates. Biometrika, 28(3-4):321--377, 1936.
• 13: Alexandar Jaffe , Mor Naaman , Tamir Tassa , Marc Davis, Generating summaries and visualization for large collections of geo-referenced photographs, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178692]
• 14: Jimin Jia , Nenghai Yu , Xian-Sheng Hua, Annotating personal albums via web mining, Proceeding of the 16th ACM international conference on Multimedia, October 26-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1459359.1459421]
• 15: Yushi Jing , Shumeet Baluja, Pagerank for product image search, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367540]
• 16: Lyndon Kennedy , Mor Naaman , Shane Ahern , Rahul Nair , Tye Rattenbury, How flickr helps us make sense of the world: context and content in community-contributed media collections, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291384]
• 17: Tae-Kyun Kim , Josef Kittler , Roberto Cipolla, Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.29 n.6, p.1005-1018, June 2007 [doi>10.1109/TPAMI.2007.1037]
• 18: P. Lai and C. Fyfe. Kernel and nonlinear canonical correlation analysis. International Journal of Neural Systems, 16(12):2639--2664, 2004.
• 19: Jiebo Luo , Jie Yu , Dhiraj Joshi , Wei Hao, Event recognition: viewing the world with a third eye, Proceeding of the 16th ACM international conference on Multimedia, October 26-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1459359.1459574]
• 20: Aude Oliva , Antonio Torralba, Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope, International Journal of Computer Vision, v.42 n.3, p.145-175, May-June 2001 [doi>10.1023/A:1011139631724]
• 21: Till Quack , Bastian Leibe , Luc Van Gool, World-scale mining of objects and events from community photo collections, Proceedings of the 2008 international conference on Content-based image and video retrieval, July 07-09, 2008, Niagara Falls, Canada [doi>10.1145/1386352.1386363]
• 22: G. Schindler, P. Krishnamurthy, R. Lublinerman, Y. Liu, and F. Dellaert. Detecting and matching repeated patterns for automatic geo-tagging in urban environments. In IEEE Conference on Computer Vision and Pattern Recognition, 2008.
• 23: John R. Smith , Shih-Fu Chang, Visually Searching the Web for Content, IEEE MultiMedia, v.4 n.3, p.12-20, July 1997 [doi>10.1109/93.621578]
• 24: A. Sorokin and D. Forsyth. Utility data annotation with amazon mechanical turk. In IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 1--8, 2008.
• 25: Antonio Torralba , Rob Fergus , William T. Freeman, 80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.30 n.11, p.1958-1970, November 2008 [doi>10.1109/TPAMI.2008.128]
• 26: A. Vinokourov, J. Shawe-Taylor, and N. Cristianini. Inferring a semantic representation of text via cross-language correlation analysis. Advances in Neural Information Processing Systems, pages 1497--1504, 2003.
• 27: Changhu Wang , Lei Zhang , Hong-Jiang Zhang, Learning to reduce the semantic gap in web image retrieval and annotation, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore [doi>10.1145/1390334.1390396]
• 28: Xin-Jing Wang , Lei Zhang , Xirong Li , Wei-Ying Ma, Annotating Images by Mining Image Search Results, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.30 n.11, p.1919-1932, November 2008 [doi>10.1109/TPAMI.2008.127]
• 29: Kilian Quirin Weinberger , Malcolm Slaney , Roelof Van Zwol, Resolving tag ambiguity, Proceeding of the 16th ACM international conference on Multimedia, October 26-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1459359.1459375]
• 30: Lei Wu , Xian-Sheng Hua , Nenghai Yu , Wei-Ying Ma , Shipeng Li, Flickr distance, Proceeding of the 16th ACM international conference on Multimedia, October 26-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1459359.1459364]
• 31: Jie Yu , Jiebo Luo, Leveraging probabilistic season and location context models for scene understanding, Proceedings of the 2008 international conference on Content-based image and video retrieval, July 07-09, 2008, Niagara Falls, Canada [doi>10.1145/1386352.1386379]
• 32: W. Zheng, X. Zhou, C. Zou, and L. Zhao. Facial expression recognition using kernel canonical correlation analysis (KCCA). IEEE Transactions on Neural Networks, 17(1):233--238, 2006.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms, Experimentation

Keywords:
canonical correlations, geographic annotation, web image annotation