Annotating personal albums via web mining

ABSTRACT

Nowadays personal albums are becoming more and more popular due to the explosive growth of digital image capturing devices. An effective automatic annotation system for personal albums is desired for both efficient browsing and search. Existing research on image annotation evolves through two stages: learning-based methods and web-based methods. Learning-based methods attempt to learn classifiers or joint probabilities between images and concepts, which are difficult to handle large-scale concept sets due to the lack of training data. Web-based methods leverage web image data to learn relevant annotations, which greatly expand the scale of concepts. However, they still suffer two problems: the query image lacks prior knowledge and the annotations are often noisy and incoherent. To address the above issues, we propose a web-based annotation approach to annotate a collection of photos simultaneously, instead of annotating them independently, by leveraging the abundant correlations among the photos. A multi-graph similarity propagation based semi-supervised learning (MGSP-SSL) algorithm is proposed to suppress the noises in the initial annotations from the Web. Experiments on real personal albums show that the proposed approach outperforms existing annotation methods.

REFERENCES

• 1: Josef Sivic , Andrew Zisserman, Video Google: A Text Retrieval Approach to Object Matching in Videos, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.1470, October 13-16, 2003
• 2: Jia Li , James Z. Wang, Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.9, p.1075-1088, September 2003 [doi>10.1109/TPAMI.2003.1227984]
• 3: C. Cusano, G. Ciocca, and R. Schettini. Image Annotation Using SVM. Proceedings of Internet Imaging, Vol. SPIE 5304. 2004.
• 4: S. L. Feng, R. Manmatha, and V. Lavrenko. Multiple bernoulli relevance models for image and video annotation. In Proc. of CVPR, Washington, DC, June, 2004.
• 5: David M. Blei , Michael I. Jordan, Modeling annotated data, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860460]
• 6: Changhu Wang , Feng Jing , Lei Zhang , Hong-Jiang Zhang, Scalable search-based image annotation of personal images, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178714]
• 7: Xin-Jing Wang , Lei Zhang , Feng Jing , Wei-Ying Ma, AnnoSearch: Image Auto-Annotation by Search, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.1483-1490, June 17-22, 2006 [doi>10.1109/CVPR.2006.58]
• 8: Xiaoguang Rui , Mingjing Li , Zhiwei Li , Wei-Ying Ma , Nenghai Yu, Bipartite graph reinforcement model for web image annotation, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291378]
• 9: Jingyu Cui , Fang Wen , Rong Xiao , Yuandong Tian , Xiaoou Tang, EasyAlbum: an interactive photo annotation system based on face clustering and re-ranking, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240684]
• 10: Lei Zhang , Longbin Chen , Mingjing Li , Hongjiang Zhang, Automated annotation of human faces in family albums, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957090]
• 11: D. Zhou, J. Huang, and B. Schölkopf. Learning with Local and global consistency. 18th Annual Conference on Neural Information Processing System, 2003.
• 12: J. Jeon , V. Lavrenko , R. Manmatha, Automatic image annotation and retrieval using cross-media relevance models, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860459]
• 13: J Philbin, O Chum, M Isard, J Sivic, A Zisserman. Object retrieval with large vocabularies and fast spatial matching. CVPR, 2007.
• 14: Meng Wang , Xian-Sheng Hua , Xun Yuan , Yan Song , Li-Rong Dai, Optimizing multi-graph learning: towards a unified video annotation scheme, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291431]
• 15: J. Tang, X. Hua, T. Mei, G. Qi, S. Li and X. Wu. Temporally Consistent Gaussian Random Field for Video Semantic Analysis. IEEE International Conference on Image Processing, 2007.
• 16: V. Lavrenko, R. Manmatha and J. Jeon. A Model for Learning the Semantics of Pictures. In Proc. NIPS, 2003.
• 17: Xiaofei He , Wei-Ying Ma , Hong-Jiang Zhang, Learning an image manifold for retrieval, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027532]
• 18: Rudi L. Cilibrasi , Paul M. B. Vitanyi, The Google Similarity Distance, IEEE Transactions on Knowledge and Data Engineering, v.19 n.3, p.370-383, March 2007 [doi>10.1109/TKDE.2007.48]
• 19: Jing Liu , Mingjing Li , Wei-Ying Ma , Qingshan Liu , Hanqing Lu, An adaptive graph model for automatic image annotation, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178689]
• 20: P. Duygulu , Kobus Barnard , J. F. G. de Freitas , David A. Forsyth, Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary, Proceedings of the 7th European Conference on Computer Vision-Part IV, p.97-112, May 28-31, 2002
• 21: J. Kandola, J. Shawe-Taylor, N. Cristianini. Learning Semantic Similarity. Annual Conference on Neural Information Processing System, 2003.
• 22: Xin-Jing Wang , Wei-Ying Ma , Gui-Rong Xue , Xing Li, Multi-model similarity propagation and its application for web image retrieval, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027746]
• 23: D. Lowe. Local feature view clustering for 3D object recognition. In Proc. CVPR, 2001.
• 24: Herwig Lejsek , Fridrik H. Ásmundsson , Björn Thór Jónsson , Laurent Amsaleg, Scalability of local image descriptors: a comparative study, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180760]
• 25: Susanne Boll , Philipp Sandhaus , Ansgar Scherp , Utz Westermann, Semantics, content, and structure of many for the creation of personal photo albums, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291385]
• 26: X. Lian, L. Chen, J. X. Yu, G. Wang, G. Yu. Similarity Match Over High Speed Time-Series Streams. ICDE 2007: 1086--1095
• 27: Forouzan Golshani, EIC's Message: Multimedia is Correlated Media, IEEE MultiMedia, v.11 n.1, p.C2, January 2004 [doi>10.1109/MMUL.2004.1318099]
• 28: Amit Zunjarwad , Hari Sundaram , Lexing Xie, Contextual wisdom: social relations and correlations for multimedia event annotation, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291382]
• 29: Yu-Ru Lin , Hari Sundaram , Yun Chi , Junichi Tatemura , Belle L. Tseng, Detecting splogs via temporal dynamics using self-similarity analysis, ACM Transactions on the Web (TWEB), v.2 n.1, p.1-35, February 2008 [doi>10.1145/1326561.1326565]
• 30: Matthew Cooper , Jonathan Foote , Andreas Girgensohn , Lynn Wilcox, Temporal event clustering for digital photo collections, ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), v.1 n.3, p.269-288, August 2005 [doi>10.1145/1083314.1083317]
• 31: Ruofei Zhang , Zhongfei (Mark) Zhang , Mingjing Li , Wei-Ying Ma , Hong-Jiang Zhang, A Probabilistic Semantic Model for Image Annotation and Multi-Modal Image Retrieva, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.846-851, October 17-20, 2005 [doi>10.1109/ICCV.2005.16]
• 32: W. Klas, R. King. Context-Aware Multimedia. Encyclopedia of Multimedia 2006
• 33: L. Hardman, J. Ossenbruggen. Creating meaningful multimedia presentations. ISCAS 2006
• 34: Jing Liu , Bin Wang , Mingjing Li , Zhiwei Li , Weiying Ma , Hanqing Lu , Songde Ma, Dual cross-media relevance model for image annotation, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291380]
• 35: E. Chang, et al. CBSA: Content-Based Soft Annotation for Multimodal Image Retrieval Using Bayes Point Machines. CirSysVideo, 2003. 13(1): p. 26--38.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms, Experimentation

Keywords:
image annotation, multi-graph, personal albums, semi-supervised learning, similarity propagation