Multi-progressive model for web image annotation

ABSTRACT

Web image annotation is a very important method to effectively index and search images on the internet. Many web image annotation approaches utilized not only the visual information but also the texts emerged with web image. However, they failed to utilize the whole relations among annotations, which reflect the specific semantic content of images. In this paper we propose a novel Multi-Progressive Model (MPM) for web image annotation that leverages word correlations between available texts of web images and an automatic-built vocabulary. The proposed approach treat the available text of web images as initial annotations, and extend them by using a pre-defined lexicon to include more words which are potentially relevant to the target image. It then rank initial and extended annotations by taking advantage of whole words relations without bringing huge computation. The multi-progressive model can be viewed as a greedy optimization algorithm that approximately optimizes the joint annotation probability in a progressive way. Experimental results on web images demonstrate the effectiveness of the proposed model.

REFERENCES

• 1: Mori, Y., Takahashi, H., Oka, R. 1999. Image-to-word Transformation Based on Dividing and Vector Quantizing Images with Words. MISRM.
• 2: P. Duygulu , Kobus Barnard , J. F. G. de Freitas , David A. Forsyth, Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary, Proceedings of the 7th European Conference on Computer Vision-Part IV, p.97-112, May 28-31, 2002
• 3: David M. Blei , Michael I. Jordan, Modeling annotated data, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860460]
• 4: J. Jeon , V. Lavrenko , R. Manmatha, Automatic image annotation and retrieval using cross-media relevance models, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860459]
• 5: Lavrenko, V., Manmatha, R., Jeon, J. 2003. A Model for Learning the Semantics of Pictures. In Proceedings of NIPS.
• 6: Feng, S. L., Manmatha, R., Lavrenko, V. 2004. Multiple Bernoulli Relevance Models for Image and Video Annotation. In Proceedings of CVPR.
• 7: Changhu Wang , Feng Jing , Lei Zhang , Hong-Jiang Zhang, Scalable search-based image annotation of personal images, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178714]
• 8: Xin-Jing Wang , Lei Zhang , Feng Jing , Wei-Ying Ma, AnnoSearch: Image Auto-Annotation by Search, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.1483-1490, June 17-22, 2006 [doi>10.1109/CVPR.2006.58]
• 9: Rui X. G., Yu N. H., Wang T. F., Li M. J. 2007. A Search-Based Web Image Annotation Method. In Proceedings of ICME
• 10: Wang B., Li Z. W., Yu N. H., Li M. J. 2007. Image Annotation in a Progressive Way. In Proceedings of ICME
• 11: Zhang L., Hu Y., Li M., Ma W., Zhang H. 2007. Image Annotation Refinement Using NSC-Based Word Correlation. In Proceedings of ACM Multimedia
• 12: Lei Zhang , Yuxiao Hu , Mingjing Li , Weiying Ma , Hongjiang Zhang, Efficient propagation for face annotation in family albums, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027689]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Experimentation, Measurement

Keywords:
multi-progressive model, web image annotation