Scalable Markov model-based image annotation

ABSTRACT

In this paper, we propose a novel Markov model-based formulation for the image annotation problem. In this formulation, we treat image annotation as a graph ranking problem, by defining all possible labels in the lexicon as the states of a Markov chain. To fully utilize the correlation between labels, a query-biased transition matrix is dynamically constructed according to the query image. Based on this formulation, a scalable Markov model-based image annotation (MBIA) algorithm is presented to rank all the possible labels. To be scalable, we adapt search techniques on a Web-scale image set, and make MBIA capable of annotating arbitrary images with unlimited vocabulary. By fully exploring the correlation between labels, MBIA leads to superior performance than standard techniques. Experimental results on the typical Corel dataset and U. Washington dataset show the effectiveness and efficiency of the proposed algorithm.

REFERENCES

• 1: David M. Blei , Michael I. Jordan, Modeling annotated data, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860460]
• 2: Chang, E., Kingshy, G., Sychay, G., and Wu, G. CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines. IEEE Trans. on CSVT, 13(1):26--38, Jan. 2003.
• 3: Cusano, C., G. Ciocca, and R. Schettini. Image Annotation Using Svm. in Proceedings of Internet imaging IV, Vol. SPIE 5304. 2004.
• 4: P. Duygulu , Kobus Barnard , J. F. G. de Freitas , David A. Forsyth, Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary, Proceedings of the 7th European Conference on Computer Vision-Part IV, p.97-112, May 28-31, 2002
• 5: J. Jeon , V. Lavrenko , R. Manmatha, Automatic image annotation and retrieval using cross-media relevance models, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860459]
• 6: Yohan Jin , Latifur Khan , Lei Wang , Mamoun Awad, Image annotations by combining multiple evidence & wordNet, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101305]
• 7: Lavrenko, V., Manmatha, R., and Jeon, J. "A Model for Learning the Semantics of Pictures," Proc. Advances in Neutral Information Processing Systems, 2003.
• 8: Jia Li , James Z. Wang, Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.9, p.1075-1088, September 2003 [doi>10.1109/TPAMI.2003.1227984]
• 9: Xirong Li , Le Chen , Lei Zhang , Fuzong Lin , Wei-Ying Ma, Image annotation by large-scale content-based image retrieval, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180764]
• 10: Jianbo Shi , Jitendra Malik, Normalized Cuts and Image Segmentation, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.731, June 17-19, 1997
• 11: Y. Mori, H. Takahashi, and R. Oka. Image-to-word transformation based on dividing and vector quantizing images with words. In First International Workshop on Multimedia Intelligent Storage and Retrieval Management, 1999.
• 12: Changhu Wang , Feng Jing , Lei Zhang , Hong-Jiang Zhang, Image annotation refinement using random walk with restarts, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180774]
• 13: Changhu Wang , Feng Jing , Lei Zhang , Hong-Jiang Zhang, Scalable search-based image annotation of personal images, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178714]
• 14: Wang, C., Jing, F., Zhang, L., and Zhang, H. 2007. Content-based Image Annotation Refinement. In Proceedings of the 2007 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR. IEEE Computer Society, Minneapolis, Minnesota, 2007, 1--8.
• 15: Lei Zhang , Le Chen , Feng Jing , Kefeng Deng , Wei-Ying Ma, EnjoyPhoto: a vertical image search engine for enjoying high-quality photos, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180719]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms, Experimentation

Keywords:
Markov model, image annotation