An adaptive graph model for automatic image annotation

ABSTRACT

Automatic keyword annotation is a promising solution to enable more effective image search by using keywords. In this paper, we propose a novel automatic image annotation method based on manifold ranking learning, in which the visual and textual information are well integrated. Due to complex and unbalanced data distribution and limited prior information in practice, we design two new schemes to make manifold ranking efficient for image annotation. Firstly, we design a new scheme named the Nearest Spanning Chain (NSC) to generate an adaptive similarity graph, which is robust across data distribution and easy to implement. Secondly, the word-to-word correlations obtained from WordNet and the pairwise co-occurrence are taken into consideration to expand the annotations and prune irrelevant annotations for each image. Experiments conducted on standard Corel dataset and web image dataset demonstrate the effectiveness and efficiency of the proposed method for image annotation.

REFERENCES

• 1: Budanitsky, A. and Hirst, G. Semantic distance in WordNet: An experimental, application-oriented evaluation of five measures. In Workshop on WordNet and Other Lexical Resources, 2 nd of the North American Chapter of the ACL, Pittsburgh, 2001.
• 2: Claudio, C., Gianluigi, C., Raimondo, S. Image annotation using SVM. In Proceeding Of Internet imaging IV, Vol. SPIE, 2004.
• 3: Cai, D., Yu, S., Wen, J.R. and Ma, W.Y. VIPS: a vision-based page segmentation algorithm. Microsoft Technical Report (MSR-TR-2003-79), 2003.
• 4: Edward Chang, Kingshy Goh, Gerard Sychay, Gang Wu. CBSA: content-base soft annotation for multimodal image retrieval using bayes point machines. CirSysVideo, pp. 26--38, 13(1), 2003.
• 5: David M. Blei , Michael I. Jordan, Modeling annotated data, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860460]
• 6: Gustavo Carneiro , Nuno Vasconcelos, A database centric view of semantic image annotation and retrieval, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076129]
• 7: Jingrui He , Mingjing Li , Hong-Jiang Zhang , Hanghang Tong , Changshui Zhang, Manifold-ranking based image retrieval, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027531]
• 8: Xiaofei He , Shuicheng Yan , Yuxiao Hu , Hong-Jiang Zhang, Learning a Locality Preserving Subspace for Visual Recognition, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.385, October 13-16, 2003
• 9: Xiaofei He , Deng Cai , Haifeng Liu , Wei-Ying Ma, Locality preserving indexing for document representation, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009012]
• 10: Xiaofei He , Wei-Ying Ma , Hong-Jiang Zhang, Learning an image manifold for retrieval, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027532]
• 11: Zhigang Hua , Xiang-Jun Wang , Qingshan Liu , Hanqing Lu, Semantic knowledge extraction and annotation for web images, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101253]
• 12: J. Jeon , V. Lavrenko , R. Manmatha, Automatic image annotation and retrieval using cross-media relevance models, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860459]
• 13: Jiang, J. and Conrath, D. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings on International Conference on Research in Computational Linguistics, 1997.
• 14: Kobus Barnard , Pinar Duygulu , David Forsyth , Nando de Freitas , David M. Blei , Michael I. Jordan, Matching words and pictures, The Journal of Machine Learning Research, 3, 3/1/2003
• 15: Jia Li , James Z. Wang, Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.9, p.1075-1088, September 2003 [doi>10.1109/TPAMI.2003.1227984]
• 16: Pucher, M. Performance Evaluation of WordNet-based Semantic Relatedness Measures for Word Prediction in Conversational Speech. In Sixth International Workshop on Computational Semantics, Tilburg, Netherlands, 2005.
• 17: P. Duygulu , Kobus Barnard , J. F. G. de Freitas , David A. Forsyth, Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary, Proceedings of the 7th European Conference on Computer Vision-Part IV, p.97-112, May 28-31, 2002
• 18: Jia-Yu Pan , Hyung-Jeong Yang , Christos Faloutsos , Pinar Duygulu, Automatic multimedia cross-modal correlation discovery, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1014135]
• 19: R. Manmatha, V. Lavrenko, and J. Jeon, A Model for Learning the Semantics of Pictures. In Proc. of the 17th Annual Conf. on Neural Information Processing Systems, 2003.
• 20: S. L. Feng, R. Manmatha and V. Lavrenko. Multiple Bernouli Relevance Models for Image and Video Annotation. In Proc. Of CVPR, Washington, DC, June, 2004.
• 21: Jianbo Shi , Jitendra Malik, Normalized Cuts and Image Segmentation, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.731, June 17-19, 1997
• 22: Hanghang Tong , Jingrui He , Mingjing Li , Changshui Zhang , Wei-Ying Ma, Graph based multi-modality learning, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101337]
• 23: Wojciech, M., Hanspeter, P., Matt, B. A data-driven reflectance model. In Proc. of SIGGRAPH, 2003.
• 24: Zhou, D., Bousquet, O., Lal, T.N., Weston, J., and Schölkopf, B. Ranking on Data Manifolds. 18th Annual Conf. on Neural Information Processing System, pp. 169--176, 2003.
• 25: Zhou, D., J. Huang and B. Schölkopf. Learning with local and global consistency. 18 th Annual Conference on Neural Information Processing Systems, 2003.
• 26: Yohan Jin , Latifur Khan , Lei Wang , Mamoun Awad, Image annotations by combining multiple evidence & wordNet, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101305]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms, Experimentation

Keywords:
image annotation, image retrieval, manifold ranking