A semi-naïve Bayesian method incorporating clustering with pair-wise constraints for auto image annotation

ABSTRACT

We propose a novel approach for auto image annotation. In our approach, we first perform the segmentation of images into regions, followed by clustering of regions, before learning the relationship between concepts and region clusters using the set of training images with pre-assigned concepts. The main focus of this paper is two-fold. First, in the learning stage, we perform clustering of regions into region clusters by incorporating pair-wise constraints which are derived by considering the language model underlying the annotations assigned to training images. Second, in the annotation stage, we employ a semi-naïve Bayes model to compute the posterior probability of concepts given the region clusters. Experiment results show that our proposed system utilizing these two strategies outperforms the state-of-the-art techniques in annotating large image collection.

REFERENCES

• 1: Kobus Barnard , Pinar Duygulu , David Forsyth , Nando de Freitas , David M. Blei , Michael I. Jordan, Matching words and pictures, The Journal of Machine Learning Research, 3, 3/1/2003
• 2: Mikhail Bilenko , Sugato Basu , Raymond J. Mooney, Integrating constraints and metric learning in semi-supervised clustering, Proceedings of the twenty-first international conference on Machine learning, p.11, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015360]
• 3: David M. Blei , Michael I. Jordan, Modeling annotated data, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860460]
• 4: Chad Carson , Serge Belongie , Hayit Greenspan , Jitendra Malik, Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.8, p.1026-1038, August 2002 [doi>10.1109/TPAMI.2002.1023800]
• 5: J. Jeon , V. Lavrenko , R. Manmatha, Automatic image annotation and retrieval using cross-media relevance models, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860459]
• 6: Igor Kononenko, Semi-naive Bayesian classifier, Proceedings of the European working session on learning on Machine learning, p.206-219, March 1991, Porto, Portugal
• 7: Lavrenko, V., Manmatha, R. & Jeon, J.: A model for learning the semantics of pictures. Neural Information Processing System (NIPS), 2003.
• 8: Jia Li , James Z. Wang, Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.9, p.1075-1088, September 2003 [doi>10.1109/TPAMI.2003.1227984]
• 9: Jimin Liu , Tat-Seng Chua, Building semantic perceptron net for topic spotting, Proceedings of the 39th Annual Meeting on Association for Computational Linguistics, p.378-385, July 06-11, 2001, Toulouse, France [doi>10.3115/1073012.1073061]
• 10: Mori, Y., Takahashi, H. & Oka, R.: Image-to-word transformation based on dividing and vector quantizing images with words. First Int'l Workshop on multimedia Intelligent Storage & Retrieval Management, 1999.
• 11: Platt, J.C: "Probabilities for SV machines," In A.Smola, P. Bartlett, B. Scholkopf, and D. Schuurmans,editors, Advances in Large Margin Classifiers, pages 61--74, Cambridge, MA, 1999, MIT Press.
• 12: Shi, R., Feng, H.M., Chua, T.-S. & Lee, C.-H., An adaptive image content representation and segmentation approach to automatic image annotation. Int'l Conf. on Image and Video Retrieval, July 21-23, 2004
• 13: Kiri Wagstaff , Claire Cardie , Seth Rogers , Stefan Schrödl, Constrained K-means Clustering with Background Knowledge, Proceedings of the Eighteenth International Conference on Machine Learning, p.577-584, June 28-July 01, 2001
• 14: http://www.loc.gov/rr/print/tgm1/

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Object recognition

General Terms:
Algorithms, Experimentation

Keywords:
image annotation, pair-wise constraint, semi-naïve Bayes, semi-supervised clustering