What is a complete set of keywords for image description & annotation on the web

ABSTRACT

Does there exist a compact set of keywords that can completely and effectively cover the image annotation problem by expanding from it? In this paper, we answer this question by presenting a complete set framework for image annotation, which is motivated by the existence of semantic ontology. To generate this set, we propose a cross model optimization strategy from both textual and visual information for topic decomposition, based on a so-called Bipartite LSA model, which minimize multimodal error energy functions in a probabilistic Latent Semantic Analysis model. To achieve complete set based annotation, we present a Gaussian-Kernel-Generative process based keyword generation procedure, which analogizes keyword annotation in a probabilistic generative manner. A group of experiments is performed on Washington University image database and 80,000 Flickr images with comparisons to the state-of-the-arts. Finally, potential advantages and future improvements of our framework are discussed outside the scope of topic modeling.

REFERENCES

• 1: S. Zinger, C. Millet, B. Mathieu, G. Grefenstette, P. Hède, and P.-A. Moëllic. 2005. Extracting an Ontology of Portrayable Objects from WordNet. In Proceedings of the MUSCLE/ImageCLEF Workshop on Image and Video Retrieval Evaluation, 2005.
• 2: T.K. Landauer, P.W. Foltz and D. Laham. 1998. An Introduction to Latent Semantic Analysis. Discourse Processes, 1998.
• 3: T. Hofmann. 1999. Probabilistic Latent Semantic Indexing. In Proceedings of the 22nd annual international ACM SIGIR, 1999.
• 4: D. Blei, A. Ng and M. Jordan. 2003. Latent Dirichlet allocation. Journal of Machine Learning Research, 2003, 3:993--1022.
• 5: K. Barnard, P. Duygulu, N. de Freitas, D. Forsyth, D. Blei and M. Jordan. 2003. Matching words and pictures. Journal of Machine Learning Research, 2003.
• 6: D. Blei and M. I. Jordan. 2003. Modeling annotated data. In Proceedings of the 26th Intl. ACM SIGIR, 2003.
• 7: X. Rui, M. Li, Z. Li, W.Y. Ma and N. Yu. 2007. Bipartite graph reinforcement model for web image annotation. In Proceedings of the ACM International Conference on Multimedia, 2007.
• 8: Y. Lu, L. Zhang, Q. Tian and W.Y. Ma. 2008. What are the High-Level Concepts with Small Semantic Gaps?. CVPR, 2008.
• 9: Xianming Liu, Rongrong Ji, Hongxun Yao, Pengfei Xu, Xiaoshuai Sun, Tianqiang Liu. "Cross-Media Manifold Learning for Image Retrieval & Annotation". ACM MIR 2008, pp: 141--148, 2008.
• 10: Rongrong Ji, Hongxun Yao, "Visual & Textual Fusion for Region Retrieval: From Both Fuzzy Matching and Bayesian Reasoning Aspects," ACM Conference on Multimedia Information Retrieval (MIR), pp.159--168, 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Design, Experimentation

Keywords:
image annotation, keyword selection, ontology, semantic items