Integrated graph-based semi-supervised multiple/single instance learning framework for image annotation

ABSTRACT

Recently, many learning methods based on multiple-instance (local) or single-instance (global) representations of images have been proposed for image annotation. Their performances on image annotation, however, are mixed as for certain concepts the single-instance representations of images are more suitable, while for some other concepts the multiple-instance representations are better. Thus in this paper, we explore an unified learning framework that combines the multiple-instance and single-instance representations for image annotation. More specifically, we propose an integrated graph-based semi-supervised learning framework to utilize these two types of representations simultaneously, and explore an effective and computationally efficient strategy to convert the multiple-instance representation into a single-instance one. Experiments conducted on the Coral image dataset show the effectiveness and efficiency of the proposed integrated framework.

REFERENCES

• 1: O. Chapelle, A. Zien, and B. Scholkopf. Semi-Supervised Learning. MIT Press, 2006.
• 2: Yixin Chen , James Z. Wang, Image Categorization by Learning and Reasoning with Regions, The Journal of Machine Learning Research, 5, p.913-939, 12/1/2004
• 3: Yixin Chen , Jinbo Bi , James Z. Wang, MILES: Multiple-Instance Learning via Embedded Instance Selection, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.28 n.12, p.1931-1947, December 2006 [doi>10.1109/TPAMI.2006.248]
• 4: C. Cusano, G. Ciocca and R. Schettini. Image Annotation Using SVM. Proceedings of Internet Imaging, vol. SPIE 5304, 2004.
• 5: Yining Deng , b. s. Manjunath, Unsupervised Segmentation of Color-Texture Regions in Images and Video, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.8, p.800-810, August 2001 [doi>10.1109/34.946985]
• 6: Thomas G. Dietterich , Richard H. Lathrop , Tomás Lozano-Pérez, Solving the multiple instance problem with axis-parallel rectangles, Artificial Intelligence, v.89 n.1-2, p.31-71, Jan. 1997 [doi>10.1016/S0004-3702(96)00034-3]
• 7: J. Jeon , V. Lavrenko , R. Manmatha, Automatic image annotation and retrieval using cross-media relevance models, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860459]
• 8: Oded Maron , Aparna Lakshmi Ratan, Multiple-Instance Learning for Natural Scene Classification, Proceedings of the Fifteenth International Conference on Machine Learning, p.341-349, July 24-27, 1998
• 9: Rouhollah Rahmani , Sally A. Goldman, MISSL: multiple-instance semi-supervised learning, Proceedings of the 23rd international conference on Machine learning, p.705-712, June 25-29, 2006, Pittsburgh, Pennsylvania [doi>10.1145/1143844.1143933]
• 10: M. Stricker and M. Orengo. Similarity of color images. Proceedings of Storage and Retrieval for Image and Video Databases (SPIE 2420), 2000.
• 11: Jinhui Tang , Xian-Sheng Hua , Guo-Jun Qi , Meng Wang , Tao Mei , Xiuqing Wu, Structure-sensitive manifold ranking for video concept detection, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291430]
• 12: Jinhui Tang , Xian-Sheng Hua , Guo-Jun Qi , Xiuqing Wu, Typicality ranking via semi-supervised multiple-instance learning, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291296]
• 13: J. Tang, X.-S. Hua, G.-J. Qi, Y. Song and X. Wu. Video Annotation Based on Kernel Linear Neighborhood Propagation. IEEE Transactions on Multimedia, Vol.10, Issue 4, 2008.
• 14: Hanghang Tong , Jingrui He , Mingjing Li , Changshui Zhang , Wei-Ying Ma, Graph based multi-modality learning, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101337]
• 15: Changhu Wang , Feng Jing , Lei Zhang , Hong-Jiang Zhang, Image annotation refinement using random walk with restarts, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180774]
• 16: D. Wang, J. Li, and B. Zhang. Multiple-instance learning via random walk. European Conference on Machine Learning, 2006.
• 17: Fei Wang , Changshui Zhang, Label propagation through linear neighborhoods, Proceedings of the 23rd international conference on Machine learning, p.985-992, June 25-29, 2006, Pittsburgh, Pennsylvania [doi>10.1145/1143844.1143968]
• 18: Meng Wang , Xian-Sheng Hua , Xun Yuan , Yan Song , Li-Rong Dai, Optimizing multi-graph learning: towards a unified video annotation scheme, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291431]
• 19: Semi-Supervised Learning with Graphs. PhD Thesis, CMU-LTI-05-192, 2005.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

General Terms:
Algorithms, Experimentation

Keywords:
image annotation, multiple/single instance learning