Generic image classification using visual knowledge on the web

ABSTRACT

In this paper, we describe a generic image classification system with an automatic knowledge acquisition mechanism from the World-Wide Web. Due to the recent spread of digital imaging devices, the demand for image recognition of various kinds of real world scenes becomes greater. For realizing it, visual knowledge on various kinds of scenes is required. Then, we propose gathering visual knowledge on real world scenes for generic image classification from the World-Wide Web. Our system gathers a large number of images from the Web automatically and makes use of them as training images for generic image classification. It consists of three modules, which are an image-gathering module, an image-learning module and an image classification module. The image-gathering module gathers images related to given class keywords from the Web automatically. The learning module extracts image features from gathered images and associates them with each class. The image classification module classifies an unknown image into one of the classes corresponding to the class keywords by using the association between image features and classes. In the experiments, we achieved a classification rate 44.6% for generic images by using images gathered from the World-Wide Web automatically as training images.

REFERENCES

• 1: K. Barnard and D. A. Forsyth. Learning the semantics of words and pictures. In Proc. of IEEE International Conference on Computer Vision, volume II, pages 408--415, 2001.
• 2: Serge Belongie , Chad Carson , Hayit Greenspan , Jitendra Malik, Recognition of Images in Large Databases Using a Learning Framework, University of California at Berkeley, Berkeley, CA, 1997
• 3: S. C. Deerwester, S. T. Dumais, T. K. Landauer, G. W. Furnas, and R. A. Harshman. Indexing by latent semantic analysis. Journal of the American Society of Information Science, 41(6):391--407, 1990.
• 4: Yining Deng , b. s. Manjunath, Unsupervised Segmentation of Color-Texture Regions in Images and Video, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.8, p.800-810, August 2001 [doi>10.1109/34.946985]
• 5: P. Duygulu , Kobus Barnard , J. F. G. de Freitas , David A. Forsyth, Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary, Proceedings of the 7th European Conference on Computer Vision-Part IV, p.97-112, May 28-31, 2002
• 6: Charles Frankel , Michael J Swain , Vassilis Athitsos, WebSeer: An Image Search Engine for the World Wide Web, University of Chicago, Chicago, IL, 1996
• 7: James Hafner , Harpreet S. Sawhney , Will Equitz , Myron Flickner , Wayne Niblack, Efficient Color Histogram Indexing for Quadratic Form Distance Functions, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.17 n.7, p.729-736, July 1995 [doi>10.1109/34.391417]
• 8: Y. Mori, H. Takahashi, and R. Oka. Image-to-word transformation based on dividing and vector quantizing images with words. In Proc. of First International Workshop on Multimedia Intelligent Storage and Retrieval Management, 1999.
• 9: N. Otsu and T. Kurita. A new scheme for practical flexible and intelligent vision systems. In Proc. of IAPR Workshop on Computer Vision, pages 431--435, 1988.
• 10: Neil C. Rowe , Brian Frew, Automatic caption localization for photographs on World Wide Web pages, Information Processing and Management: an International Journal, v.34 n.1, p.95-107, 1998 [doi>10.1016/S0306-4573(97)00048-4]
• 11: Yossi Rubner , Jan Puzicha , Carlo Tomasi , Joachim M. Buhmann, Empirical evaluation of dissimilarity measures for color and texture, Computer Vision and Image Understanding, v.84 n.1, p.25-43, October 2001 [doi>10.1006/cviu.2001.0934]
• 12: Yossi Rubner , Carlo Tomasi , Leonidas J. Guibas, The Earth Mover's Distance as a Metric for Image Retrieval, International Journal of Computer Vision, v.40 n.2, p.99-121, Nov. 2000 [doi>10.1023/A:1026543900054]
• 13: Gerard Salton , Christopher Buckley, Term-weighting approaches in automatic text retrieval, Information Processing and Management: an International Journal, v.24 n.5, p.513-523, 1988 [doi>10.1016/0306-4573(88)90021-0]
• 14: G. Salton and C. Buckley. Improving retrieval performance by relevance feedback. Journal of the American Society for Information Science, 41(4):288--297, 1990.
• 15: Stan Sclaroff , Marco La Cascia , Saratendu Sethi, Unifying textual and visual cues for content-based image retrieval on the World Wide Web, Computer Vision and Image Understanding, v.75 n.1-2, p.86-98, July/Aug. 1999 [doi>10.1006/cviu.1999.0765]
• 16: John R. Smith , Shih-Fu Chang, Visually Searching the Web for Content, IEEE MultiMedia, v.4 n.3, p.12-20, July 1997 [doi>10.1109/93.621578]
• 17: James Z. Wang , Jia Li, Learning-based linguistic indexing of pictures with 2--d MHMMs, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641104]
• 18: James Z. Wang , Jia Li , Gio Wiederhold, SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.9, p.947-963, September 2001 [doi>10.1109/34.955109]
• 19: K. Yanai. Image collector: An image-gathering system from the World-Wide Web employing keyword-based search engines. In Proc. of IEEE International Conference on Multimedia and Expo, pages 704--707, 2001.
• 20: K. Yanai. Image collector II: A system for gathering more than one thousand images from the web for one keyword. In Proc. of IEEE International Conference on Multimedia and Expo, volume I, pages 785--788, 2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.m Miscellaneous

General Terms:
Design, Experimentation

Keywords:
image classification, image gathering, web image mining