Autonomous visual model building based on image crawling through internet search engines

ABSTRACT

In this paper, we propose an autonomous learning scheme to automatically build visual semantic concept models from the output data of Internet search engines without any manual labeling work. First of all, images are gathered by crawling through the Internet using a search engine such as Google. Then, we model the search results as "Quasi-Positive Bags" in the Multiple-Instance Learning (MIL) framework. We call this generalized MIL (GMIL). We propose an algorithm called "Bag K-Means" to find the maximum Diverse Density (DD) without the existence of negative bags. A cost function is found as K-Means with special "Bag Distance". We also propose a solution called "Uncertain Labeling Density" (ULD) which describes the target density distribution of instances in the case of quasi-positive bags. A "Bag Fuzzy K-Means" is presented to get the maximum of ULD. By this generalized MIL with ULD, the model for a particular concept is learned from the crawled images of the Internet search engines. Experiments show that our algorithm can get correct models for the concepts we are interested in. Compared to the original Google Image Search, our algorithm shows improved accuracy.

REFERENCES

• 1: Xiaodan Song , Ching-Yung Lin , Ming-Ting Sun, Cross-Modality Automatic Face Model Training from Large Video Databases, Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04) Volume 5, p.91, June 27-July 02, 2004
• 2: Oded Maron , Tomas Lozano-Perez, Learning from ambiguity, 1998
• 3: Oded Maron , Tomás Lozano-Pérez, A framework for multiple-instance learning, Proceedings of the 1997 conference on Advances in neural information processing systems 10, p.570-576, July 1998, Denver, Colorado, United States
• 4: Oded Maron , Aparna Lakshmi Ratan, Multiple-Instance Learning for Natural Scene Classification, Proceedings of the Fifteenth International Conference on Machine Learning, p.341-349, July 24-27, 1998
• 5: Thomas G. Dietterich , Richard H. Lathrop , Tomás Lozano-Pérez, Solving the multiple instance problem with axis-parallel rectangles, Artificial Intelligence, v.89 n.1-2, p.31-71, Jan. 1997 [doi>10.1016/S0004-3702(96)00034-3]
• 6: Robert Amar , Daniel R. Dooly , Sally A. Goldman , Qi Zhang, Multiple-Instance Learning of Real-Valued Data, Proceedings of the Eighteenth International Conference on Machine Learning, p.3-10, June 28-July 01, 2001
• 7: Jun Wang , Jean-Daniel Zucker, Solving the Multiple-Instance Problem: A Lazy Learning Approach, Proceedings of the Seventeenth International Conference on Machine Learning, p.1119-1126, June 29-July 02, 2000
• 8: Stuart Andrews , Thomas Hofmann , Ioannis Tsochantaridis, Multiple instance learning with generalized support vector machines, Eighteenth national conference on Artificial intelligence, p.943-944, July 28-August 01, 2002, Edmonton, Alberta, Canada
• 9: A. Schneider, "Weighted possibilistic clustering algorithms", Proc. of the 9th IEEE International Conference on Fuzzy Systems. Texas, 2000, 1, 176--180.
• 10: R.N. Dave, and R. Krishnapuram, "Robust clustering methods: a unified view", IEEE Transactions on Fuzzy Systems, May 1997, 5, 2, 270--293
• 11: A. Amir, M. Berg, S.-F. Chang, G. Iyengar, C.-Y. Lin, A. Natsev, C. Neti, H. Nock, M. Naphade, W. Hsu, J. R. Smith, B. Tseng, Y. Wu, D. Zhang, "IBM Research TRECVID-2003 Video Retrieval Syste<http://www.ctr.columbia.edu/papers_advent/03/ibmcutrec03.html>," Proc. of TRECVID 2003 Workshop.
• 12: Paul Viola , Michael J. Jones, Robust Real-Time Face Detection, International Journal of Computer Vision, v.57 n.2, p.137-154, May 2004 [doi>10.1023/B:VISI.0000013087.49260.fb]
• 13: R. Lienhart, A. Kuranov <http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/k/Kuranov:Alexander.html> and V. Pisarevsky <http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/p/Pisarevsky:Vadim.html>, "Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection," DAGM-Symposium <http://www.informatik.uni-trier.de/~ley/db/conf/dagm/dagm2003.html>, 2003, 297--304.
• 14: M.J. Jones and J.M. Rehg, "Statistical color models with application to skin detection," Proc. of CVPR, 1999, 274--280.
• 15: Q. Zhang, and S. A. Goldman, "EM-DD: an improved multi-instance learning technique", Proc. of Advances in Neural Information Processing Systems, Cambridge, MA, MIT Press, 2002, 1073--1080.
• 16: C.-Y. Lin, B. L. Tseng and J. R. Smith, "Video Collaborative Annotation Forum: Establishing Ground-Truth Labels on Large Multimedia Datasets," Proc. of NIST Text Retrieval Conf. (TREC), Gaithersburg, MD, November 2003.
• 17: Ching-Yung Lin , Belle L. Tseng , Milind Naphade , Apostol Natsev , John R. Smith, MPEG-7 video automatic labeling system, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957033]
• 18: Y. Rui, T. Huang, M. Ortega and S. Mehrotra, "Relevance Feedback: A Power Tool for Interactive Content-Based Image Retrieval," IEEE Transactions on Circuits and Systems for Video Technology, 8/5 (1998), 644--655.
• 19: P. Aigrain, H. Zhang, and D. Petkovic, ''Content-Based Representation and Retrieval of Visual Media: A State-of-theArt Review,'' Multimedia Tools and Applications, Vol. 3, 179--202, November 1996.
• 20: T. Minka, "An Image Database Browser that Learns from User Interaction," MIT Technical Report TR#365, MIT, 1996.
• 21: Sally A. Goldman , Yan Zhou, Enhancing Supervised Learning with Unlabeled Data, Proceedings of the Seventeenth International Conference on Machine Learning, p.327-334, June 29-July 02, 2000
• 22: Y. Wu, Q. Tian, T. S. Huang, "Discriminant-EM Algorithm with Application to Image Retrieval," Proc. of CVPR, Vol. I, pp. 222--227, Hilton Head Island, SC, June, 2000
• 23: G. Salton, and C. Buckle, "Improving retrieval performance by relevance feedback," Journal of the American Society for Information Science Vol. 41, 288--297, 1990
• 24: David D. Lewis , William A. Gale, A sequential algorithm for training text classifiers, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.3-12, July 03-06, 1994, Dublin, Ireland

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms

Keywords:
automatic training, content-based image retrieval, cross-modality, image crawling, multiple-instance learning, quasi-positive bag, uncertain labeling density