Unsupervised modeling and recognition of object categories with combination of visual contents and geometric similarity links

ABSTRACT

This paper proposes a probabilistic approach for unsupervised modeling and recognition of object categories which combines two types of complementary visual evidence, visual contents and inter-connected links between the images. By doing so, our approach not only increases modeling and recognition performance but also provides possible solutions to several problems including modeling of geometric information, computational complexity, and the inherent ambiguity of visual words. Our approach can be incorporated in any generative models, but here we consider two popular models, pLSA and LDA. Experimental results show that the topic models updated by adding link analysis terms significantly improve the standard pLSA and LDA models. Furthermore, we presented competitive performances on unsupervised modeling, ranking of training images, classification of unseen images, and localization tasks with MSRC and PASCAL2005 datasets.

REFERENCES

• 1: Alexander C. Berg , Tamara L. Berg , Jitendra Malik, Shape Matching and Object Recognition Using Low Distortion Correspondences, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.26-33, June 20-26, 2005 [doi>10.1109/CVPR.2005.320]
• 2: David M. Blei , Andrew Y. Ng , Michael I. Jordan, Latent dirichlet allocation, The Journal of Machine Learning Research, 3, p.993-1022, 3/1/2003 [doi>10.1162/jmlr.2003.3.4-5.993]
• 3: L. Cao and L. Fei-Fei. Spatial coherent latent topic model for concurrent object segmentation and classification. In ICCV, 2007.
• 4: D. Cohn and T. Hofmann. The missing link - a probabilistic model of document content and hypertext connectivity. In NIPS, 2001.
• 5: E. Erosheva, S. Fienberg, and J. Lafferty. Mixed-membership models of scientific publications. PNAS, 101(1):220--5227, 2004.
• 6: L. Fei-Fei. Bag of words models: Recognizing and learning object categories. In CVPR Short Courses, 2007.
• 7: L. Fei-Fei and P. Perona. A bayesian hierarchical model for learning natural scene categories. In CVPR, 2005.
• 8: A. Frome, Y. Singer, F. Sha, and J. Malik. Learning globally-consistent local distance functions for shape-based image retrieval and classification. In ICCV, 2007.
• 9: Kristen Grauman , Trevor Darrell, The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1458-1465, October 17-20, 2005 [doi>10.1109/ICCV.2005.239]
• 10: Kristen Grauman , Trevor Darrell, Unsupervised Learning of Categories from Sets of Partially Matching Image Features, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.19-25, June 17-22, 2006 [doi>10.1109/CVPR.2006.322]
• 11: T. Hofmann. Probabilistic latent semantic analysis. In NIPS, 1999.
• 12: J. Winn , A. Criminisi , T. Minka, Object Categorization by Learned Universal Visual Dictionary, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1800-1807, October 17-20, 2005 [doi>10.1109/ICCV.2005.171]
• 13: G. Kim, C. Faloutsos, and M. Hebert. Unsupervised modeling of object categories using link analysis techniques. In CVPR, 2008.
• 14: Svetlana Lazebnik , Cordelia Schmid , Jean Ponce, Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.2169-2178, June 17-22, 2006 [doi>10.1109/CVPR.2006.68]
• 15: Marius Leordeanu , Martial Hebert, A Spectral Technique for Correspondence Problems Using Pairwise Constraints, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1482-1489, October 17-20, 2005 [doi>10.1109/ICCV.2005.20]
• 16: Marius Leordeanu , Martial Hebert, Efficient MAP approximation for dense energy functions, Proceedings of the 23rd international conference on Machine learning, p.545-552, June 25-29, 2006, Pittsburgh, Pennsylvania [doi>10.1145/1143844.1143913]
• 17: D. Liu and T. Chen. Unsupervised image categorization and object localization using topic models and correspondences between images. In ICCV, 2007.
• 18: Hongyu Liu , Evangelos Milios , Jeannette Janssen, Probabilistic models for focused web crawling, Proceedings of the 6th annual ACM international workshop on Web information and data management, November 12-13, 2004, Washington DC, USA [doi>10.1145/1031453.1031458]
• 19: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 20: Krystian Mikolajczyk , Cordelia Schmid, Scale & Affine Invariant Interest Point Detectors, International Journal of Computer Vision, v.60 n.1, p.63-86, October 2004 [doi>10.1023/B:VISI.0000027790.02288.f2]
• 21: R. Nallapati and W. Cohen. Link-plsa-lda: A new unsupervised model for topics and influence in blogs. In ICWSM, 2008.
• 22: J. C. Niebles and L. Fei-Fei. A hierarchical model of shape and appearance for human action classification. In CVPR, 2007.
• 23: M. Richardson and P. Domingos. The intelligent surfer: Probabilistic combination of link and content information in pagerank. In NIPS, 2002.
• 24: Bryan C. Russell , William T. Freeman , Alexei A. Efros , Josef Sivic , Andrew Zisserman, Using Multiple Segmentations to Discover Objects and their Extent in Image Collections, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.1605-1614, June 17-22, 2006 [doi>10.1109/CVPR.2006.326]
• 25: Josef Sivic , Bryan C. Russell , Alexei A. Efros , Andrew Zisserman , William T. Freeman, Discovering Objects and their Localization in Images, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.370-377, October 17-20, 2005 [doi>10.1109/ICCV.2005.77]
• 26: Erik B. Sudderth , Antonio Torralba , William T. Freeman , Alan S. Willsky, Learning Hierarchical Models of Scenes, Objects, and Parts, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1331-1338, October 17-20, 2005 [doi>10.1109/ICCV.2005.137]
• 27: Ulrike Luxburg, A tutorial on spectral clustering, Statistics and Computing, v.17 n.4, p.395-416, December 2007 [doi>10.1007/s11222-007-9033-z]
• 28: X. Wang and E. Grimson. Spatial latent dirichlet allocation. In NIPS, 2007.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Perceptual reasoning

General Terms:
Algorithms, Experimentation

Keywords:
image retrieval, object recognition