Semi-supervised learning of object categories from paired local features

ABSTRACT

This paper presents a semi-supervised learning (SSL) approach to find similarities of images using statistics of local matches. SSL algorithms are well known for leveraging a large amount of unlabeled data as well as a small amount of labeled data to boost classification performance. Our approach proposes to formulate the problem of matching two images as an SSL based classification problem of image pairs with a minimal amount of labeled pairs. We apply a Gaussian random field model to represent each image pair as vertices in a weighted graph and the optimal configuration of the field is obtained by harmonic energy minimization. A symmetrical feature selection criterion is first introduced to select robust matches of local keypoints between two images. The Mallows distance is then adopted to combine multiple cues from statistics of local matches. Our experiments confirm that our SSL based approach not only boost classification performance but also improve robustness of the learned category model using only simple local keypoint features.

REFERENCES

• 1: Avrim Blum , Tom Mitchell, Combining labeled and unlabeled data with co-training, Proceedings of the eleventh annual conference on Computational learning theory, p.92-100, July 24-26, 1998, Madison, Wisconsin, United States [doi>10.1145/279943.279962]
• 2: W.J. Christmas, J. Kittler, and M. Petrou. Matching of Road Segments Using Probabilistic Relaxation: reducing the computational requirements. Imaging and Vision for control and guidance of aerospace vehicles, 2004.
• 3: Ralph Ewerth , Bernd Freisleben, Semi-supervised learning for semantic video retrieval, Proceedings of the 6th ACM international conference on Image and video retrieval, p.154-161, July 09-11, 2007, Amsterdam, The Netherlands [doi>10.1145/1282280.1282308]
• 4: Li Fei-Fei , Rob Fergus , Pietro Perona, One-Shot Learning of Object Categories, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.28 n.4, p.594, April 2006 [doi>10.1109/TPAMI.2006.79]
• 5: R. Fergus, P. Perona, and A. Zisserman. Object class recognition by unsupervised scale-invariant learning. CVPR, 2003.
• 6: L. Grady and G. Funka-Lea. Multi-label image segmentation for medical applications based on graph-theoretic electrical potentials. ECCV Workshop, 2004.
• 7: Kristen Grauman , Trevor Darrell, Unsupervised Learning of Categories from Sets of Partially Matching Image Features, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.19-25, June 17-22, 2006 [doi>10.1109/CVPR.2006.322]
• 8: T. Hertz, N. Shental, A. Bar-Hillel, and D. Weinshall. Enhancing image and video retrieval: Learning via equivalence constraints. CVPR, 2003.
• 9: Alexey Kostin , Josef Kittler , William Christmas, Object recognition by symmetrised graph matching using relaxation labelling with an inhibitory mechanism, Pattern Recognition Letters, v.26 n.3, p.381-393, February 2005 [doi>10.1016/j.patrec.2004.10.020]
• 10: H. Kuck, P. Carbonetto, and N. de Freitas. A constrained semi-supervised learning approach to data association. ECCV, 2004.
• 11: Svetlana Lazebnik , Cordelia Schmid , Jean Ponce, Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.2169-2178, June 17-22, 2006 [doi>10.1109/CVPR.2006.68]
• 12: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 13: C. L. Mallows. A note on asymptotic joint normality. Annals of Math. Stat., 1972.
• 14: K. Mikolajczyk , C. Schmid, An Affine Invariant Interest Point Detector, Proceedings of the 7th European Conference on Computer Vision-Part I, p.128-142, May 28-31, 2002
• 15: Chong-Wah Ngo , Wan-Lei Zhao , Yu-Gang Jiang, Fast tracking of near-duplicate keyframes in broadcast domain with transitivity propagation, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180827]
• 16: Ahmed Rebai , Alexis Joly , Nozha Boujemaa, Interpretability based interest points detection, Proceedings of the 6th ACM international conference on Image and video retrieval, p.33-40, July 09-11, 2007, Amsterdam, The Netherlands [doi>10.1145/1282280.1282285]
• 17: J. Ros, C. Laurent, and G. Lefebvre. A cascade of unsupervised and supervised neural networks for natural image classification. CIVR, 2006.
• 18: Chuck Rosenberg , Martial Hebert , Henry Schneiderman, Semi-Supervised Self-Training of Object Detection Models, Proceedings of the Seventh IEEE Workshops on Application of Computer Vision (WACV/MOTION'05) - Volume 1, p.29-36, January 05-07, 2005 [doi>10.1109/ACVMOT.2005.107]
• 19: Frederik Schaffalitzky , Andrew Zisserman, Multi-view Matching for Unordered Image Sets, or "How Do I Organize My Holiday Snaps?", Proceedings of the 7th European Conference on Computer Vision-Part I, p.414-431, May 28-31, 2002
• 20: Nicu Sebe , Theo Gevers , Sietse Dijkstra , Joost van de Weije, Evaluation of Intensity and Color Corner Detectors for Affine Invariant Salient Regions, Proceedings of the 2006 Conference on Computer Vision and Pattern Recognition Workshop, p.18, June 17-22, 2006 [doi>10.1109/CVPRW.2006.75]
• 21: J. Sivic, B. C. Russell, A. A. Efros, A. Zisserman, and W. T.Freeman. Discovering object categories in image collections, 2005. MIT-CSAIL-TR-2005-012.
• 22: J. Stottinger, J. Amores, N. Sebe, A. Hanbury, N. Boujemaa, and T. Gevers. Object categorisation with color interest points. CIVR, Demo, 2007.
• 23: Q. Tian, J. Yu, Q. Xue, and N. Sebe. A new analysis of the value of unlabeled data in semi-supervised learning for image retrieval. ICME, 2004.
• 24: Juyang Weng , Narendra Ahuja , Thomas S. Huang, Matching Two Perspective Views, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.14 n.8, p.806-825, August 1992 [doi>10.1109/34.149592]
• 25: Wen Wu , Jie Yang, SmartLabel: an object labeling tool using iterated harmonic energy minimization, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180839]
• 26: Hao Zhang , Alexander C. Berg , Michael Maire , Jitendra Malik, SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.2126-2136, June 17-22, 2006 [doi>10.1109/CVPR.2006.301]
• 27: W. Zhao, Y.G. Jiang, and C.W. Ngo. Keyframe retrieval by keypoints: Can point to point matching help?. CIVR, 2006.
• 28: W. Zhao, Y.G. Jiang, and C.W. Ngo. Keyframe retrieval by keypoints: Can point to point matching help?. CIVR, 2006.
• 29: X.J. Zhu, Z. Ghahramani, and J. Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. ICML, 2003.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Object recognition

General Terms:
Algorithms

Keywords:
object classification, semi-supervised learning