Representing shape with a spatial pyramid kernel

ABSTRACT

The objective of this paper is classifying images by the object categories they contain, for example motorbikes or dolphins. There are three areas of novelty. First, we introduce a descriptor that represents local image shape and its spatial layout, together with a spatial pyramid kernel. These are designed so that the shape correspondence between two images can be measured by the distance between their descriptors using the kernel. Second, we generalize the spatial pyramid kernel, and learn its level weighting parameters (on a validation set). This significantly improves classification performance. Third, we show that shape and appearance kernels may be combined (again by learning parameters on a validation set).

Results are reported for classification on Caltech-101 and retrieval on the TRECVID 2006 data sets. For Caltech-101 it is shown that the class specific optimization that we introduce exceeds the state of the art performance by more than 10%.

REFERENCES

• 1: Francis R. Bach , Gert R. G. Lanckriet , Michael I. Jordan, Multiple kernel learning, conic duality, and the SMO algorithm, Proceedings of the twenty-first international conference on Machine learning, p.6, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015424]
• 2: A Bissacco, M. H. Yang, and S. Soatto. Detecting humans via their pose. In NIPS, 2006.
• 3: Gunilla Borgefors, Hierarchical Chamfer Matching: A Parametric Edge Matching Algorithm, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.10 n.6, p.849-865, November 1988 [doi>10.1109/34.9107]
• 4: A. Bosch, A. Zisserman, and X. Muoz. Scene classification via plsa. In Proc. ECCV, 2006.
• 5: C. Chang and C. Lin. LIBSVM: a library for support vector machines, 2001. Software available at urlhttp://www.csie.ntu.edu.tw/cjlin/libsvm.
• 6: G. Csurka, C. Bray, C. Dance, and L. Fan. Visual categorization with bags of keypoints. In Workshop on Statistical Learning in Computer Vision, ECCV, 2004.
• 7: Navneet Dalal , Bill Triggs, Histograms of Oriented Gradients for Human Detection, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.886-893, June 20-26, 2005 [doi>10.1109/CVPR.2005.177]
• 8: Li Fei-Fei , Rob Fergus , Pietro Perona, Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories, Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04) Volume 12, p.178, June 27-July 02, 2004
• 9: P. F. Felzenszwalb. Learning models for object recognition. In Proc. CVPR, 2001.
• 10: D. Gavrila and V. Philomin. Real-time object detection for "smart" vehicles. In Proc. ICCV, 1999.
• 11: Kristen Grauman , Trevor Darrell, The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1458-1465, October 17-20, 2005 [doi>10.1109/ICCV.2005.239]
• 12: D. Haussler. Convolution kernels on discrete structures. Technical Report UCS-CRL-99-10, 1999.
• 13: Anil K. Jain , Kalle Karu, Learning Texture Discrimination Masks, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.18 n.2, p.195-205, February 1996 [doi>10.1109/34.481543]
• 14: Svetlana Lazebnik , Cordelia Schmid , Jean Ponce, Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.2169-2178, June 17-22, 2006 [doi>10.1109/CVPR.2006.68]
• 15: Darrin P. Lewis , Tony Jebara , William Stafford Noble, Nonstationary kernel combination, Proceedings of the 23rd international conference on Machine learning, p.553-560, June 25-29, 2006, Pittsburgh, Pennsylvania [doi>10.1145/1143844.1143914]
• 16: Tony Lindeberg , Jonas Gårding, Shape-adapted smoothing in estimation of 3-D depth cues from affine distortions of local 2-D brightness structure, Proceedings of the third European conference on Computer vision (vol. 1), p.389-400, June 1994, Stocklholm, Sweden
• 17: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 18: Jim Mutch , David G. Lowe, Multiclass Object Recognition with Sparse, Localized Features, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.11-18, June 17-22, 2006 [doi>10.1109/CVPR.2006.200]
• 19: Andreas Opelt , Axel Pinz , Andrew Zisserman, Incremental learning of object detectors using a visual shape alphabet, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.3-10, June 17-22, 2006 [doi>10.1109/CVPR.2006.153]
• 20: Bastian Leibe , Edgar Seemann , Bernt Schiele, Pedestrian Detection in Crowded Scenes, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.878-885, June 20-26, 2005 [doi>10.1109/CVPR.2005.272]
• 21: Jamie Shotton , Andrew Blake , Roberto Cipolla, Contour-Based Learning for Object Detection, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.503-510, October 17-20, 2005 [doi>10.1109/ICCV.2005.63]
• 22: B. Stenger , A. Thayananthan , P. H. S. Torr , R. Cipolla, Filtering Using a Tree-Based Estimator, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.1063, October 13-16, 2003
• 23: Martin Szummer , Rosalind W. Picard, Indoor-Outdoor Image Classification, Proceedings of the 1998 International Workshop on Content-Based Access of Image and Video Databases (CAIVD '98), p.42, January 03-03, 1998
• 24: J. Thureson and S. Carlsson. Appearance based qualitative image description for object class recognition. In Proc. ECCV, 2004.
• 25: Gang Wang , Ye Zhang , Li Fei-Fei, Using Dependent Regions for Object Categorization in a Generative Framework, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.1597-1604, June 17-22, 2006 [doi>10.1109/CVPR.2006.324]
• 26: Hao Zhang , Alexander C. Berg , Michael Maire , Jitendra Malik, SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.2126-2136, June 17-22, 2006 [doi>10.1109/CVPR.2006.301]
• 27: J. Zhang , M. Marszałek , S. Lazebnik , C. Schmid, Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study, International Journal of Computer Vision, v.73 n.2, p.213-238, June 2007 [doi>10.1007/s11263-006-9794-4]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis

General Terms:
Measurement, Performance

Keywords:
object and video retrieval, shape features, spatial pyramid kernel