Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

We start from the state-of-the-art Bag of Words pipeline that in the 2008 benchmarks of TRECvid and PASCAL yielded the best performance scores. We have contributed to that pipeline, which now forms the basis to compare various fast alternatives for all of its components: (i) For descriptor extraction we propose a fast algorithm to densely sample SIFT and SURF, and we compare several variants of these descriptors. (ii) For descriptor projection we compare a k-means visual vocabulary with a Random Forest. As a preprojection step we experiment with PCA on the descriptors to decrease projection time. (iii) For classification we use Support Vector Machines and compare the x² kernel with the RBF kernel. Our results lead to a 10-fold speed increase without any loss of accuracy and to a 30-fold speed increase with 17% loss of accuracy, where the latter system does real-time classification at 26 images per second.

REFERENCES

• 1: Herbert Bay , Andreas Ess , Tinne Tuytelaars , Luc Van Gool, Speeded-Up Robust Features (SURF), Computer Vision and Image Understanding, v.110 n.3, p.346-359, June, 2008 [doi>10.1016/j.cviu.2007.09.014]
• 2: M. Everingham and J. Winn. The Pascal VOC challenge 2007 development kit. Technical report, University of Leeds, 2007.
• 3: Pierre Geurts , Damien Ernst , Louis Wehenkel, Extremely randomized trees, Machine Learning, v.63 n.1, p.3-42, April 2006 [doi>10.1007/s10994-006-6226-1]
• 4: J. M. Geusebroek, A. W. M. Smeulders, and J. van de Weijer. Fast anisotropic gauss filtering. IEEE Transactions on Image Processing, 12: 938--943, 2003.
• 5: M. Grabner, H. Grabner, and H. Bischof. Fast approximated SIFT. In ACCV, 2006.
• 6: Yu-Gang Jiang , Chong-Wah Ngo , Jun Yang, Towards optimal bag-of-features for object categorization and semantic video retrieval, Proceedings of the 6th ACM international conference on Image and video retrieval, p.494-501, July 09-11, 2007, Amsterdam, The Netherlands [doi>10.1145/1282280.1282352]
• 7: Frederic Jurie , Bill Triggs, Creating Efficient Codebooks for Visual Recognition, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.604-610, October 17-20, 2005 [doi>10.1109/ICCV.2005.66]
• 8: Svetlana Lazebnik , Cordelia Schmid , Jean Ponce, Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.2169-2178, June 17-22, 2006 [doi>10.1109/CVPR.2006.68]
• 9: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 10: M. Marszałek, C. Schmid, H. Harzallah, and J. van de Weijer. Learning representations for visual object class recognition. Pascal VOC 2007 challenge workshop. ICCV, 2007.
• 11: Krystian Mikolajczyk , Bastian Leibe , Bernt Schiele, Local Features for Object Class Recognition, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1792-1799, October 17-20, 2005 [doi>10.1109/ICCV.2005.146]
• 12: K. Mikolajczyk and J. Matas. Improving descriptors for fast tree matching by optimal linear projection. In ICCV, 2007.
• 13: Krystian Mikolajczyk , Cordelia Schmid, A Performance Evaluation of Local Descriptors, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.10, p.1615-1630, October 2005 [doi>10.1109/TPAMI.2005.188]
• 14: Frank Moosmann , Eric Nowak , Frederic Jurie, Randomized Clustering Forests for Image Classification, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.30 n.9, p.1632-1646, September 2008 [doi>10.1109/TPAMI.2007.70822]
• 15: F. Moosmann, B. Triggs, and F. Jurie. Fast discriminative visual codebooks using randomized clustering forests. In NIPS, pages 985--992, 2006.
• 16: David Nister , Henrik Stewenius, Scalable Recognition with a Vocabulary Tree, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.2161-2168, June 17-22, 2006 [doi>10.1109/CVPR.2006.264]
• 17: J. Shotton, M. Johnson, and R. Cipolla. Semantic texton forests for image categorization and segmentation. In CVPR, 2008.
• 18: Alan F. Smeaton , Paul Over , Wessel Kraaij, Evaluation campaigns and TRECVid, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178722]
• 19: M. A. Tahir, K. van de Sande, J. Uijlings, F. Yan, X. Li, K. Mikolajczyk, J. Kittler, T. Gevers, and A. Smeulders. Uva and surrey @ pascal voc 2008. Pascal VOC 2008 challenge workshop. ECCV, 2008.
• 20: Koen E.A. van de Sande , Theo Gevers , Cees G.M. Snoek, A comparison of color features for visual concept classification, Proceedings of the 2008 international conference on Content-based image and video retrieval, July 07-09, 2008, Niagara Falls, Canada [doi>10.1145/1386352.1386376]
• 21: J. Zhang , M. Marszałek , S. Lazebnik , C. Schmid, Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study, International Journal of Computer Vision, v.73 n.2, p.213-238, June 2007 [doi>10.1007/s11263-006-9794-4]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.7 Feature Measurement

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis

General Terms:
Algorithms, Measurement

Keywords:
bag of words, computational efficiency, feature extraction, image retrieval, random forest