Object retrieval using configurations of salient regions

ABSTRACT

In this paper, we describe an approach for retrieving a ranked list of images containing the same object as the query. Images in the corpus are primitively represented by a set of salient region descriptors so that the retrieval approach can works despite of changes in scale, illumination, and partial occlusion. Based on the sparse frequency vector representation method, we have improved the bag-of-features method to integrate the spatial co-occurrence information into the image representation.

Instead of using the spatial configuration as a further verification stage following a prior filter stage, our method integrates the spatial information into the image representation and merges two stages in order to speed up the retrieval performance. To efficiently use the spatial configurations of salient regions, we propose a practical method to explore the content of image and flexibly cluster the salient regions into groups of neighbours. And the combination of information from the local co-occurrence of salient regions with the sparse frequency vector representation method is a major contribution of our work.

Experiment results on a ground-truth dataset and complexity comparison are provided to demonstrate the advantage of our way of using spatial information for object retrieval work.

REFERENCES

• 1: T. S. Bindulal and M. R. Kaimal, 2007, "Object Coding using a Shape Adaptive Wavelet Transform with Scalable WDR Method," presented at Image Processing, 2007. ICIP 2007. IEEE International Conference on.
• 2: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 3: J. M. Gauch, 1999, "Image segmentation and analysis via multiscale gradient watershed hierarchies," Image Processing, IEEE Transactions on, vol. 8, pp. 69--79.
• 4: Frederic Jurie , Bill Triggs, Creating Efficient Codebooks for Visual Recognition, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.604-610, October 17-20, 2005 [doi>10.1109/ICCV.2005.66]
• 5: David G. Lowe, Object Recognition from Local Scale-Invariant Features, Proceedings of the International Conference on Computer Vision-Volume 2, p.1150, September 20-25, 1999
• 6: Marcin Marszaek , Cordelia Schmid, Spatial Weighting for Bag-of-Features, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.2118-2125, June 17-22, 2006 [doi>10.1109/CVPR.2006.288]
• 7: J. Matas, O. Chum, M. Urban, and T. Pajdla, 2004, "Robust wide-baseline stereo from maximally stable extremal regions," Image and Vision Computing, vol. 22, pp. 761--767.
• 8: Krystian Mikolajczyk , Cordelia Schmid, Scale & Affine Invariant Interest Point Detectors, International Journal of Computer Vision, v.60 n.1, p.63-86, October 2004 [doi>10.1023/B:VISI.0000027790.02288.f2]
• 9: Krystian Mikolajczyk , Cordelia Schmid, A Performance Evaluation of Local Descriptors, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.10, p.1615-1630, October 2005 [doi>10.1109/TPAMI.2005.188]
• 10: K. Mikolajczyk , T. Tuytelaars , C. Schmid , A. Zisserman , J. Matas , F. Schaffalitzky , T. Kadir , L. Van Gool, A Comparison of Affine Region Detectors, International Journal of Computer Vision, v.65 n.1-2, p.43-72, November 2005 [doi>10.1007/s11263-005-3848-x]
• 11: David Nister , Henrik Stewenius, Scalable Recognition with a Vocabulary Tree, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.2161-2168, June 17-22, 2006 [doi>10.1109/CVPR.2006.264]
• 12: E. Nowak, F. Jurie, and B. Triggs, 2006, "Sampling Strategies for Bag-of-Features Image Classification," in Computer Vision - ECCV 2006, pp. 490--503.
• 13: B. Ommer and J. Buhmann, 2006, "Learning Compositional Categorization Models," in Computer Vision - ECCV 2006, pp. 316--329.
• 14: J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, 2007, "Object retrieval with large vocabularies and fast spatial matching," presented at Computer Vision and Pattern Recognition, 2007. CVPR '07. IEEE Conference on.
• 15: T. Quack, V. Ferrari, B. Leibe, and L. A. V. G. L. Van Gool, 2007, "Efficient Mining of Frequent and Distinctive Feature Configurations," presented at Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on.
• 16: Yossi Rubner , Carlo Tomasi , Leonidas J. Guibas, The Earth Mover's Distance as a Metric for Image Retrieval, International Journal of Computer Vision, v.40 n.2, p.99-121, Nov. 2000 [doi>10.1023/A:1026543900054]
• 17: S. Cabello, P. G., C. Knauer, G. Rote, 2005, "Matching Point Sets with respect to the Earth Mover's Distance" presented at Proceedings of the European Symposium on Algorithms.
• 18: Nikhil V. Shirahatti , Kobus Barnard, Evaluating Image Retrieval, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.955-961, June 20-26, 2005 [doi>10.1109/CVPR.2005.147]
• 19: Josef Sivic , Andrew Zisserman, Video Google: A Text Retrieval Approach to Object Matching in Videos, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.1470, October 13-16, 2003
• 20: J. Sivic and A. Zisserman, 2004, "Video data mining using configurations of viewpoint invariant regions," presented at Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on.
• 21: Timor Kadir , Michael Brady, Saliency, Scale and Image Description, International Journal of Computer Vision, v.45 n.2, p.83-105, November 2001 [doi>10.1023/A:1012460413855]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Clustering

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.3 Clustering

General Terms:
Algorithms, Experimentation, Performance

Keywords:
frequency vector, neighborhood clustering, object retrieval, salient region, spatial configuration