Query aware visual similarity propagation for image search reranking

ABSTRACT

Image search reranking is an effective approach to refining the text-based image search result. In the reranking process, the estimation of visual similarity is critical to the performance. However, the existing measures, based on global or local features, cannot be adapted to different queries. In this paper, we propose to estimate a query aware image similarity by incorporating the global visual similarity, local visual similarity and visual word co-occurrence into an iterative propagation framework. After the propagation, a query aware image similarity combining the advantages of both global and local similarities is achieved and applied to image search reranking. The experiments on a real-world Web image dataset demonstrate that the proposed query aware similarity outperforms the global, local similarity and their linear combination, for image search reranking.

REFERENCES

• 1: N. Ben-Haim, B. Babenko, and S. Belongie. Improving web-based image search via content based clustering. In SLAM, pages 106--111, 2006.
• 2: T. Chang and C.-C. Kuo. Texture analysis and classification with tree-structured wavelet transform. In IEEE Transactions on Image Processing, volume 2, pages 429--441, 1993.
• 3: R. L. Cilibrasi and P. M. B. Vitanyi. The google similarity distance. In IEEE Transactions on Knowledge and Data Engineering, volume 19, pages 370--383, 2007.
• 4: W. H. Hsu, L. S. Kennedy, and S.-F. Chang. Video search reranking via information bottleneck principle. In ACM Multimedia, pages 35--44, 2006.
• 5: W. H. Hsu, L. S. Kennedy, and S.-F. Chang. Video search reranking through random walk over document-level context graph. In ACM Multimedia, pages 971--980, 2007.
• 6: K. Jarvelin and J. Kekalainen. IR evaluation methods for retrieving highly relevant documents, 2000. In ACM SIGIR, pages 41--48, 2000.
• 7: Y. Jing and S. Baluja. Visualrank: Applying pagerank to large-scale image search. In IEEE Transactions on attern Analysis and Machine Intelligence, volume 30, pages 1877--1890, 2008.
• 8: D. G. Lowe. Object recognition from local scale-invariant features. In ICCV, volume 2, pages 1150--1157, 1999.
• 9: W.-Y. Ma and H. J. Zhang. Benchmarking of image features for content-based retrieval. In Conference Record of the Thirty-Second Asilomar Conference on Signals, Systems & Computers, volume 1, pages 253--257, Nov 1998.
• 10: D. K. Park, Y. S. Jeon, and C. S. Won. Efficient use of local edge histogram descriptor. In ACM Multimedia, pages 51--54, 2000.
• 11: K. Sparck Jones. A statistical interpretation of term specificity and its application in retrieval. Document retrieval systems, pages 132--142, 1988.
• 12: X. Tian, L. Yang, J. Wang, Y. Yang, X. Wu, and X.-S. Hua. Bayesian video search reranking. In ACM Multimedia, pages 131--140, 2008.
• 13: X.-J. Wang, W.-Y. Ma, G.-R. Xue, and X. Li. Multi-model similarity propagation and its application for web image retrieval. In ACM Multimedia, pages 944--951, 2004.
• 14: R. Yan, E. Hauptmann, and R. Jin. Multimedia search with pseudo-relevance feedback. In CIVR, pages 238--247, 2003.
• 15: J. Yang, Y.-G. Jiang, A. G. Hauptmann, and C.-W. Ngo. Evaluating bag-of-visual-words representations in scene classification. In MIR, pages 197--206, 2007.
• 16: H. Zitouni, S. Sevil, D. Ozkan, and P. Duygulu. Re-ranking of web image search results using a graph algorithm. In ICPR, pages 1--4, 2008.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms, Experimentation, Performance

Keywords:
image search, similarity, visual reranking