CrowdReranking: exploring multiple search engines for visual search reranking

ABSTRACT

Most existing approaches to visual search reranking predominantly focus on mining information within the initial search results. However, the initial ranked list cannot provide enough cues for reranking by itself due to the typically unsatisfying visual search performance. This paper presents a new method for visual search reranking called CrowdReranking, which is characterized by mining relevant visual patterns from image search results of multiple search engines which are available on the Internet. Observing that different search engines might have different data sources for indexing and methods for ranking, it is reasonable to assume that there exist different search results yet certain common visual patterns relevant to a given query among those results. We first construct a set of visual words based on the local image patches collected from multiple image search engines. We then explicitly detect two kinds of visual patterns, i.e., salient and concurrent patterns, among the visual words. Theoretically, we formalize reranking as an optimization problem on the basis of the mined visual patterns and propose a close-form solution. Empirically, we conduct extensive experiments on several real-world search engines and one benchmark dataset, and show that the proposed CrowdReranking is superior to the state-of-the-art works.

REFERENCES

• 1: Sergey Brin , Lawrence Page, The anatomy of a large-scale hypertextual Web search engine, Computer Networks and ISDN Systems, v.30 n.1-7, p.107-117, April 1, 1998
• 2: L.B. Cremeant and R.M. Murra. Stability analysis of interconnected nonlinear systems under matrix feedback. In Proceedings of IEEE Conference on Decision and Control, 2003.
• 3: R. Fergus , L. Fei-Fei , P. Perona , A. Zisserman, Learning Object Categories from Google"s Image Search, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1816-1823, October 17-20, 2005 [doi>10.1109/ICCV.2005.142]
• 4: Flickr. http://www.flickr.com/.
• 5: Google image and video search. http://image.google.com/, http://video.google.com/.
• 6: Winston H. Hsu , Lyndon S. Kennedy , Shih-Fu Chang, Video search reranking through random walk over document-level context graph, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291446]
• 7: Winston H. Hsu , Lyndon S. Kennedy , Shih-Fu Chang, Video search reranking via information bottleneck principle, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180654]
• 8: Kalervo Järvelin , Jaana Kekäläinen, IR evaluation methods for retrieving highly relevant documents, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.41-48, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345545]
• 9: Yushi Jing , Shumeet Baluja, Pagerank for product image search, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367540]
• 10: Lyndon S. Kennedy , Shih-Fu Chang, A reranking approach for context-based concept fusion in video indexing and retrieval, Proceedings of the 6th ACM international conference on Image and video retrieval, p.333-340, July 09-11, 2007, Amsterdam, The Netherlands [doi>10.1145/1282280.1282331]
• 11: Michael S. Lew , Nicu Sebe , Chabane Djeraba , Ramesh Jain, Content-based multimedia information retrieval: State of the art and challenges, ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), v.2 n.1, p.1-19, February 2006 [doi>10.1145/1126004.1126005]
• 12: Y. Liu, T. Mei, X.-S. Hua, J. Tang, X. Wu, and S. Li. Learning to video search rerank via pseudo preference feedback. In Proceedings of IEEE International Conference on Multimedia&Expo, 2008.
• 13: Yuan Liu , Tao Mei , Xiuqing Wu , Xian-Sheng Hua, Optimizing video search reranking via minimum incremental information loss, Proceeding of the 1st ACM international conference on Multimedia information retrieval, October 30-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1460096.1460139]
• 14: Michel Vidal-Naquet , Shimon Ullman, Object Recognition with Informative Features and Linear Classification, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.281, October 13-16, 2003
• 15: T. Mei, X.-S. Hua, W. Lai, L. Yang, and et al. MSRA-USTC-SJTU at TRECVID 2007: High-level feature extraction and search. In TREC Video Retrieval Evaluation Online Proceedings, 2007.
• 16: Microsoft Live image and video search. http://www.live.com/?scope=images/, http://www.live.com/?scope=videos/.
• 17: Apostol (Paul) Natsev , Alexander Haubold , Jelena Tešić , Lexing Xie , Rong Yan, Semantic concept-based query expansion and re-ranking for multimedia retrieval, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291448]
• 18: S.-E. Robertson and K.-S. Jones. Simple, proven approaches to text retrieval. Cambridge University Computer Laboratory Technical Report TR356, 1997.
• 19: Josef Sivic , Andrew Zisserman, Video Google: A Text Retrieval Approach to Object Matching in Videos, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.1470, October 13-16, 2003
• 20: Xinmei Tian , Linjun Yang , Jingdong Wang , Yichen Yang , Xiuqing Wu , Xian-Sheng Hua, Bayesian video search reranking, Proceeding of the 16th ACM international conference on Multimedia, October 26-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1459359.1459378]
• 21: TRECVID. http://www-nlpir.nist.gov/projects/trecvid/.
• 22: Lior Wolf , Stanley Bileschi, A Critical View of Context, International Journal of Computer Vision, v.69 n.2, p.251-261, August 2006 [doi>10.1007/s11263-006-7538-0]
• 23: Yahoo image and video search. http://image.yahoo.com/, http://video.yahoo.com/.
• 24: R. Yan, A. Hauptmann, and R. Jin. Multimedia search with pseudo-relevance feedback. In Proceedings of ACM International Conference on Image and Video Retrieval, 2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms, Experimentation, Performance

Keywords:
data mining, search reranking, visual search