Visual query suggestion

ABSTRACT

Query suggestion is an effective approach to improve the usability of image search. Most existing search engines are able to automatically suggest a list of textual query terms based on users' current query input, which can be called Textual Query Suggestion. This paper proposes a new query suggestion scheme named Visual Query Suggestion (VQS) which is dedicated to image search. It provides a more effective query interface to formulate an intent-specific query by joint text and image suggestions. We show that VQS is able to more precisely and more quickly help users specify and deliver their search intents. When a user submits a text query, VQS first provides a list of suggestions, each containing a keyword and a collection of representative images in a dropdown menu. If the user selects one of the suggestions, the corresponding keyword will be added to complement the initial text query as the new text query, while the image collection will be formulated as the visual query. VQS then performs image search based on the new text query using text search techniques, as well as content-based visual retrieval to refine the search results by using the corresponding images as query examples. We compare VQS with three popular image search engines, and show that VQS outperforms these engines in terms of both the quality of query suggestion and search performance.

REFERENCES

• 1: Ask image search: http://www.ask.com/?tool=img.
• 2: Flickr: http://www.flickr.com/.
• 3: Google image search: http://images.google.com/.
• 4: Microsoft bing image search: http://www.bing.com/?scope=images.
• 5: Yahoo! image search: http://images.search.yahoo.com/.
• 6: D. Beeferman and A. Berger. Agglomerative clustering of a search engine query log. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 407--416, Boston, US, 2000.
• 7: S. Boyd and L. Vandenberghe. Convex optimization. Cambridge University Press, 2004.
• 8: C. Carpineto, R. de Mori, G. Romano, and B. Bigi. An information-theoretic approach to automatic query expansion. ACM transactions on Information Systems, 19(1):1--27,New York, US, 2001.
• 9: R. Datta, D. Joshi, J. Li, and J. Z. Wang. Image retrieval: ideas, influences, and trends of the new age. ACM Computing Surveys, 40(2):1--60, New York,US, 2008.
• 10: B. Frey and D. Dueck. Clustering by passing messages between data points. Science, 319(5814):726, 2007.
• 11: R. J. Gerrig and P. G. Zimbardo. Psychology and Life (16 Edition). Allyn & Bacon, 2001.
• 12: D. Heesch and S. Rüger. Nnk networks for content-based image retrieval. In Proceedings of European Conference on InformationRetrieval, pages 253--266, Sunderland, UK, 2004.
• 13: W. Hsu, L. Kennedy, and S.-F. Chang. Video search reranking via information bottleneck principle. In Proceedings of ACM SIGMM International Conference on Multimedia, pages 35--44, Santa Barbara, USA, 2006.
• 14: Y. Jia, J. Wang, C. Zhang, and X.-S. Hua. Finding image exemplars using fast sparse affinity propagation. In Proceedings of ACM SIGMM International Conference on Multimedia, pages 639--642, Vancouver, Canada, 2008.
• 15: S. Kullback and R. A. Leibler. On information and sufficiency. Annals of Mathematical Statistics, 22(1):79--86, 1951.
• 16: A. M. Lam-Adesina and G. J. F. Jones. Applying summarization techniques for term selection in relevance feedback. In Proceedings of ACM SIGIR Internation Conference on Information Retrieval, pages 1--9, New Orleans, US,2001.
• 17: M. S. Lew, N. Sebe, C. Djeraba, and R. Jain. Content-based multimedia information retrieval: state of the art and challenges. ACM Transactions on Multimedia Computing, Communications and Applications, 2(1):1--19, New York, USA, 2006.
• 18: Y. Liu, T. Mei, and X.-S. Hua. CrowdReranking: Exploring multiple search engines for visual search reranking. In Proceedings of ACM SIGIR Internation Conference on Information Retrieval, Boston, USA, 2009.
• 19: T. Mei, X.-S. Hua, W. Lai, L. Yang, Z.-J. Zha, Y. Liu, Z. Gu, G.-J. Qi, M. Wang, J. Tang, X. Yuan, Z. Lu, and J. Liu. MSRA-USTC-SJTU at TRECVID 2007: High-level feature extraction and search. In TREC Video Retrieval Evaluation Online Proceedings, 2007.
• 20: B.-Y. Ricardo, H. Carlos, and M. Marcelo. Query recommendation using query logs in search engines. In Proceedings of International Conference on Extending Database Technology, pages 588--596, Heraklion, Greece, 2004.
• 21: B. Sigurbjornsson and R. van Zwol. Flickr tag recommendation based on collective knowledge. In Proceedings of International conference on World Wide Web, pages 327--336, Beijing, China, 2008.
• 22: A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(12):1349--1380, 2000.
• 23: K. Weinberger, M. Slaney, and R. V. Zwol. Resolving tag ambiguity. In Proceedings of ACM SIGMM International Conference on Multimedia, pages 111--120, Vancouver, Canada, 2008.
• 24: J.-R. Wen, J.-Y. Nie, and H.-J. Zhang. Clustering user queries of a search engine. In Proceedings of International Conference on World Wide Web, pages 162--168, Hong Kong, China, 2003.
• 25: J. Xu and W. B. Croft. Query expansion using local and global document analysis. In Proceedings of ACM SIGIR Internation Conference on Information Retrieval, pages 4--11, Zurich, Switzerland, 1996.
• 26: S. Yu, D. Cai, J.-R. Wen, and W.-Y. Ma. Improving pseudo-relevance feedback in web information retrieval using web page segmentation. In Proceedings of International Conference on World Wide Web, pages 11--18, Budapest, Hungary, 2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

General Terms:
Algorithms, Experimentation, Human Factors

Keywords:
image search., query suggestion