Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

The Web has become the largest information repository in the world; thus, effectively and efficiently searching the Web becomes a key challenge. Interactive Web search divides the search process into several rounds, and for each round the search engine interacts with the user for more knowledge of the user's information requirement. Previous research mainly uses the text information on Web pages, while little attention is paid to other modalities. This article shows that Web search performance can be significantly improved if imagery is considered in interactive Web search. Compared with text, imagery has its own advantage: the time for “reading” an image is as little as that for reading one or two words, while the information brought by an image is as much as that conveyed by a whole passage of text. In order to exploit the advantages of imagery, a novel interactive Web search framework is proposed, where image snippets are first extracted from Web pages and then provided, along with the text snippets, to the user for result presentation and relevance feedback, as well as being presented alone to the user for image suggestion. User studies show that it is more convenient for the user to identify the Web pages he or she expects and to reformulate the initial query. Further experiments demonstrate the promise of introducing multimodal techniques into the proposed interactive Web search framework.

REFERENCES

• 1: Peter Anick, Using terminological feedback for web search refinement: a log-based study, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860453]
• 2: Peter G. Anick , Suresh Tipirneni, The paraphrase search assistant: terminological feedback for iterative information seeking, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.153-159, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312670]
• 3: Ricardo A. Baeza-Yates , Berthier Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 4: Kobus Barnard , Matthew Johnson, Word sense disambiguation with pictures, Artificial Intelligence, v.167 n.1-2, p.13-30, September 2005 [doi>10.1016/j.artint.2005.04.009]
• 5: Deng Cai , Xiaofei He , Zhiwei Li , Wei-Ying Ma , Ji-Rong Wen, Hierarchical clustering of WWW image search results using visual, textual and link information, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027747]
• 6: Cai, D., Yu, S.-P., Wen, J.-R., and Ma, W.-Y. 2003. Vips: A vision-based page segmentation algorithm. Tech. Rep. No. MSR-TR-2003-79, Microsoft.
• 7: Soumen Chakrabarti, Mining the Web: Discovering Knowledge from HyperText Data, Science & Technology Books, 2002
• 8: Chapman, A. 1993. Making Sense: Teaching Critical Reading Across the Curriculum. The College Board, New York.
• 9: Coltheart, V. 1999. Fleeting Memories: Cognition of Brief Visual Stimuli. MIT Press, Cambridge, MA.
• 10: Simon Dennis , Peter Bruza , Robert McArthur, Web searching: a process-oriented experimental study of three interactive search paradigms, Journal of the American Society for Information Science and Technology, v.53 n.2, p.120-133, January 15, 2002 [doi>10.1002/asi.10015]
• 11: Xin Fan , Xing Xie , Zhiwei Li , Mingjing Li , Wei-Ying Ma, Photo-to-search: using multimodal queries to search the web from mobile devices, Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval, November 10-11, 2005, Hilton, Singapore [doi>10.1145/1101826.1101851]
• 12: Chien-Kang Huang , Lee-Feng Chien , Yen-Jen Oyang, Relevant term suggestion in interactive web search based on contextual information in query session logs, Journal of the American Society for Information Science and Technology, v.54 n.7, p.638-649, May 2003 [doi>10.1002/asi.10256]
• 13: Alejandro Jaimes , Mike Christel , Sébastien Gilles , Ramesh Sarukkai , Wei-Ying Ma, Multimedia information retrieval: what is it, and why isn't anyone using it?, Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval, p.3-8, November 10-11, 2005, Hilton, Singapore [doi>10.1145/1101826.1101829]
• 14: Jing, Y. and Croft, W. 1994. An association thesaurus for information retrieval. In Proceedings of the International Conference on Intelligent Multimedia Information Retrieval Systems, New York, 146--160.
• 15: Thorsten Joachims, Optimizing search engines using clickthrough data, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, July 23-26, 2002, Edmonton, Alberta, Canada [doi>10.1145/775047.775067]
• 16: Steve Jones , Mark S. Staveley, Phrasier: a system for interactive document retrieval using keyphrases, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.160-167, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312671]
• 17: Diane Kelly , Nicholas J. Belkin, Display time as implicit feedback: understanding task effects, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009057]
• 18: Diane Kelly , Jaime Teevan, Implicit feedback for inferring user preference: a bibliography, ACM SIGIR Forum, v.37 n.2, p.18-28, Fall 2003 [doi>10.1145/959258.959260]
• 19: Jürgen Koenemann , Nicholas J. Belkin, A case for interaction: a study of interactive information retrieval behavior and effectiveness, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.205-212, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238487]
• 20: Reiner Kraft , Jason Zien, Mining anchor text for query refinement, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988763]
• 21: Yonggang Qiu , Hans-Peter Frei, Concept based query expansion, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.160-169, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160713]
• 22: Ian Ruthven, Re-examining the potential effectiveness of interactive query expansion, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860475]
• 23: Xuehua Shen , Bin Tan , ChengXiang Zhai, Context-sensitive information retrieval using implicit feedback, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076045]
• 24: Silverstein, C., Henzinger, M., Marais, H., and Moricz, M. 1998. Analysis of a very large AltaVista query log. Tech. Rep. No.1998-014, Digital Systems Research Center.
• 25: Ruihua Song , Haifeng Liu , Ji-Rong Wen , Wei-Ying Ma, Learning block importance models for web pages, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988700]
• 26: Ryen W. White , Ian Ruthven , Joemon M. Jose, A study of factors affecting the utility of implicit relevance feedback, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076044]
• 27: Allison Woodruff , Andrew Faulring , Ruth Rosenholtz , Julie Morrsion , Peter Pirolli, Using thumbnails to search the Web, Proceedings of the SIGCHI conference on Human factors in computing systems, p.198-205, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365098]
• 28: Jinxi Xu , W. Bruce Croft, Improving the effectiveness of information retrieval with local context analysis, ACM Transactions on Information Systems (TOIS), v.18 n.1, p.79-112, Jan. 2000 [doi>10.1145/333135.333138]
• 29: Xue, X.-B., Zhou, Z.-H., and Zhang, Z. 2006. Improve Web search using image snippets. In Proceedings of the 21st National Conference on Artificial Intelligence, Boston, MA, 1431--1436.
• 30: Jun Yang , Qing Li , Yueting Zhuang, OCTOPUS: aggressive search of multi-modality data using multifaceted knowledge base, Proceedings of the 11th international conference on World Wide Web, May 07-11, 2002, Honolulu, Hawaii, USA [doi>10.1145/511446.511454]
• 31: Ruofei Zhang , Zhongfei (Mark) Zhang , Mingjing Li , Wei-Ying Ma , Hong-Jiang Zhang, A Probabilistic Semantic Model for Image Annotation and Multi-Modal Image Retrieva, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.846-851, October 17-20, 2005 [doi>10.1109/ICCV.2005.16]
• 32: Zhou, Z.-H. and Dai, H.-B. 2007. Exploiting image contents in Web search. In Proceedings of the 20th International Joint Conference on Artificial Intelligence, Hydrabad, India, 2928--2933.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Information filtering

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Relevance feedback; Query formulation

General Terms:
Algorithms, Design, Experimentation, Human Factors

Keywords:
Image snippet, image suggestion, interactive Web search, multimodality, relevance feedback, term suggestion