Findex: search result categories help users when document ranking fails

ABSTRACT

Long web search result lists can be hard to browse. We demonstrated experimentally, in a previous study, the usefulness of a categorization algorithm and filtering interface. However, the nature of interaction in real settings is not known from an experiment in laboratory settings. To address this problem, we provided our categorizing web search user interface to 16 users for a two month period. The interactions with the system were logged and the users' opinions were elicited with two questionnaires. The results show that categories are successfully used as part of users' search habits. They are helpful when the result ranking of the search engine fails. In those cases, the users are able to access results that locate far in the rank order list with the categories. Users can also formulate simpler queries and find needed results with the help of the categories. In addition, the categories are beneficial when more than one result is needed like in an exploratory or undirected search task.

REFERENCES

• 1: Peter Anick, Using terminological feedback for web search refinement: a log-based study, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860453]
• 2: Andrei Broder, A taxonomy of web search, ACM SIGIR Forum, v.36 n.2, Fall 2002 [doi>10.1145/792550.792552]
• 3: Peter Bruza , Robert McArthur , Simon Dennis, Interactive Internet search: keyword, directory and query reformulation mechanisms compared, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.280-287, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345598]
• 4: Hao Chen , Susan Dumais, Bringing order to the Web: automatically categorizing search results, Proceedings of the SIGCHI conference on Human factors in computing systems, p.145-152, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332418]
• 5: Douglass R. Cutting , David R. Karger , Jan O. Pedersen , John W. Tukey, Scatter/Gather: a cluster-based approach to browsing large document collections, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.318-329, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133214]
• 6: Dennis, S., McArthur, R., and Bruza, P.: Searching the World Wide Web Made Easy? The Cognitive Load Imposed by Query Refinement Mechanisms. Proceedings of the Third Australian Document Computing Conference, ADCS'98 (Sidney, Australia), University of Sidney, TR-518, 1998, 65--71.
• 7: Drori, O.: Display of Search Results in Google-based Yahoo! vs. LCC&K Interfaces: A Comparison Study. Proceedings of Informing Science 2003 Conference (Pori, Finland), Informing Science 2003, 309--320.
• 8: Susan Dumais , Hao Chen, Hierarchical classification of Web content, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.256-263, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345593]
• 9: Susan Dumais , Edward Cutrell , Hao Chen, Optimizing search by showing results in context, Proceedings of the SIGCHI conference on Human factors in computing systems, p.277-284, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365116]
• 10: S. T. Dumais , G. W. Furnas , T. K. Landauer , S. Deerwester , R. Harshman, Using latent semantic analysis to improve access to textual information, Proceedings of the SIGCHI conference on Human factors in computing systems, p.281-285, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57214]
• 11: Marti A. Hearst , Jan O. Pedersen, Reexamining the cluster hypothesis: scatter/gather on retrieval results, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.76-84, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243216]
• 12: Christoph Höscher , Gerhard Strube, Web search behavior of Internet experts and newbies, Proceedings of the 9th international World Wide Web conference on Computer networks : the international journal of computer and telecommunications netowrking, p.337-346, June 2000, Amsterdam, The Netherlands
• 13: Bernard J. Jansen , Udo Pooch, A review of web searching studies and a framework for future research, Journal of the American Society for Information Science and Technology, v.52 n.3, p.235-246, Feb. 1, 2001 [doi>10.1002/1097-4571(2000)9999:9999<::AID-ASI1607>3.3.CO;2-6]
• 14: Jansen, B., Spink, A., Bateman, J., and Saracevic, T.: Searchers, The Subjects They Search, and Sufficiency: A Study of a Large Sample of Excite Searchers. 1998 World Conference on the WWW and Internet (Orlando, USA), 1998.
• 15: Steve Jones , Gordon W. Paynter, Automatic extraction of document keyphrases for use in digital libraries: evaluation and applications, Journal of the American Society for Information Science and Technology, v.53 n.8, p.653-677, August 2002 [doi>10.1002/asi.10068]
• 16: Steve Jones , Matt Jones , Shaleen Deo_andA2, Using keyphrases as search result surrogates on small screen devices, Personal and Ubiquitous Computing, v.8 n.1, p.55-68, February 2004 [doi>10.1007/s00779-004-0258-y]
• 17: Käki, M. and Aula, A.: Findex: Improving Search Result Use through Automatic Filtering Categories. To appear in Interacting with Computers, Elsevier.
• 18: Peter Pirolli , Patricia Schank , Marti Hearst , Christine Diehl, Scatter/gather browsing communicates the topic structure of a very large text collection, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.213-220, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238489]
• 19: Pratt, W. and Fagan, L.: The Usefulness of Dynamically Categorizing Search Results. Journal of the American Medical Informatics Association. Vol. 7, No. 6, 2000, 605--617.
• 20: Daniel E. Rose , Danny Levinson, Understanding user goals in web search, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988675]
• 21: Amanda Spink , Bernard J. Jansen , Dietmar Wolfram , Tefko Saracevic, From E-Sex to E-Commerce: Web Search Changes, Computer, v.35 n.3, p.107-109, March 2002 [doi>10.1109/2.989940]
• 22: Jaime Teevan , Christine Alvarado , Mark S. Ackerman , David R. Karger, The perfect search engine is not enough: a study of orienteering behavior in directed search, Proceedings of the SIGCHI conference on Human factors in computing systems, p.415-422, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985745]
• 23: Yi-fang Brook Wu , Latha Shankar , Xin Chen, Finding more useful information faster from web search results, Proceedings of the twelfth international conference on Information and knowledge management, November 03-08, 2003, New Orleans, LA, USA [doi>10.1145/956863.956975]
• 24: Oren Zamir , Oren Etzioni, Grouper: a dynamic clustering interface to Web search results, Proceeding of the eighth international conference on World Wide Web, p.1361-1374, May 1999, Toronto, Canada
• 25: Oren Zamir , Oren Etzioni, Web document clustering: a feasibility demonstration, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.46-54, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290956]
• 26: Hua-Jun Zeng , Qi-Cai He , Zheng Chen , Wei-Ying Ma , Jinwen Ma, Learning to cluster web search results, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009030]
• 27: http://snowball.tartarus.org
• 28: http://www.altavista.com
• 29: http://www.google.com/apis
• 30: http://www.iboogie.com
• 31: http://www.teoma.com
• 32: http://www.vivisimo.com
• 33: http://www.wisenut.com

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Selection process; Clustering

Keywords:
categorization, clustering, information access, search user interfaces, web search