Users can change their web search tactics: Design guidelines for categorized overviews

ABSTRACT

Categorized overviews of web search results are a promising way to support user exploration, understanding, and discovery. These search interfaces combine a metadata-based overview with the list of search results to enable a rich form of interaction. A study of 24 sophisticated users carrying out complex tasks suggests how searchers may adapt their search tactics when using categorized overviews. This mixed methods study evaluated categorized overviews of web search results organized into thematic, geographic, and government categories. Participants conducted four exploratory searches during a 2-hour session to generate ideas for newspaper articles about specified topics such as ''human smuggling.'' Results showed that subjects explored deeper while feeling more organized, and that the categorized overview helped subjects better assess their results, although no significant differences were detected in the quality of the article ideas. A qualitative analysis of searcher comments identified seven tactics that participants reported adopting when using categorized overviews. This paper concludes by proposing a set of guidelines for the design of exploratory search interfaces. An understanding of the impact of categorized overviews on search tactics will be useful to web search researchers, search interface designers, information architects and web developers.

REFERENCES

• Aula, 2004.: Aula, A. (2004). Enhancing the readability of search result summaries. Paper presented at the Proceedings Volume 2 of the Conference HCI 2004: Design for Life, Leeds, UK. Retrieved May 30, 2007, from http://www.cs.uta.fi/~aula/aula_summary.pdf.
• Bates, 1990.: Marcia J. Bates, Where should the person stop and the information search interface start?, Information Processing and Management: an International Journal, v.26 n.5, p.575-591, 1990 [doi>10.1016/0306-4573(90)90103-9]
• Bates, 1979.: Information search tactics. Journal of the American Society for Information Science. v30. 205-214.
• Borlund, 2003.: Borlund, P. (2003). The IIR evaluation model: A framework for evaluation of interactive information retrieval systems. Information Research, 8(3), paper no. 152.
• Drori and Alon, 2003.: Using documents classification for displaying search results list. Journal of Information Science. v29 i2. 97-106.
• Dumais et al., 2001.: Susan Dumais , Edward Cutrell , Hao Chen, Optimizing search by showing results in context, Proceedings of the SIGCHI conference on Human factors in computing systems, p.277-284, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365116]
• Ericsson and Simon, 1984.: Protocol Analysis: Verbal Reports as Data. MIT Press, Cambridge, MA.
• Fidel, 1985.: Moves in online searching. Online Review. v9 i1. 61-74.
• Garcia and Sicilia, 2003.: User interface tactics in ontology-based information seeking. Psychology Journal. v1 i3. 242-255.
• Garfield, 2005.: Garfield, E. (2005). The agony and the ecstasy-The history and meaning of the journal impact factor. Paper presented at the International Congress on Peer Review and Biomedical Publication, Chicago, IL. Retrieved May 30, 2007, from http://garfield.library.upenn.edu/papers/jifchicago2005.pdf.
• Golovchinsky, 1997.: Gene Golovchinsky, Queries? Links? Is there a difference?, Proceedings of the SIGCHI conference on Human factors in computing systems, p.407-414, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258820]
• Greene et al., 2000.: Stephan Greene , Gary Marchionini , Catherine Plaisant , Ben Shneiderman, Previews and overviews in digital libraries: designing surrogates to support visual information seeking, Journal of the American Society for Information Science, v.51 n.4, p.380-393, March 1, 2000 [doi>10.1002/(SICI)1097-4571(2000)51:4<380::AID-ASI7>3.3.CO;2-X]
• Guba and Lincoln, 1982.: Epistemological and methodological bases of naturalistic inquiry. Educational Communication and Technology. v30 i4. 233-252.
• Hearst, 1999a.: The use of categories and clusters for organizing retrieval results. In: Strzalkowski, T. (Ed.), Natural Language Information Retrieval, Kluwer Academic Publishers, Boston. pp. 333-373.
• Hearst, 1999b.: User interfaces and visualization. In: Baeza-Yates, R., Ribeiro-Neto, B. (Eds.), Modern Information Retrieval, Addison-Wesley, Reading, MA. pp. 257-323.
• Hearst et al., 2002.: Marti Hearst , Ame Elliott , Jennifer English , Rashmi Sinha , Kirsten Swearingen , Ka-Ping Yee, Finding the flow in web site search, Communications of the ACM, v.45 n.9, September 2002 [doi>10.1145/567498.567525]
• Jaccard, 1983.: Statistics for the Behavioral Sciences. Wadsworth Publishing Company, Belmont, CA.
• Jacko and Salvendy, 1996.: Hierarchical menu design: Breadth, depth, and task complexity. Perceptual and Motor Skills. v82. 1187-1201.
• Janecek and Pu, 2005.: An evaluation of semantic fisheye views for opportunistic search in an annotated image collection. Journal of Digital Libraries. v5 i1. 42-56.
• Jansen et al., 2000.: Bernard J. Jansen , Amanda Spink , Tefko Saracevic, Real life, real users, and real needs: a study and analysis of user queries on the web, Information Processing and Management: an International Journal, v.36 n.2, p.207-227, Jan.1.2000 [doi>10.1016/S0306-4573(99)00056-4]
• Kabel et al., 2004.: Suzanne Kabel , Robert de Hoog , Bob J. Wielinga , Anjo Anjewierden, The added value of task and ontology-based markup for information retrieval, Journal of the American Society for Information Science and Technology, v.55 n.4, p.348-362, February 15, 2004 [doi>10.1002/asi.10379]
• K$#228;ki, 2005.: Mika Käki, Findex: search result categories help users when document ranking fails, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1054991]
• Kiger, 1984.: John I. Kiger, The depth/breadth trade-off in the design of menu-driven user interfaces, International Journal of Man-Machine Studies, v.20 n.2, p.201-213, Feb. 1984 [doi>10.1016/S0020-7373(84)80018-8]
• Klein et al., 2002.: Visual information retrieval with the SuperTable+Scatterplot. In: Proceedings of the Sixth International Conference on Information Visualisation (IV '02), IEEE Computer Society, New York. pp. 70-75.
• Klein et al., 2003.: Peter Klein , Harald Reiterer , Frank Müller , Tobias Limbach, Metadata Visualisation with VisMeB, Proceedings of the Seventh International Conference on Information Visualization, p.600, July 16-18, 2003
• Koenemann and Belkin, 1996.: Jürgen Koenemann , Nicholas J. Belkin, A case for interaction: a study of interactive information retrieval behavior and effectiveness, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.205-212, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238487]
• Kules, 2006a.: Kules, B. (2006a). Methods for evaluating changes in search tactics induced by exploratory search systems. Paper presented at the ACM SIGIR 2006 Workshop on Evaluating Exploratory Search Systems. Retrieved May 30, 2007, from http://www.takomasoftware.com/techreports/KulesESSEval-20060630b.pdf.
• Kules, 2006b.: William Michael Kules, III , Ben Shneiderman, Supporting exploratory web search with meaningful and stable categorized overviews, University of Maryland at College Park, College Park, MD, 2006
• Kules et al., 2006.: Bill Kules , Jack Kustanowitz , Ben Shneiderman, Categorizing web search results into meaningful and stable categories using fast-feature techniques, Proceedings of the 6th ACM/IEEE-CS joint conference on Digital libraries, June 11-15, 2006, Chapel Hill, NC, USA [doi>10.1145/1141753.1141801]
• Kules and Shneiderman, submitted for publication.: Kules, B., & Shneiderman, B. (submitted for publication). Using meaningful and stable categories to support exploratory web search: Two formative studies.
• Kwasnik, 1999.: The role of classification in knowledge representation and discovery. Library Trends. v48 i1. 22-47.
• Louie et al., 2003.: Louie, A. J., Maddox, E. L., & Washington, W. (2003). Using faceted classification to provide structure for information architecture. Paper presented at the 62nd ASIS Annual Meeting, Washington, DC. Retrieved May 30, 2007, from http://depts.washington.edu/pettt/presentations/conf_2003/IASummit.pdf.
• Marchionini, 1995.: Gary Marchionini, Information seeking in electronic environments, Cambridge University Press, New York, NY, 1995
• Miller, 1981.: The depth/breadth trade-off in hierarchical computer menus. Proceedings of the Human Factors Society. 296-300.
• Nielsen et al., 2002.: Janni Nielsen , Torkil Clemmensen , Carsten Yssing, Getting access to what goes on in people's heads?: reflections on the think-aloud technique, Proceedings of the second Nordic conference on Human-computer interaction, October 19-23, 2002, Aarhus, Denmark [doi>10.1145/572020.572033]
• Pirolli and Card, 1995.: Peter Pirolli , Stuart Card, Information foraging in information access environments, Proceedings of the SIGCHI conference on Human factors in computing systems, p.51-58, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223911]
• Pirolli et al., 1996.: Peter Pirolli , Patricia Schank , Marti Hearst , Christine Diehl, Scatter/gather browsing communicates the topic structure of a very large text collection, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.213-220, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238489]
• Risden et al., 2000.: Kirsten Risden , Mary P. Czerwinski , Tamara Munzner , Daniel B. Cook, An initial examination of ease of use for 2D and 3D information visualizations of Web content, International Journal of Human-Computer Studies, v.53 n.5, p.695-714, Nov. 2000 [doi>10.1006/ijhc.2000.0413]
• Sebrechts et al., 1999.: Marc M. Sebrechts , John V. Cugini , Sharon J. Laskowski , Joanna Vasilakis , Michael S. Miller, Visualization of search results: a comparative evaluation of text, 2D, and 3D interfaces, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.3-10, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312634]
• Shneiderman et al., 1997.: Shneiderman, B., Byrd, D., & Croft, W.B. (1997). Clarifying search: A user-interface framework for text searches. D-Lib Magazine.
• Shneiderman et al., 1998.: Ben Shneiderman , Donald Byrd , W. Bruce Croft, Sorting out searching: a user-interface framework for text searches, Communications of the ACM, v.41 n.4, p.95-98, April 1998 [doi>10.1145/273035.273069]
• Shneiderman et al., 2000.: Ben Shneiderman , David Feldman , Anne Rose , Xavier Ferré Grau, Visualizing digital library search results with categorical and hierarchical axes, Proceedings of the fifth ACM conference on Digital libraries, p.57-66, June 02-07, 2000, San Antonio, Texas, United States [doi>10.1145/336597.336637]
• Shneiderman et al., 2006.: Creativity support tools: Report from a US National Science Foundation sponsored workshop. International Journal of Human-Computer Interaction. v20 i2. 61-77.
• Shneiderman and Plaisant, 2004.: Ben Shneiderman , Catherine Plaisant, Designing the User Interface: Strategies for Effective Human-Computer Interaction (4th Edition), Pearson Addison Wesley, 2004
• Shneiderman and Plaisant, 2006.: Ben Shneiderman , Catherine Plaisant, Strategies for evaluating information visualization tools: multi-dimensional in-depth long-term case studies, Proceedings of the 2006 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization, May 23-23, 2006, Venice, Italy [doi>10.1145/1168149.1168158]
• Soergel, 1974.: Construction and Maintenance of Indexing Languages and Thesauri. Wiley, New York.
• Soergel, 1999.: Dagobert Soergel, The rise of ontologies or the reinvention of classification, Journal of the American Society for Information Science, v.50 n.12, p.1119-1120, Oct. 1999 [doi>10.1002/(SICI)1097-4571(1999)50:12<1119::AID-ASI12>3.0.CO;2-I]
• Swan and Allen, 1998.: Russell C. Swan , James Allan, Aspect windows, 3-D visualizations, and indirect comparisons of information retrieval systems, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.173-181, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290987]
• Tanin et al., 2000.: Browsing large online data with Query Previews. In: Proceedings of the Symposium on New Paradigms in Information Visualization and Manipulation (NPIVM) 2000, ACM Press, Washington, DC.
• Teitelbaum and Granda, 1983.: Richard C. Teitelbaum , Richard E. Granda, The effects of positional constancy on searching menus for information, Proceedings of the SIGCHI conference on Human Factors in Computing Systems, p.150-153, December 12-15, 1983, Boston, Massachusetts, United States [doi>10.1145/800045.801600]
• Toms et al., 2003.: Elaine G. Toms , Luanne Freund , Richard Kopak , Joan C. Bartlett, The effect of task domain on search, Proceedings of the 2003 conference of the Centre for Advanced Studies on Collaborative research, p.303-312, October 06-09, 2003, Toronto, Ontario, Canada
• Tullis, 1988.: Screen design. In: Helander, M. (Ed.), Handbook of Human-Computer Interaction, Elsevier Science Publishers, Amsterdam, The Netherlands. pp. 377-411.
• Vakkari, 2000.: Vakkari, P. (2000). eCognition and changes of search terms and tactics during task performance: A longitudinal case study. Paper presented at the Proceedings of the RIAO 2000 Conference. Retrieved May 30, 2007, from http://www.info.uta.fi/vakkari/Vakkari_Tactics_RIAO2000.pdf.
• Vickery, 1960.: Faceted Classification: A Guide to Construction and Use of Special Schemes. Aslib, London.
• White et al., 2006.: White, R., Muresan, G., & Marchionini, G. (2006). Evaluating Exploratory Search Systems - SIGIR 2006 Workshop Call for Papers. Retrieved May 30, 2007, from http://www.umiacs.umd.edu/~ryen/eess.
• White et al., 2006.: Supporting exploratory search. Communications of the ACM. v49 i4. 36-39.
• Wildemuth, 2004.: Barbara M. Wildemuth, The effects of domain knowledge on search tactic formulation, Journal of the American Society for Information Science and Technology, v.55 n.3, p.246-258, February 2004 [doi>10.1002/asi.10367]
• Yee et al., 2003.: Ka-Ping Yee , Kirsten Swearingen , Kevin Li , Marti Hearst, Faceted metadata for image search and browsing, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642681]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software

          Nouns: World Wide Web (WWW)
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology; Interaction styles (e.g., commands, menus, forms, direct manipulation)

General Terms:
Design, Human Factors, Performance

Keywords:
Categorization, Categorized overviews, Categorized search results, Exploratory search, Human-computer interaction, Information seeking, Search result visualization, Search user interfaces, Web search