What do exploratory searchers look at in a faceted search interface?

ABSTRACT

This study examined how searchers interacted with a web-based, faceted library catalog when conducting exploratory searches. It applied eye tracking, stimulated recall interviews, and direct observation to investigate important aspects of gaze behavior in a faceted search interface: what components of the interface searchers looked at, for how long, and in what order. It yielded empirical data that will be useful for both practitioners (e.g., for improving search interface designs), and researchers (e.g., to inform models of search behavior). Results of the study show that participants spent about 50 seconds per task looking at (fixating on) the results, about 25 seconds looking at the facets, and only about 6 seconds looking at the query itself. These findings suggest that facets played an important role in the exploratory search process.

REFERENCES

• 1: Antelman, K., Lynema, E., & Pace, A. (2006). Toward a 21st Century Library Catalog. Information Technology and Libraries, 25(3), 128--139.
• 2: Marcia J. Bates, Where should the person stop and the information search interface start?, Information Processing and Management: an International Journal, v.26 n.5, p.575-591, 1990 [doi>10.1016/0306-4573(90)90103-9]
• 3: Borlund, P. (2000). Experimental components for the evaluation of interactive information retrieval systems. Journal of Documentation, 56(1), 71--90.
• 4: Borlund, P. (2003). The IIR evaluation model: A framework for evaluation of interactive information retrieval systems. Information Research, 8(3), paper no. 152. Retrieved September 19, 2008, from http://informationr.net/ir/8-3/paper152.html.
• 5: Katriina Byström , Kalervo Järvelin, Task complexity affects information seeking and use, Information Processing and Management: an International Journal, v.31 n.2, p.191-213, March/April 1995 [doi>10.1016/0306-4573(94)00041-Z]
• 6: Robert Capra , Gary Marchionini , Jung Sun Oh , Fred Stutzman , Yan Zhang, Effects of structure and interaction style on distinct search tasks, Proceedings of the 7th ACM/IEEE-CS joint conference on Digital libraries, June 18-23, 2007, Vancouver, BC, Canada [doi>10.1145/1255175.1255267]
• 7: Edward Cutrell , Zhiwei Guan, What are you looking for?: an eye-tracking study of information usage in web search, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240690]
• 8: Joseph H. Goldberg , Mark J. Stimson , Marion Lewenstein , Neil Scott , Anna M. Wichansky, Eye tracking in web search tasks: design implications, Proceedings of the 2002 symposium on Eye tracking research & applications, March 25-27, 2002, New Orleans, Louisiana [doi>10.1145/507072.507082]
• 9: Laura A. Granka , Thorsten Joachims , Geri Gay, Eye-tracking analysis of user behavior in WWW search, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009079]
• 10: Hearst, M. (2006). Design recommendations for hierarchical faceted search interfaces. Paper presented at the ACM SIGIR 2006 Workshop on Faceted Search. Retrieved September 27, 2007, from http://flamenco.berkeley.edu/papers/faceted--workshop06.pdf.
• 11: Marti Hearst , Ame Elliott , Jennifer English , Rashmi Sinha , Kirsten Swearingen , Ka-Ping Yee, Finding the flow in web site search, Communications of the ACM, v.45 n.9, September 2002 [doi>10.1145/567498.567525]
• 12: Henderson, L., & Tallman, J. I. (2006). Stimulated recall methodology. In Stimulated recall and mental models: Tools for teaching and learning computer information literacy (pp. 55--91). Lanham, MD: Scarecrow Press.
• 13: Mika Käki, Findex: search result categories help users when document ranking fails, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1054991]
• 14: Jacob, R. & and Karn, K. (2003) Eye Tracking in Human-Computer Interaction and Usability Research: Ready to Deliver the Promises (Section Commentary), in The Mind's Eye: Cognitive and Applied Aspects of Eye Movement Research, ed. by J. Hyona, R. Radach, and H. Deubel, pp. 573--605, Amsterdam, Elsevier Science.
• 15: Kuhlthau, C. C. (1991). Inside the search process: Information seeking from the user's perspective. Journal of the American Society for Information Science, 42(5), 361--371.
• 16: Kules, B., & Capra, R. (2008) Creating exploratory tasks for a faceted search interface. Paper presented at the Second Workshop on Human-Computer Interaction (HCIR 2008).
• 17: Bill Kules , Ben Shneiderman, Users can change their web search tactics: Design guidelines for categorized overviews, Information Processing and Management: an International Journal, v.44 n.2, p.463-484, March, 2008 [doi>10.1016/j.ipm.2007.07.014]
• 18: Library of Congress (2008). On the Record: Report of the Working Group on the Future of Bibliographic Control. Retrieved September 13, 2008, from http://www.loc.gov/bibliographic-future/news/lcwg-ontherecord-jan08-final.pdf.
• 19: Lori Lorigo , Bing Pan , Helene Hembrooke , Thorsten Joachims , Laura Granka , Geri Gay, The influence of task and gender on search and evaluation behavior using Google, Information Processing and Management: an International Journal, v.42 n.4, p.1123-1131, July 2006 [doi>10.1016/j.ipm.2005.10.001]
• 20: Lown, C. (2008). A Transaction Log Analysis of NCSU's Faceted Navigation OPAC. Master's paper, University of North Carolina at Chapel Hill, 2008.
• 21: Gary Marchionini, Information seeking in electronic environments, Cambridge University Press, New York, NY, 1997
• 22: Gary Marchionini, Exploratory search: from finding to understanding, Communications of the ACM, v.49 n.4, April 2006 [doi>10.1145/1121949.1121979]
• 23: Pan, B., Hembrooke, H., Joachims, T., Lorigo, L., Gay, G., & Granka, L. (2007). In Google we trust: Users' decisions on rank, position and relevance. Journal of Computer-Mediated Communication, 12(3), article 3.
• 24: Tito Sierra , Joseph Ryan , Jason Casden, The ncsu catalog research testbed: a tool for evaluating faceted library catalog interfaces, Proceedings of the 8th ACM/IEEE-CS joint conference on Digital libraries, June 16-20, 2008, Pittsburgh PA, PA, USA [doi>10.1145/1378889.1379012]
• 25: Tunkelang, D. (2006). Dynamic Category Sets: an approach for faceted search. Paper presented at the ACM SIGIR 2006 Workshop on Faceted Search. Retrieved September 27, 2007, http://www.cs.cmu.edu/~quixote/DynamicCategorySets.pdf.
• 26: Pertti Vakkari, Task complexity, problem structure and information actions: integrating studies on information seeking and retrieval, Information Processing and Management: an International Journal, v.35 n.6, p.819-837, 35 [doi>10.1016/S0306-4573(99)00028-X]
• 27: Vakkari, P. (2001). A theory of the task--based information retrieval process: A summary and generalisation of a longitudinal study. Journal of Documentation, 57(1), 44--60.
• 28: Max L. Wilson , m.c. schraefel, A longitudinal study of exploratory and keyword search, Proceedings of the 8th ACM/IEEE-CS joint conference on Digital libraries, June 16-20, 2008, Pittsburgh PA, PA, USA [doi>10.1145/1378889.1378899]
• 29: Ryen W. White , Bill Kules , Steven M. Drucker , m.c. schraefel, Introduction, Communications of the ACM, v.49 n.4, April 2006 [doi>10.1145/1121949.1121978]
• 30: Ryen W. White , Steven M. Drucker , Gary Marchionini , Marti Hearst , m. c. schraefel, Exploratory search and HCI: designing and evaluating interfaces to support exploratory search interaction, CHI '07 extended abstracts on Human factors in computing systems, April 28-May 03, 2007, San Jose, CA, USA [doi>10.1145/1240866.1241100]
• 31: Ka-Ping Yee , Kirsten Swearingen , Kevin Li , Marti Hearst, Faceted metadata for image search and browsing, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642681]
• 32: Zhang, Y., & Salaba, A. (2007). User research and testing of FRBR prototype systems. Paper presented at the Annual Meeting of the American Society for Information Science and Technology (ASIST).

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Experimentation, Human Factors, Measurement

Keywords:
OPAC, exploratory search, eye tracking, faceted search, online public access catalogs, task design, user studies