Characterizing the influence of domain expertise on web search behavior

ABSTRACT

Domain experts search differently than people with little or no domain knowledge. Previous research suggests that domain experts employ different search strategies and are more successful in finding what they are looking for than non-experts. In this paper we present a large-scale, longitudinal, log-based analysis of the effect of domain expertise on web search behavior in four different domains (medicine, finance, law, and computer science). We characterize the nature of the queries, search sessions, web sites visited, and search success for users identified as experts and non-experts within these domains. Large-scale analysis of real-world interactions allows us to understand how expertise relates to vocabulary, resource use, and search task under more realistic search conditions than has been possible in previous small-scale studies. Building upon our analysis we develop a model to predict expertise based on search behavior, and describe how knowledge about domain expertise can be used to present better results and query suggestions to users and to help non-experts gain expertise.

REFERENCES

• 1: Eugene Agichtein , Eric Brill , Susan Dumais , Robert Ragno, Learning user interaction models for predicting web search result preferences, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148175]
• 2: Allen, B. L. (1991). Topic knowledge and online catalog search formulation. Lib. Quart., 61(2), 188--213.
• 3: Bhavnani, S. K. (2001). Important cognitive components of domain-specific search knowledge. Proc. TREC, 571--578.
• 4: Suresh K. Bhavnani, Domain-specific search strategies for the effective retrieval of healthcare and shopping information, CHI '02 extended abstracts on Human factors in computing systems, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/506443.506508]
• 5: Suresh K. Bhavnani , Christopher K. Bichakjian , Timothy M. Johnson , Roderick J. Little , Frederick A. Peck , Jennifer L. Schwartz , Victor J. Strecher, Strategy hubs: Domain portals to help find comprehensive information, Journal of the American Society for Information Science and Technology, v.57 n.1, p.4-24, January 2006 [doi>10.1002/asi.v57:1]
• 6: Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Earlbaum.
• 7: Michael Collins, Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithms, Proceedings of the ACL-02 conference on Empirical methods in natural language processing, p.1-8, July 06, 2002 [doi>10.3115/1118693.1118694]
• 8: Downey, D., Dumais, S. & Horvitz, E. (2007). Models of searching and browsing: Languages, studies and application. Proc. IJCAI, 2740--2747.
• 9: Geoffrey B. Duggan , Stephen J. Payne, Knowledge in the head and on the web: using topic expertise to aid search, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357062]
• 10: Luanne Freund , Elaine G. Toms, Enterprise search behaviour of software engineers, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148297]
• 11: Helene A. Hembrooke , Laura A. Granka , Geraldine K. Gay , Elizabeth D. Liddy, The effects of expertise and feedback on search term selection and subsequent learning: Research Articles, Journal of the American Society for Information Science and Technology, v.56 n.8, p.861-871, June 2005 [doi>10.1002/asi.v56:8]
• 12: Christoph Hölscher , Gerhard Strube, Web search behavior of Internet experts and newbies, Computer Networks: The International Journal of Computer and Telecommunications Networking, v.33 n.1-6, p.337-346, June 2000
• 13: Hsieh-Yee, I. (1993). Effects of search experience and subject knowledge on the search tactics of novice and experienced searchers. JASIST, 44(3), 161--174.
• 14: Diane Kelly , Colleen Cool, The effects of topic familiarity on information search behavior, Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries, July 14-18, 2002, Portland, Oregon, USA [doi>10.1145/544220.544232]
• 15: Marchionini, G., Lin, X. & Dwiggins, S. (1990). Effects of search and subject expertise on information seeking in a hypertext environment. Proc. ASIS, 129--142.
• 16: Xuehua Shen , Susan Dumais , Eric Horvitz, Analysis of topic dynamics in web search, Special interest tracks and posters of the 14th international conference on World Wide Web, May 10-14, 2005, Chiba, Japan [doi>10.1145/1062745.1062889]
• 17: Amanda Spink , Bernard J. Jansen, Web Search: Public Searching of the Web (Information Science and Knowledge Management), Springer-Verlag New York, Inc., Secaucus, NJ, 2005
• 18: Jaime Teevan , Susan T. Dumais , Daniel J. Liebling, To personalize or not to personalize: modeling queries with variation in user intent, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore [doi>10.1145/1390334.1390364]
• 19: Pertti Vakkari , Mikko Pennanen , Sami Serola, Changes of search terms and tactics while writing a research proposal A longitudinal case study, Information Processing and Management: an International Journal, v.39 n.3, p.445-463, May 2003 [doi>10.1016/S0306-4573(02)00031-6]
• 20: Ryen W. White , Steven M. Drucker, Investigating behavioral variability in web search, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242576]
• 21: Ryen W. White , Susan Dumais , Jaime Teevan, How medical expertise influences web search interaction, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore [doi>10.1145/1390334.1390506]
• 22: Ryen W. White , Dan Morris, Investigating the querying and browsing behavior of advanced search engine users, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277787]
• 23: Barbara M. Wildemuth, The effects of domain knowledge on search tactic formulation, Journal of the American Society for Information Science and Technology, v.55 n.3, p.246-258, February 2004 [doi>10.1002/asi.10367]
• 24: Zhang, X., Anghelescu, H. G. B. & Yuan, X. (2005). Domain knowledge, search behavior, and search effectiveness of engineering and science students. Inf. Res., 10(2), 217.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

General Terms:
Experimentation, Human Factors

Keywords:
domain expertise, web search