Report on ACM SIGIR 2006 workshop on evaluating exploratory search systems

ABSTRACT

Exploratory search systems (ESS) are designed to help users move beyond simply finding information toward using that information to support learning, analysis, and decision-making. The evaluation of the interactive systems designed specifically to help exploratory searchers is a challenging area, worthy of further discussion in the research community. In this article we report on a workshop conducted in conjunction with the ACM SIGIR Conference in Seattle, USA, in August 2006. The workshop involved researchers, academics, and practitioners discussing the formative and summative evaluation of ESS.

REFERENCES

• 1: Allan, J. (2003). HARD Track Overview in TREC 2003: High accuracy retrieval from documents. In Proceedings of the Text Retrieval Conference, pp 24--37.
• 2: Lisa Feldman Barrett , Daniel J. Barrett, An introduction to computerized experience sampling in psychology, Social Science Computer Review, v.19 n.2, p.175-185, June 12 2001 [doi>10.1177/089443930101900204]
• 3: Pia Borlund , Peter Ingwersen, Measures of relative relevance and ranked half-life: performance indicators for interactive IR, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.324-331, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291019]
• 4: Steve Cronen-Townsend , Yun Zhou , W. Bruce Croft, Predicting query performance, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564429]
• 5: Dumais, S. and Belkin, N. J. (2005). The TREC Interactive Track: Putting the user into search. In Voorhees, E. and Harman, D. (Eds.) TREC: Experiment and Evaluation in Information Retrieval, MIT Press.
• 6: Klahr, D. (2000). Exploring science: The cognition and development of discovery processes. Bradford Books, Cambridge, MA.
• 7: Eric Lagergren , Paul Over, Comparing interactive information retrieval systems across sites: the TREC-6 interactive track matrix experiment, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.164-172, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290986]
• 8: Landauer, T. K. (2002). On the computational basis of learning and cognition: Arguments from LSA. The psychology of learning and motivation, 41: 43--84.
• 9: Ryen W. White , Bill Kules , Steven M. Drucker , m.c. schraefel, Introduction, Communications of the ACM, v.49 n.4, April 2006 [doi>10.1145/1121949.1121978]