Comparing interactive information retrieval systems across sites

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Comparing interactive information retrieval systems across sites: the TREC-6 interactive track matrix experiment

Full text

Pdf (1.12 MB)

Source

Annual ACM Conference on Research and Development in Information Retrieval archive
Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval table of contents

Melbourne, Australia

Pages: 164 - 172

Year of Publication: 1998

ISBN:1-58113-015-5

Authors

Eric Lagergren	Statistical Engineering Division, National Institute of Standards and Technology (NIST), Gaithersburg, MD
Paul Over	Information Access and User Interfaces Division, National Institute of Standards and Technology (NIST), Gaithersburg, MD

Sponsors

University of Melbourne : University of Melbourne
SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 3, Downloads (12 Months): 57, Citation Count: 17

Additional Information:

references cited by index terms collaborative colleagues

Tools and Actions:	Request Permissions Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/290941.290986 What is a DOI?

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Allan, J., Callan, J., Croft, W. B., Ballesteros, L., Byrd, D., Swan, R., & Xu, J. (in press). INQUERY Does Battle with TREC-6. In E. M. Voorhees & D. K. Harman (Eds.), The Sixth Text REtrieval Conference (TRBC-6). Gaithersburg, MD, USA.
	2	Beaulieu, M. M., & Gatford, M. J. (in press). Interactive Okapi at TREC-6. In E. M. Voorhees & D. K. Harman (Eds.), The Sixth Text REtrieval Conference (TREC-6). Gaithersburg, MD, USA.
	3	Belkin, N. J., Perez Carballo, J., Lin, S., Park, S. Y., Rieh, S. Y., Savage, P., Sikora, C., & Xie, H. (in press). Rutgers' TREC-6 Interactive Track Experience. In E. M. Voorhees & D. K. Harman (Eds.), The Sixth Text REtrieval Conference (TREC-6). Gaithersburg, MD, USA.
	4	Fuller, M., Kaszkiel, C. L., Ng, P., Vines, P., Wilkinson, R., & Zobel, J. (in press). MDS TREC6 Report. In E. M. Voorhees & D. K. Harman (Eds.), The Sixth Text REtrieval Conference (TREC-6). Gaithersburg, MD, USA.
	5	Hersh, W., & Day, B. (in press). A Comparison of Boolean and Natural Language Searching for the TREC-6 Interactive Task. In E. M. Voorhees & D. K. Harman (Eds.), The Sixth Text REtrieval Conference (TREC-6). Gaithersburg, MD, USA.
	6	David Hull, Using statistical testing in the evaluation of retrieval experiments, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.329-338, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160758]
	7	Jones, B., & Kenward, M. G. (1989). The design and analysis of cross-over trials. London and New York: Chapman and Hall.
	8	Larson, R. R., & McDonough, J. (in press). Cheshire II at TREC-6: Interactive Probabilistic Retrieval. In E. M. Voorhees & D. K. Harman (Eds.), The Sixth Text REtrieval Conference (TREC-6). Gaithersburg, MD, USA.
	9	Littell, R., Milliken, G., Stroup, W., & Wolfinger, R. (1996). SAS System for Mixed Models. Cary, NC, USA: SAS Institute.
	10	McDonald, J., Ogden, W., & Foltz, P. (in press). Interactive information retrieval using term relationship networks. In E. M. Voorhees & D. K. Harman (Eds.), The Sixth Text REtrieval Conference (TREC-6). Gaithersburg, MD, USA.
	11	Neter, J., Wasserman, W., & Kutner, M. (1990). Applied Linear Statistical Models. Boston, MA, USA: Irwin.
	12	NIST. (1998a). TREC-6 Interactive Track Home Page {URL}. www-nlpir, hist. gov/-over/t6i.
	13	NIST. (1998b). The TRec Home Page {URL}. trec.nist.gov.
	14	NIST. (1998c). The ZPRISE 2.0 Home Page {URL}. www-nlpir.nist.gov/-over/zp2.
	15	Robertson, S. E. (1981). The methodology of information retrieval experiments. In K. Sparck Jones (Ed.), Information retrieval experiment (pp. 9-31). Butterworths.
	16	S. E. Robertson, On sample sizes for non-matched-pair IR experiments, Information Processing and Management: an International Journal, v.26 n.6, p.739-753, 1990 [doi>10.1016/0306-4573(90)90049-8]
	17	Schmidt-Wesche, B., Mack, R., & Cesar, C. L. (in press). IBM Search UI Prototype Evaluation at the Interactive Track of TREC-6. In E. M. Voorhees & D. K. Harman (Eds.), The Sixth Text REtrieval Conference (TREC-6). Gaithersburg, MD, USA.
	18	Sumner, R., Jr., Yang, K., Akers, R., & Shaw, W. M., Jr. (in press). Interactive Retrieval using IRIS: TREC-6 Experiments. In E. M. Voorhees & D. K. Harman (Eds.), The Sixth Text REtrieval Conference (TREC-6). Gaithersburg, MD, USA.
	19	Russell C. Swan , James Allan, Aspect windows, 3-D visualizations, and indirect comparisons of information retrieval systems, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.173-181, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290987]
	20	Jean Tague-Sutcliffe, The pragmatics of information retrieval experimentation, revisited, Information Processing and Management: an International Journal, v.28 n.4, p.467-490, July-Aug. 1992 [doi>10.1016/0306-4573(92)90005-K]
	21	Voorhees, E., & Harman, D. (1997). Overview of the Fifth Text REtrieval Conference (TREC-5). In E. M. Voorhees & D. K. Harman (Eds.), The Fifth Text REtrieval Conference (TREC-5) (pp. 1-28). Gaithersburg, MD, USA.
	22	Voorhees, E. M., & Harman, D. K. (Eds.). (in press). The Sixth Text REtrieval Conference (TREC-6). Gaithersburg, MD, USA.

CITED BY 17

	Kimberley Deninger , Caroline Cassigneul, The relevance factor, Journal of Computing Sciences in Colleges, v.16 n.4, p.337-340, 2001
	Anton Leuski, Evaluating document clustering for interactive information retrieval, Proceedings of the tenth international conference on Information and knowledge management, October 05-10, 2001, Atlanta, Georgia, USA
	Mingfang Wu , Michael Fuller , Ross Wilkinson, Searcher performance in question answering, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.375-381, September 2001, New Orleans, Louisiana, United States
	William Hersh , Andrew Turpin , Susan Price , Benjamin Chan , Dale Kramer , Lynetta Sacherek , Daniel Olson, Do batch and user evaluations give the same results?, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.17-24, July 24-28, 2000, Athens, Greece
	Russell C. Swan , James Allan, Aspect windows, 3-D visualizations, and indirect comparisons of information retrieval systems, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.173-181, August 24-28, 1998, Melbourne, Australia
	Mingfang Wu , Michael Fuller , Ross Wilkinson, The role of a judge in a user based retrieval experiment (poster session), Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.331-333, July 24-28, 2000, Athens, Greece
	Ellen M. Voorhees , Donna Harman, The text retrieval conferences (TRECS), Proceedings of a workshop on held at Baltimore, Maryland: October 13-15, 1998, October 13-15, 1998, Baltimore, Maryland
	Ying Sun , Paul B. Kantor, Cross-Evaluation: A new model for information system evaluation, Journal of the American Society for Information Science and Technology, v.57 n.5, p.614-628, March 2006
	Catherine L. Smith , Paul B. Kantor, User adaptation: good results from poor systems, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore
	Ryen W. White , Gary Marchionini , Gheorghe Muresan, Editorial: Evaluating exploratory search systems, Information Processing and Management: an International Journal, v.44 n.2, p.433-436, March, 2008
	I. E. Kuralenok , I. S. Nekrestyanov, Evaluation of Text Retrieval Systems, Programming and Computing Software, v.28 n.4, p.226-242, July-August 2002
	Douglas W. Oard , Julio Gonzalo , Mark Sanderson , Fernando López-Ostenero , Jianqiang Wang, Interactive Cross-Language Document Selection, Information Retrieval, v.7 n.1-2, p.205-228, January-April 2004
	Ryen W. White , Gheorghe Muresan , Gary Marchionini, Report on ACM SIGIR 2006 workshop on evaluating exploratory search systems, ACM SIGIR Forum, v.40 n.2, December 2006
	Michael Chen , Marti Hearsty , Jason Hong , James Lin, Cha-Cha: a system for organizing intranet search results, Proceedings of the 2nd conference on USENIX Symposium on Internet Technologies and Systems, p.5-5, October 11-14, 1999, Boulder, Colorado
	Ellen M. Voorhees, On test collections for adaptive information retrieval, Information Processing and Management: an International Journal, v.44 n.6, p.1879-1885, November, 2008
	Munehiro Nakazato , Ljubomir Manola , Thomas S. Huang, Group-based interface for content-based image retrieval, Proceedings of the Working Conference on Advanced Visual Interfaces, May 22-24, 2002, Trento, Italy
	Diane Kelly, Methods for Evaluating Interactive Information Retrieval Systems with Users, Foundations and Trends in Information Retrieval, v.3 n.1—2, p.1-224, January 2009