Measures of relative relevance and ranked half-life

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Measures of relative relevance and ranked half-life: performance indicators for interactive IR

Full text

Pdf (982 KB)

Source

Annual ACM Conference on Research and Development in Information Retrieval archive
Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval table of contents

Melbourne, Australia

Pages: 324 - 331

Year of Publication: 1998

ISBN:1-58113-015-5

Authors

Pia Borlund	Dept. of IR Theory, Royal School of Library and Information Science
Peter Ingwersen	Dept. of IR Theory, Royal School of Library and Information Science

Sponsors

University of Melbourne : University of Melbourne
SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 9, Downloads (12 Months): 60, Citation Count: 25

Additional Information:

references cited by index terms collaborative colleagues

Tools and Actions:	Request Permissions Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/290941.291019 What is a DOI?

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Belkin, N.J. Cognitive models and information transfer. Social Science Information Studies, (4), 1984, 11 I- ! 29.
	2	Belkin, N.J., Oddy, R. & Brooks, H. ASK for information retrieval: Part I. Journal of Documentation, (38), 1982, 61-71.
	3	Borlund, P. and Ingwersen, P. The development of a method for the evaluation of interactive information retrieval systems. Journal of Documentation, (53)3, 1997, 225-250.
	4	Giorgio Brajnik , Stefano Mizzaro , Carlo Tasso, Evaluating user interfaces to information retrieval systems: a case study on user support, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.128-136, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243249]
	5	Cleverdon, C.W & Keen, E.M. Factors determining the performance of indexing systems. Vol. 1: Design. Vol. 2: Results. Cranfield, UK: Aslib Cranfield Research Project, ! 966.
	6	Cooper, W.S. A Definition of Relevance for Information Retrieval. Information Storage and Retrieval (7)1, 1971, 19-37.
	7	Cooper, W.S. On Selecting a Measure of Retrieval effectiveness, Part 1. Journal of the American Society for Information Science, (24)2, 1973, 87-100.
	8	Donna Harman, Overview of the first TREC conference, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.36-47, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160692]
	9	Donna Harman, Overview of the second text retrieval conference (TREC-2), Information Processing and Management: an International Journal, v.31 n.3, p.271-289, May-June 1995
	10	Hammn, D. (Ed.). Overview of the Third Text REtrieval Conference (TREC-3). National Institute of Standards and Technology Special Publication 500- 225, Gaithersburg, Md. 20899.
	11	Harman, D. (Ed.). Overview of the Fourth Text REtrieval Conference (TREC-4). National Institute of Standards and Technology, Gaithersburg, Md. 20899.
	12	Donna K. Harman, The TREC conferences, Readings in information retrieval, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1997
	13	Harter, S.P. Psychological Relevance and information Science. Journal of American Society for Information Science, (43), 1992, 602-615.
	14	Hillman, D.J. The Notion of Relevance (I). American Documentation, (15) 1, 1964, 26-34.
	15	Ingwersen, P. Cognitive perspectives of information retrieval interaction: Elements of a cognitive IR theory. Journal of Documentation, (52)1, 1996,3-50.
	16	E. Michael Keen, Presenting results of experimental retrieval comparisons, Information Processing and Management: an International Journal, v.28 n.4, p.491-502, July-Aug. 1992 [doi>10.1016/0306-4573(92)90006-L]
	17	Ingwersen, P. Search procedures in the library analysed from the cognitive point of view. Journal of Documentation, (38), 1982, 165-19 I.
	18	S. E. Robertson , M. M. Hancock-Beaulieu, On the evaluation of IR systems, Information Processing and Management: an International Journal, v.28 n.4, p.457-466, July-Aug. 1992 [doi>10.1016/0306-4573(92)90004-J]
	19	Tefko Saracevic, Evaluation of evaluation in information retrieval, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.138-146, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215351]
	20	Saracevic, T. Relevance: A review of and a framework for the thinking on the notion in Information Science. Journal of American Society for information Science, (26), 1975, 321-343.
	21	Saracevic, T. Relevance reconsidered '96. In: Ingwersen, P. and Pors, N.O., eds. Information Science: Integration in Perspective. Copenhagen: Royal School of Librarianship, 1996, 201-218.
	22	Saracevic, T., Mokros, H. & Su, L.T. Nature of interaction between users and intermediaries in on-line searching: qualitative analysis. ASIS Proceeding, 1990, 47-54.
	23	Tefko Saracevic , Paul Kantor , Alice Y. Chamis , Donna Trivison, A study of information seeking and retrieving. I. background and methodology, Readings in information retrieval, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1997
	24	Miranda Lee Pao, Term and citation retrieval: a field study, Information Processing and Management: an International Journal, v.29 n.1, p.95-112, Jan.–Feb. 1993 [doi>10.1016/0306-4573(93)90026-A]
	25	Carol Tenopir , Pamela Cahn, TARGET & FREESTYLE: Dialog and Mead join the relevance ranks, Online, v.18 n.3, p.31-47, May 1994
	26	C. J. Van Rijsbergen, Information Retrieval, Butterworth-Heinemann, Newton, MA, 1979
	27	Peter Wallis , James A. Thom, Relevance judgments for assessing recall, Information Processing and Management: an International Journal, v.32 n.3, p.273-286, May 1996 [doi>10.1016/0306-4573(95)00061-5]

CITED BY 25

	Peter Ingwerswen, Users in context, Lectures on information retrieval, Springer-Verlag New York, Inc., New York, NY, 2001
	Ellen M. Voorhees, Evaluation by highly relevant documents, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.74-82, September 2001, New Orleans, Louisiana, United States
	Kalervo Järvelin , Jaana Kekäläinen, IR evaluation methods for retrieving highly relevant documents, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.41-48, July 24-28, 2000, Athens, Greece
	Kalervo Järvelin , Jaana Kekäläinen, Cumulated gain-based evaluation of IR techniques, ACM Transactions on Information Systems (TOIS), v.20 n.4, p.422-446, October 2002
	Jaana Kekäläinen , Kalervo Järvelin, User-oriented evaluation methods for information retrieval: a case study based on conceptual models for query expansion, Exploring artificial intelligence in the new millennium, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2003
	Pia Borlund, The concept of relevance in IR, Journal of the American Society for Information Science and Technology, v.54 n.10, p.913-925, August 2003
	Vincenzo Della Mea , Stefano Mizzaro, Measuring retrieval effectiveness: a new proposal and a first experimental validation, Journal of the American Society for Information Science and Technology, v.55 n.6, p.530-543, April 2004
	Eric Crestan , Claude de Loupy, Natural language processing for browse help, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom
	Jaana Kekäläinen, Binary and graded relevance in IR evaluations: comparison of the effects on ranking of IR systems, Information Processing and Management: an International Journal, v.41 n.5, p.1019-1033, September 2005
	Bruno Martins , Mário J. Silva , Marcirio Silveira Chaves, Challenges and resources for evaluating geographical IR, Proceedings of the 2005 workshop on Geographic information retrieval, November 04-04, 2005, Bremen, Germany
	Tomaž Klobučar, Evaluation of personalized search for learning resources, Proceedings of the 6th WSEAS International Conference on Distance Learning and Web Engineering, p.50-55, September 22-24, 2006, Lisbon, Portugal
	I. E. Kuralenok , I. S. Nekrestyanov, Evaluation of Text Retrieval Systems, Programming and Computing Software, v.28 n.4, p.226-242, July-August 2002
	Ryen W. White , Gheorghe Muresan , Gary Marchionini, Report on ACM SIGIR 2006 workshop on evaluating exploratory search systems, ACM SIGIR Forum, v.40 n.2, December 2006
	Paul Thomas , David Hawking, Evaluation by comparing result sets in context, Proceedings of the 15th ACM international conference on Information and knowledge management, November 06-11, 2006, Arlington, Virginia, USA
	Eric Crestan , Claude de Loupy, Browsing help for faster document retrieval, Proceedings of the 20th international conference on Computational Linguistics, p.576-es, August 23-27, 2004, Geneva, Switzerland
	Arthur R. Taylor , Colleen Cool , Nicholas J. Belkin , William J. Amadio, Relationships between categories of relevance criteria and stage in task completion, Information Processing and Management: an International Journal, v.43 n.4, p.1071-1084, July, 2007
	Kuansan Wang , Toby Walker , Zijian Zheng, PSkip: estimating relevance ranking quality from web search clickthrough data, Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, June 28-July 01, 2009, Paris, France
	Kalervo Järvelin, An analysis of two approaches in information retrieval: From frameworks to study designs, Journal of the American Society for Information Science and Technology, v.58 n.7, p.971-986, May 2007
	H. O. Nyongesa , S. Maleki-Dizaji, User modelling using evolutionary interactive reinforcement learning, Information Retrieval, v.9 n.3, p.343-355, June 2006
	Orland Hoeber , Xue Dong Yang, Evaluating WordBars in exploratory Web search scenarios, Information Processing and Management: an International Journal, v.44 n.2, p.485-510, March, 2008
	Iain Campbell, Interactive Evaluation of the Ostensive ModelUsing a New Test Collection of Images with Multiple Relevance Assessments, Information Retrieval, v.2 n.1, p.89-114, February 2000
	Norbert Fuhr, A probability ranking principle for interactive information retrieval, Information Retrieval, v.11 n.3, p.251-265, June 2008
	Ye Diana Wang , Guisseppi Forgionne, Testing a decision-theoretic approach to the evaluation of information retrieval systems, Journal of Information Science, v.34 n.6, p.861-876, December 2008
	Alistair Moffat , Justin Zobel, Rank-biased precision for measurement of retrieval effectiveness, ACM Transactions on Information Systems (TOIS), v.27 n.1, p.1-27, December 2008
	Diane Kelly, Methods for Evaluating Interactive Information Retrieval Systems with Users, Foundations and Trends in Information Retrieval, v.3 n.1—2, p.1-224, January 2009