Presentation schemes for component analysis in IR experiments

ABSTRACT

Information retrieval research, at least as conceived by the SIGIR community, is fundamentally experimental in nature. As such, the presentation of results from controlled, reproducible experiments lies at the core of our work. Many reports follow the same general format: authors propose a new retrieval method, whose performance on some well-defined task is compared against a baseline. Authors also report results from alternative configurations, e.g., variations in parameters, turning off (ablation) of different components, etc. The presentation of experimental results forms an integral part of the conferences and journals that comprise the medium in which knowledge is disseminated.

REFERENCES

• 1: N. J. Belkin , P. Kantor , E. A. Fox , J. A. Shaw, Combining the evidence of multiple query representations for information retrieval, Information Processing and Management: an International Journal, v.31 n.3, p.431-448, May-June 1995
• 2: Jimmy Lin , Dina Demner-Fushman, The role of knowledge in conceptual retrieval: a study in the domain of clinical medicine, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148191]
• 3: J. McCarthy. AI as Sport. Science, 276(5318):1518--1519, June 1997.
• 4: Edward R. Tufte, The visual display of quantitative information, Graphics Press, Cheshire, CT, 1986

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software
          Subjects: Performance evaluation (efficiency and effectiveness)