Quantative analysis of the impact of judging inconsistency on the performance of relevance feedback

ABSTRACT

Practical constrains of user interfaces make the user's judgment (during the feedback loop) deviate from real thoughts (when the full document is read).This is often overlooked in evaluation of relevance feedback.This paper quantitatively analyze the impact of judging inconsistency on the performance of relevance feedback.

REFERENCES

• 1: X. Jin, J. French, and J. Michel. Saic and university of virginia at trec 2005: Hard track. In TREC 2005.
• 2: X. Jin, J. C. French, and J. Michel. Toward consistent evaluation of relevance feedback approaches in multimedia retrieval. In Adaptive Multimedia Retrieval pages 191--206, 2005.
• 3: J. Rocchio. Relevance feedback in information retrieval. In G. Salton, editor, The SMART Retrieval System: Experiments in Automatic Document Processing pages 313--323. Prentice-Hall, 1971.
• 4: G. Salton and C. Buckley. Improving retrieval performance by relevance feedback. JASIS 41(4):288--297, 1990.
• 5: X. Shen and C.Zhai. Active feedback-UIUC TREC-2003 HARD experiments. In TREC pages 662--666, 2003.
• 6: HongJiang Zhang , Zhong Su, Relevance Feedback in CBIR, Proceedings of the IFIP TC2/WG2.6 Sixth Working Conference on Visual Database Systems: Visual and Multimedia Information Management, p.21-35, May 29-31, 2002

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Relevance feedback

General Terms:
Experimentation, Measurement, Performance

Keywords:
judging inconsistency, performance evaluation, relevance feedback