Picture this: preferences for image search

ABSTRACT

We demonstrate a system designed to elicit relative relevance judgments from users to rank images with respect to an image query. The system has been deployed and in use publicly for approximately one year. Furthermore, preference data collected from the users has been made available for research purposes. Further details regarding research on this system is available from Bennett et al. [1].

REFERENCES

• 1: P. N. Bennett, D. M. Chickering, and A. Mityagin. Learning consensus opinion: Mining data from a labeling game. In WWW 2009, 2009.
• 2: C. Cleverdon. The significance of the cranfield tests on index languages. In SIGIR '91, pages 3--12, 1991.
• 3: F. Radlinski and T. Joachims. Minimally invasive randomization for collecting unbiased preferences from clickthrough logs. In AAAI 2005, 2005.
• 4: F. Radlinski, R. Kleinberg, and T. Joachims. Learning diverse rankings with multi-armed bandits. In ICML 2008, 2008.
• 5: S. E. Robertson. The probability ranking principle in ir. Journal of Documentation, 33:294--304, 1977. Reprinted in: K. Sparck Jones and P. Willett (eds), Readings in Information Retrieval. Morgan Kaufmann, 1997. (pp 281--286).
• 6: L. von Ahn and L. Dabbish. Labeling images with a computer game. In ACM CHI, 2004.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Selection process

General Terms:
Algorithms, Performance

Keywords:
learning preferences, preference judgments