Users' stopping behaviors and estimates of recall

ABSTRACT

This paper investigates subjects' stopping behaviors and estimates of recall in an interactive information retrieval (IIR) experiment. Subjects completed four recall-oriented search tasks and were asked to estimate how many of the relevant documents they believed they had found after each task. Subjects also responded to an interview question probing their reasons for stopping a search. Results show that most subjects believed they found about 51-60% of the relevant documents and that this estimate was correlated positively with number of documents saved and actual recall, even though subjects' recall estimates were inaccurate. Reasons given for stopping search are also explored.

REFERENCES

• 1: David C. Blair , M. E. Maron, An evaluation of retrieval effectiveness for a full-text document-retrieval system, Communications of the ACM, v.28 n.3, p.289-299, March 1985 [doi>10.1145/3166.3197]
• 2: Brown, G. J., Pitts, M.J., and Wetherbe, J.C. (2007). Cognitive rules for terminating information search in online tasks. MIS Quarterly 31(1), 89--104.
• 3: Kelly, D., Cushing, A., Dostert, M.&Niu, X. (2009). Social search bias: Can users be induced to take bad query suggestions? SILS Tech. Report, 2009-01 (http://sils.unc.edu/research/techreports.html).
• 4: Voorhees, E. M. (2006). Overview of the TREC 2005 Robust Retrieval Track. Proceedings of TREC-14.
• 5: Lisl Zach, When is “enough” enough? Modeling the information-seeking and stopping behavior of senior arts administrators: Research Articles, Journal of the American Society for Information Science and Technology, v.56 n.1, p.23-35, January 2005 [doi>10.1002/asi.v56:1]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

General Terms:
Human Factors

Keywords:
decision making, recall estimates, stopping behavior, user study