High precision retrieval using relevance-flow graph

ABSTRACT

Traditional bag-of-words information retrieval models use aggregated term statistics to measure the relevance of documents, making it difficult to detect non-relevant documents that contain many query terms by chance or in the wrong context. In-depth document analysis is needed to filter out these deceptive documents. In this paper, we hypothesize that truly relevant documents have relevant sentences in predictable patterns. Our experimental results show that we can successfully identify and exploit these patterns to significantly improve retrieval precision at top ranks.

REFERENCES

• 1: W. Bruce Croft , John Lafferty, Language Modeling for Information Retrieval, Kluwer Academic Publishers, Norwell, MA, 2003
• 2: E. Fox and J. Shaw. Combination of multiple searches. In Proc. of TREC-2, 1994.
• 3: Jeffrey C. Reynar , Adwait Ratnaparkhi, A maximum entropy approach to identifying sentence boundaries, Proceedings of the fifth conference on Applied natural language processing, p.16-19, March 31-April 03, 1997, Washington, DC [doi>10.3115/974557.974561]
• 4: Tao Tao , ChengXiang Zhai, An exploration of proximity measures in information retrieval, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277794]
• 5: Adam D. Troy , Guo-Qiang Zhang, Enhancing relevance scoring with chronological term rank, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277844]
• 6: X.-B. Xue and Z.-H. Zhou. Distributional features for text categorization. In Proc. of ECML '06, pages 497--508, 2006.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms, Experimentation, Measurement

Keywords:
re-ranking, relevance flow, relevant sentence