Discriminative probabilistic models for passage based retrieval

ABSTRACT

The approach of using passage-level evidence for document retrieval has shown mixed results when it is applied to a variety of test beds with different characteristics. One main reason of the inconsistent performance is that there exists no unified framework to model the evidence of individual passages within a document. This paper proposes two probabilistic models to formally model the evidence of a set of top ranked passages in a document. The first probabilistic model follows the retrieval criterion that a document is relevant if any passage in the document is relevant, and models each passage independently. The second probabilistic model goes a step further and incorporates the similarity correlations among the passages. Both models are trained in a discriminative manner. Furthermore, we present a combination approach to combine the ranked lists of document retrieval and passage-based retrieval.

An extensive set of experiments have been conducted on four different TREC test beds to show the effectiveness of the proposed discriminative probabilistic models for passage-based retrieval. The proposed algorithms are compared with a state-of-the-art document retrieval algorithm and a language model approach for passage-based retrieval. Furthermore, our combined approach has been shown to provide better results than both document retrieval and passage-based retrieval approaches.

REFERENCES

• 1: James Allan, Relevance feedback with too much data, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.337-343, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215380]
• 2: James P. Callan, Passage-level evidence in document retrieval, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.302-310, July 03-06, 1994, Dublin, Ireland
• 3: Gordon V. Cormack , Charles L. A. Clarke , Christopher R. Palmer , Samuel S. L. To, Passage-based query refinement (MultiText experiments for TREC-6), Information Processing and Management: an International Journal, v.36 n.1, p.133-153, Jan. 2000 [doi>10.1016/S0306-4573(99)00048-5]
• 4: Richard O. Duda , Peter E. Hart , David G. Stork, Pattern Classification (2nd Edition), Wiley-Interscience, 2000
• 5: Jianfeng Gao , Haoliang Qi , Xinsong Xia , Jian-Yun Nie, Linear discriminant model for information retrieval, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076085]
• 6: D. J. Harper. Relevance Feedback in Document Retrieval Systems: An Evaluation of Probabilistic Strategies. PhD thesis, Cambridge University, 1980.
• 7: Marti A. Hearst , Christian Plaunt, Subtopic structuring for full-length document access, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.59-68, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160695]
• 8: J. E. Dennis, Jr. , Robert B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Classics in Applied Mathematics, 16), Soc for Industrial & Applied Math, 1996
• 9: N. A. Jaleel, A. Corrada-Emmanuel, Q. Li, X. Liu, C. Wade, and J. Allan. Umass at trec 2003: Hard and qa. In Proc. of TREC-12, 2003.
• 10: Marcin Kaszkiel , Justin Zobel, Passage retrieval revisited, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.178-185, July 27-31, 1997, Philadelphia, Pennsylvania, United States
• 11: Marcin Kaszkiel , Justin Zobel, Effective ranking with arbitrary passages, Journal of the American Society for Information Science and Technology, v.52 n.4, p.344-364, Feb. 2001 [doi>10.1002/1532-2890(2000)9999:9999<::AID-ASI1075>3.3.CO;2-R]
• 12: Experimental Evaluation of Passage-Based Document Retrieval, Proceedings of the Sixth International Conference on Document Analysis and Recognition, p.592, September 10-13, 2001
• 13: D. Knaus, E. Mittendorf, and P. Schauble. Improving a basic retrieval method by links and passage level evidence. In Proc. of TREC-3, 1994.
• 14: Victor Lavrenko , W. Bruce Croft, Relevance based language models, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.120-127, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383972]
• 15: Joon Ho Lee, Analyses of multiple evidence combination, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.267-276, July 27-31, 1997, Philadelphia, Pennsylvania, United States
• 16: Xiaoyong Liu , W. Bruce Croft, Passage retrieval based on language models, Proceedings of the eleventh international conference on Information and knowledge management, November 04-09, 2002, McLean, Virginia, USA [doi>10.1145/584792.584854]
• 17: Donald Metzler , W. Bruce Croft, Combining the language model and inference network approaches to retrieval, Information Processing and Management: an International Journal, v.40 n.5, p.735-750, September 2004 [doi>10.1016/j.ipm.2004.05.001]
• 18: Elke Mittendorf , Peter Schäuble, Document and passage retrieval based on hidden Markov models, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.318-327, July 03-06, 1994, Dublin, Ireland
• 19: Ramesh Nallapati, Discriminative models for information retrieval, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009006]
• 20: Gerard Salton , J. Allan , Chris Buckley, Approaches to passage retrieval in full text information systems, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.49-58, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160693]
• 21: Mark Sanderson , Justin Zobel, Information retrieval system evaluation: effort, sensitivity, and reliability, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076064]
• 22: C. Wade and J. Allan. Passage retrieval and evaluation. Technical report, CIIR, University of Massachusetts, 2005.
• 23: Ross Wilkinson, Effective retrieval of structured documents, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.311-317, July 03-06, 1994, Dublin, Ireland
• 24: W. Xi, R. Xu-Rong, C. S. Khoo, and E.-P. Lim. Incorporating window-based passage-level evidence in document retrieval. JIS, 27(2):73--80, 2001.
• 25: Justin Zobel , Alistair Moffat , Ross Wilkinson , Ron Sacks-Davis, Efficient retrieval of partial documents, Information Processing and Management: an International Journal, v.31 n.3, p.361-377, May-June 1995

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms, Design, Experimentation

Keywords:
IR, discriminative models, passage retrieval