ProbFuse: a probabilistic approach to data fusion

ABSTRACT

Data fusion is the combination of the results of independent searches on a document collection into one single output result set. It has been shown in the past that this can greatly improve retrieval effectiveness over that of the individual results.This paper presents probFuse, a probabilistic approach to data fusion. ProbFuse assumes that the performance of the individual input systems on a number of training queries is indicative of their future performance. The fused result set is based on probabilities of relevance calculated during this training process. Retrieval experiments using data from the TREC ad hoc collection demonstrate that probFuse achieves results superior to that of the popular CombMNZ fusion algorithm.

REFERENCES

• 1: Javed A. Aslam , Mark Montague, Bayes optimal metasearch: a probabilistic model for combining the results of multiple retrieval systems (poster session), Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.379-381, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345665]
• 2: Javed A. Aslam , Mark Montague, Models for metasearch, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.276-284, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.384007]
• 3: Brian T. Bartell , Garrison W. Cottrell , Richard K. Belew, Automatic combination of multiple ranked retrieval systems, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.173-181, July 03-06, 1994, Dublin, Ireland
• 4: Steven M. Beitzel , Eric C. Jensen , Abdur Chowdhury , David Grossman , Ophir Frieder , Nazli Goharian, Fusion of effective retrieval strategies in the same information retrieval system, Journal of the American Society for Information Science and Technology, v.55 n.10, p.859-868, August 2004 [doi>10.1002/asi.20012]
• 5: Chris Buckley , Ellen M. Voorhees, Retrieval evaluation with incomplete information, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009000]
• 6: James P. Callan , Zhihong Lu , W. Bruce Croft, Searching distributed collections with inference networks, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.21-28, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215328]
• 7: N. Craswell, D. Hawking, and P. B. Thistlewaite. Merging results from isolated search engines. In Australasian Database Conference, pages 189--200, Auckland, New Zealand, 1999.
• 8: Padima Das-Gupta , Jeffrey Katzer, A study of the overlap among document representations, Proceedings of the 6th annual international ACM SIGIR conference on Research and development in information retrieval, June 06-08, 1983, Bethesda, Maryland
• 9: E. A. Fox and J. A. Shaw. Combination of multiple searches. In Proceedings of the 2nd Text REtrieval Conference (TREC-2), National Institute of Standards and Technology Special Publication 500-215, pages 243--252, 1994.
• 10: Luis Gravano , Kevin Chang , Hector Garcia-Molina , Andreas Paepcke, STARTS: Stanford Protocol Proposal for Internet Retrieval and Search, Stanford University, Stanford, CA, 1997
• 11: Donna Harman, Overview of the first TREC conference, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.36-47, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160692]
• 12: A. E. Howe and D. Dreilinger. SavvySearch: A metasearch engine that learns which search engines to query. AI Magazine, 18(2):19--25, 1997.
• 13: Leah S. Larkey , Margaret E. Connell , Jamie Callan, Collection selection and results merging with topically organized U.S. patents and TREC data, Proceedings of the ninth international conference on Information and knowledge management, p.282-289, November 06-11, 2000, McLean, Virginia, United States [doi>10.1145/354756.354830]
• 14: Steve Lawrence , C. Lee Giles, Inquirus, the NECI meta search engine, Proceedings of the seventh international conference on World Wide Web 7, p.95-105, April 1998, Brisbane, Australia
• 15: Joon Ho Lee, Analyses of multiple evidence combination, ACM SIGIR Forum, v.31 n.SI, p.267-276
• 16: D. Lillis, F. Toolan, A. Mur, L. Peng, R. Collier, and J. Dunnion. Probability-based fusion of information retrieval result sets. In Proceedings of the 16th Irish Conference on Artificial Intelligence and Cognitive Science (AICS 2005), pages 147--156, Portstewart, Northern Ireland, 2005. University of Ulster.
• 17: R. Manmatha , T. Rath , F. Feng, Modeling score distributions for combining the outputs of search engines, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.267-275, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.384005]
• 18: Mark Montague , Javed A. Aslam, Relevance score normalization for metasearch, Proceedings of the tenth international conference on Information and knowledge management, October 05-10, 2001, Atlanta, Georgia, USA [doi>10.1145/502585.502657]
• 19: Mark Montague , Javed A. Aslam, Condorcet fusion for improved retrieval, Proceedings of the eleventh international conference on Information and knowledge management, November 04-09, 2002, McLean, Virginia, USA [doi>10.1145/584792.584881]
• 20: Allison L. Powell , James C. French , Jamie Callan , Margaret Connell , Charles L. Viles, The impact of database selection on distributed searching, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.232-239, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345584]
• 21: E. Selberg and O. Etzioni. The MetaCrawler architecture for resource aggregation on the Web. IEEE Expert, (January--February):11--14, 1997.
• 22: Luo Si , Jamie Callan, Using sampled data and regression to merge search engine results, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564382]
• 23: Christopher C. Vogt , Garrison W. Cottrell, Fusion Via a Linear Combination of Scores, Information Retrieval, v.1 n.3, p.151-173, October 1999 [doi>10.1023/A:1009980820262]
• 24: E. M. Voorhees, N. K. Gupta, and B. Johnson-Laird. The collection fusion problem. In Proceedings of the Third Text REtrieval Conference (TREC-3), pages 95--104, 1994.
• 25: Ellen M. Voorhees , Narendra K. Gupta , Ben Johnson-Laird, Learning collection fusion strategies, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.172-179, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215357]
• 26: Ellen M. Voorhees , Donna Harman, Overview of the sixth text REtrieval conference (TREC-6), Information Processing and Management: an International Journal, v.36 n.1, p.3-35, Jan. 2000 [doi>10.1016/S0306-4573(99)00043-6]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Experimentation

Keywords:
ProbFuse, data fusion, information retrieval