Learning effective ranking functions for newsgroup search

ABSTRACT

Web communities are web virtual broadcasting spaces where people can freely discuss anything. While such communities function as discussion boards, they have even greater value as large repositories of archived information. In order to unlock the value of this resource, we need an effective means for searching archived discussion threads. Unfortunately the techniques that have proven successful for searching document collections and the Web are not ideally suited to the task of searching archived community discussions. In this paper, we explore the problem of creating an effective ranking function to predict the most relevant messages to queries in community search. We extract a set of predictive features from the thread trees of newsgroup messages as well as features of message authors and lexical distribution within a message thread. Our final results indicate that when using linear regression with this feature set, our search system achieved a 28.5% performance improvement compared to our baseline system.

REFERENCES

• 1: James P. Callan, Passage-level evidence in document retrieval, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.302-310, July 03-06, 1994, Dublin, Ireland
• 2: Soumen Chakrabarti , Mukul Joshi , Vivek Tawde, Enhanced topic distillation using text, markup tags, and hyperlinks, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.208-216, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383990]
• 3: Aitao Chen, A comparison of regression, neural net, and pattern recognition approaches to IR, Proceedings of the seventh international conference on Information and knowledge management, p.140-147, November 02-07, 1998, Bethesda, Maryland, United States [doi>10.1145/288627.288650]
• 4: Jinlin Chen , Baoyao Zhou , Jin Shi , Hongjiang Zhang , Qiu Fengwu, Function-based object model towards website adaptation, Proceedings of the 10th international conference on World Wide Web, p.587-596, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.372161]
• 5: D. W. Embley , Y. Jiang , Y.-K. Ng, Record-boundary discovery in Web documents, Proceedings of the 1999 ACM SIGMOD international conference on Management of data, p.467-478, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
• 6: Weiguo Fan , Michael D. Gordon , Praveen Pathak, A generic ranking function discovery framework by genetic programming for information retrieval, Information Processing and Management: an International Journal, v.40 n.4, p.587-602, May 2004 [doi>10.1016/j.ipm.2003.08.001]
• 7: Andrew T. Fiore , Scott Lee Tiernan , Marc A. Smith, Observed behavior and perceived value of authors in usenet newsgroups: bridging the gap, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503434]
• 8: Norbert Fuhr, Integration of probabilistic fact and text retrieval, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.211-222, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133202]
• 9: S. Fujita. "More Reflections on "Aboutness"". TREC-2001 Evaluation Experiments at Justsystem. In Proceedings of the Tenth Text Retrieval Conference (TREC 2001). Gaithersburg, Maryland, NIST Special Publication 500--250. 2002.
• 10: F.C. Gey, A. Chen, J. He, and J. Meggs. "Logistic regression at TREC4: Probabilistic retrieval from full text document collections." In Proceedings of the Fourth Text Retrieval Conference (TREC 4). Gaithersburg, Maryland, NIST Special Publication 500--236. 1996.
• 11: Xiao-Dong Gu , Jinlin Chen , Wei-Ying Ma , Guo-Liang Chen, Visual Based Content Understanding towards Web Adaptation, Proceedings of the Second International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems, p.164-173, May 29-31, 2002
• 12: D.K. Harman, "Overview of the Fourth Text Retrieval Conference (TREC-4)," In Proceedings of the Fourth Text Retrieval Conference (TREC-4), Gaithersburg, Maryland, NIST Special Publication 500-236, pp. 1--23, 1995.
• 13: Marti A. Hearst , Christian Plaunt, Subtopic structuring for full-length document access, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.59-68, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160695]
• 14: G. Kazai, M. Lalmas and T. Roelleke. "A Model for the Representation and Focused Retrieval of Structured Documents based on Fuzzy Aggregation", In Proceedings of the 8th International Symposium on String Processing and Information Retrieval, Laguna de San Rafael, Chile, pp. 123--135, 2001.
• 15: Marcin Kaszkiel , Justin Zobel, Passage retrieval revisited, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.178-185, July 27-31, 1997, Philadelphia, Pennsylvania, United States
• 16: Joon Ho Lee, Combining multiple evidence from different properties of weighting schemes, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.180-188, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215358]
• 17: Joon Ho Lee, Analyses of multiple evidence combination, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.267-276, July 27-31, 1997, Philadelphia, Pennsylvania, United States
• 18: D. Lewis "Applying Support Vector Machines to the TREC-2001 Batch Filtering and Routing Tasks". In Proceedings of the Tenth Text Retrieval Conference (TREC 2001). Gaithersburg, Maryland, NIST Special Publication 500--250. 2002.
• 19: Elke Mittendorf , Peter Schäuble, Document and passage retrieval based on hidden Markov models, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.318-327, July 03-06, 1994, Dublin, Ireland
• 20: A. Moffat, R. Sack-Davis, R. Wilkinson, and Zobel, J. "Retrieval of Partial Document." In Proceedings of the Second Text Retrieval Conference (TREC-2), pp.181--190. NIST Special Publication pp. 500--215, 1994.
• 21: Paul Ogilvie , Jamie Callan, Combining document representations for known-item search, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860463]
• 22: S.E. Robertson, "Overview of the Okapi Projects, Journal of Documentation, Vol. 53, No.1, pp. 3--7, 1997.
• 23: Gerard Salton , J. Allan , Chris Buckley, Approaches to passage retrieval in full text information systems, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.49-58, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160693]
• 24: J.A. Shaw & E.A. Fox, "Combination of multiple searches", In Proceedings of the 3rd Text Retrieval Conference (TREC-3). Gaithersburg, Maryland: NIST Special Publication 500-250, pp.105--107, 1995.
• 25: C.C. Vogt and G.W. Cottrell. "Fusion via linear combination for the routing problem". In Proceedings of the Sixth Text Retrieval Conference (TREC 2001). NIST Special Publication 500--250. 1998.
• 26: Ross Wilkinson, Effective retrieval of structured documents, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.311-317, July 03-06, 1994, Dublin, Ireland
• 27: W. Xi and E. A. Fox. "Machine Learning Approach for Homepage Finding task". In Proceedings of the Tenth Text Retrieval Conference (TREC 2001). Gaithersburg, Maryland, NIST Special Publication 500--250. 2002.
• 28: W. Xi, "Combining Multiple Source of Evidence for Information Retrieval," Master Thesis, Nanyang Technological University, Singapore, 2000.
• 29: Jinxi Xu , W. Bruce Croft, Query expansion using local and global document analysis, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.4-11, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243202]
• 30: Shipeng Yu , Deng Cai , Ji-Rong Wen , Wei-Ying Ma, Improving pseudo-relevance feedback in web information retrieval using web page segmentation, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775155]
• 31: Justin Zobel , Alistair Moffat , Ross Wilkinson , Ron Sacks-Davis, Efficient retrieval of partial documents, Information Processing and Management: an International Journal, v.31 n.3, p.361-377, May-June 1995

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Experimentation

Keywords:
information retrieval, linear regression, machine learning, newsgroup search, support vector machines