Bayesian online classifiers for text classification and filtering

ABSTRACT

This paper explores the use of Bayesian online classifiers to classify text documents. Empirical results indicate that these classifiers are comparable with the best text classification systems. Furthermore, the online approach offers the advantage of continuous learning in the batch-adaptive text filtering task.

REFERENCES

• 1: G. Cauwenberghs and T. Poggio. Incremental and decremental support vector machine learning. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, NIPS 2000, volume 13. The MIT Press, 2001.
• 2: D. Cox and E. Snell. Analysis of Binary Data. Chapman & Hall, London, 2nd edition, 1989.
• 3: L. Csató and M. Opper. Sparse representation for Gaussian process models. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, NIPS 2000, volume 13. The MIT Press, 2001.
• 4: Harris Drucker , Behzad Shahrary , David Gibbon, Relevance Feedback using Support Vector Machines, Proceedings of the Eighteenth International Conference on Machine Learning, p.122-129, June 28-July 01, 2001
• 5: Ted Dunning, Accurate methods for the statistics of surprise and coincidence, Computational Linguistics, v.19 n.1, March 1993
• 6: William Hersh , Chris Buckley , T. J. Leone , David Hickam, OHSUMED: an interactive retrieval evaluation and new large test collection for research, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.192-201, July 03-06, 1994, Dublin, Ireland
• 7: Thorsten Joachims, Text Categorization with Suport Vector Machines: Learning with Many Relevant Features, Proceedings of the 10th European Conference on Machine Learning, p.137-142, April 21-23, 1998
• 8: Thorsten Joachims, Making large-scale support vector machine learning practical, Advances in kernel methods: support vector learning, MIT Press, Cambridge, MA, 1999
• 9: David Dolan Lewis, Representation and learning in information retrieval, University of Massachusetts, Amherst, MA, 1992
• 10: David D. Lewis, Evaluating and optimizing autonomous text classification systems, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.246-254, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215366]
• 11: David D. Lewis , Robert E. Schapire , James P. Callan , Ron Papka, Training algorithms for linear text classifiers, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.298-306, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243277]
• 12: David J. C. MacKay, Bayesian interpolation, Neural Computation, v.4 n.3, p.415-447, May 1992 [doi>10.1162/neco.1992.4.3.415]
• 13: David J. C. MacKay, Information-based objective functions for active data selection, Neural Computation, v.4 n.4, p.590-604, July 1992 [doi>10.1162/neco.1992.4.4.590]
• 14: R. M. Neal. Monte Carlo implementation of Gaussian process models for Bayesian regression and classification. Technical Report CRG-TR-97-2, Department of Computer Science, University of Toronto, January 1997.
• 15: Hwee Tou Ng , Wei Boon Goh , Kok Leong Low, Feature selection, perception learning, and a usability case study for text categorization, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.67-73, July 27-31, 1997, Philadelphia, Pennsylvania, United States
• 16: M. Opper. Online versus offline learning from random examples: General results. Physical Review Letters, 77:4671--4674, 1996.
• 17: Manfred Opper, A Bayesian approach to on-line learning, On-line learning in neural networks, Cambridge University Press, New York, NY, 1999
• 18: S. Robertson and D. A. Hull. The TREC-9 filtering track final report. In Proceedings of the 9th Text REtrieval Conference (TREC-9), pages 25--40, 2001.
• 19: Gerard Salton , Christopher Buckley, Term-weighting approaches in automatic text retrieval, Information Processing and Management: an International Journal, v.24 n.5, p.513-523, 1988 [doi>10.1016/0306-4573(88)90021-0]
• 20: Sara Solla , Ole Winther, Optimal perceptron learning: as online Bayesian approach, On-line learning in neural networks, Cambridge University Press, New York, NY, 1999
• 21: N. A. Syed, H. Liu, and K. K. Sung. Incremental learning with support vector machines. In Proceedings of the Workshop on Support Vector Machines at the International Joint Conference on Artificial Intelligence (IJCAI-99), 1999.
• 22: C. J. Van Rijsbergen, Information Retrieval, Butterworth-Heinemann, Newton, MA, 1979
• 23: Vladimir N. Vapnik, The nature of statistical learning theory, Springer-Verlag New York, Inc., New York, NY, 1995
• 24: C. K. Williams and M. Seeger. Using the Nyström method to speed up kernel machines. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, NIPS 2000, volume 13. The MIT Press, 2001.
• 25: O. Winther. Bayesian Mean Field Algorithms for Neural Networks and Gaussian Processes. PhD thesis, University of Copenhagen, CONNECT, The Niels Bohr Institute, 1998.
• 26: Yiming Yang, A study of thresholding strategies for text categorization, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.137-145, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383975]
• 27: Yiming Yang , Xin Liu, A re-examination of text categorization methods, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.42-49, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312647]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Information filtering

General Terms:
Algorithms, Experimentation

Keywords:
bayesian, machine learning, online, text classification, text filtering