Active learning using pre-clustering

ABSTRACT

The paper is concerned with two-class active learning. While the common approach for collecting data in active learning is to select samples close to the classification boundary, better performance can be achieved by taking into account the prior data distribution. The main contribution of the paper is a formal framework that incorporates clustering into active learning. The algorithm first constructs a classifier on the set of the cluster representatives, and then propagates the classification decision to the other samples via a local noise model. The proposed model allows to select the most representative samples as well as to avoid repeatedly labeling samples in the same cluster. During the active learning process, the clustering is adjusted using the coarse-to-fine strategy in order to balance between the advantage of large clusters and the accuracy of the data representation. The results of experiments in image databases show a better performance of our algorithm compared to the current methods.

REFERENCES

• 1: Colin Campbell , Nello Cristianini , Alex J. Smola, Query Learning with Large Margin Classifiers, Proceedings of the Seventeenth International Conference on Machine Learning, p.111-118, June 29-July 02, 2000
• 2: Chapelle, O., Weston, J., & Scholkopf, B. (2002). Cluster kernels for semi-supervised learning. Advances in Neural Information Processing Systems.
• 3: Cohn, D. A., Ghahramani, Z., & Jordan, M. I. (1996). Active learning with statistical models. Journal of Artificial Intelligence research, 4, 129--145.
• 4: Kaufman, L., & Rousseeuw, P. (1990). Finding groups in data: An introduction to cluster analysis. John Wiley & Sons.
• 5: David D. Lewis , William A. Gale, A sequential algorithm for training text classifiers, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.3-12, July 03-06, 1994, Dublin, Ireland
• 6: Andrew McCallum , Kamal Nigam, Employing EM and Pool-Based Active Learning for Text Classification, Proceedings of the Fifteenth International Conference on Machine Learning, p.350-358, July 24-27, 1998
• 7: Miller, D., & Uyar, H. (1996). A mixture of experts classifier with learning based on both labelled and unlabelled data. Advances in Neural Information Processing Systems 9 (pp. 571--577).
• 8: Kamal Nigam , Andrew Kachites McCallum , Sebastian Thrun , Tom Mitchell, Text Classification from Labeled and Unlabeled Documents using EM, Machine Learning, v.39 n.2-3, p.103-134, May-June 2000 [doi>10.1023/A:1007692713085]
• 9: Thang V. Pham , Marcel Worring , Arnold W. M. Smeulders, Face detection by aggregated Bayesian network classifiers, Pattern Recognition Letters, v.23 n.4, p.451-461, February 2002 [doi>10.1016/S0167-8655(01)00177-5]
• 10: Nicholas Roy , Andrew McCallum, Toward Optimal Active Learning through Sampling Estimation of Error Reduction, Proceedings of the Eighteenth International Conference on Machine Learning, p.441-448, June 28-July 01, 2001
• 11: Greg Schohn , David Cohn, Less is More: Active Learning with Support Vector Machines, Proceedings of the Seventeenth International Conference on Machine Learning, p.839-846, June 29-July 02, 2000
• 12: Seeger, M. (2001). Learning with labeled and unlabeled data (Technical Report). Edinburgh University.
• 13: Shen, X., & Zhai, C. (2003). Active feedback - UIUC TREC-2003 HARD experiments. The 12th Text Retrieval Conference, TREC.
• 14: Anja Struyf , Mia Hubert , Peter J. Rousseeuw, Integrating robust clustering techniques in S-PLUS, Computational Statistics & Data Analysis, v.26 n.1, p.17-37, Nov. 6, 1997 [doi>10.1016/S0167-9473(97)00020-0]
• 15: Min Tang , Xiaoqiang Luo , Salim Roukos, Active learning for statistical natural language parsing, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073105]
• 16: Simon Tong , Edward Chang, Support vector machine active learning for image retrieval, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500159]
• 17: Simon Tong , Daphne Koller, Support vector machine active learning with applications to text classification, The Journal of Machine Learning Research, 2, p.45-66, 3/1/2002
• 18: Xu, Z., Yu, K., Tresp, V., Xu, X., & Wang, J. (2003). Representative sampling for text classification using support vector machines. 25th European Conf. on Information Retrieval Research, ECIR 2003. Springer.
• 19: Zhang, C., & Chen, T. (2002). An active learning framework for content-based information retrieval. IEEE trans on multimedia, 4, 260--268.
• 20: Zhang, T., & Oles, F. (2000). A probability analysis on the value of unlabeled data for classification problems. Proc. Int. Conf. on Machine Learning.
• 21: Tong Zhang , Frank J. Oles, Text Categorization Based on Regularized Linear Classification Methods, Information Retrieval, v.4 n.1, p.5-31, April 2001 [doi>10.1023/A:1011441423217]
• 22: Zhu, J., & Hastie, T. (2001). Kernel logistic regression and the import vector machine. Advances in Neural Information Processing Systems.