Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

Word sense disambiguation (WSD) is a crucial issue in bio-medical text mining since the performance of diverse biomedical text mining techniques strongly depends on the senses of lexicons. Thus, it is natural to consider lexicons as the most crucial features in WSD. However, due to the diversity of lexical space, WSD methods based on machine learning techniques with lexical features suffer from the difference between distributions of training and test documents. To tackle this problem, this paper proposes support vector machines with example-wise weights. In this method, the training distribution is made coincide with the test distribution by weighting training examples according to their similarity to all test data. The experimental results show that the distribution change between training and test data is actually recognized and the proposed method which considers this change in its training phase outperforms ordinary support vector machines.

REFERENCES

• 1: A. Arnold, R. Nallapati, and W. Cohen: Exploiting Feature Hierarchy for Transfer Learning in Naemd Entity Recognition: In Proceedings of the 46th Annual Meeting of the Association of Computational Linguistics, 245--253, 2008.
• 2: O. Bodenreider: The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Research, 32:267--270, 2004.
• 3: O. Chapelle, B. Schölkopf, and A. Zien: Semi-Supervised Learning: MIT Press, 2006.
• 4: C. Chelba and A. Acero: Adaptation of Maximum Entropy Capitalizer: Little Data Can Help a Lot: In Proceedings of Empirical Methods in Natural Language Processing and Very Large Corpora, 285--292, 2004.
• 5: H. Daumé and D. Marcu: Frustratingly Easy Domain Adaptation: In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, 256--263, 2007.
• 6: W. Duan, M. Song, and A. Yates: Fast Max-margin Clustering for Unsupervised Word Sense Disambiguation in Biomedical Texts: BMC Bioinformatics, 10(3), 2009
• 7: Hervé Déjean, Learning rules and their exceptions, The Journal of Machine Learning Research, 2, 3/1/2002
• 8: Gerard Escudero , Lluís Màrquez , German Rigau, An empirical study of the domain dependence of supervised word sense disambiguation systems, Proceedings of the 2000 Joint SIGDAT conference on Empirical methods in natural language processing and very large corpora: held in conjunction with the 38th Annual Meeting of the Association for Computational Linguistics, p.172-180, October 07-08, 2000, Hong Kong [doi>10.3115/1117794.1117816]
• 9: R. Florian, H. Hassan, A. Ittycheriah, H. Jing, N. Kambhatla, X. Luo, N. Nicolov, and S. Roukos: A Statistical Model for Multilingual Entity Detection and Tracking: In Proceedings of Human Language Technology and North American Chapter of the Association for Computational Linguistics Annual Meeting, 1--8, 2004.
• 10: V. Hatzivassiloglou, PA. Duboue, and A. Rzhetsky: Disambiguating proteins, genes, and RNA in text: A machine learning approach: In Proceedings of the Ninth International Conference on Intelligent System for Molecular Biology, 2001.
• 11: J. Heckman: Sample Selection Bias as a Specification Error: Econometrica, 47(1):153--162, 1979.
• 12: J. Huang, A. Smola, A. Gretton, K. Borgwardt, and B. Schölkopf: Correcting Sample Selection Bias by Unlabeled Data: Advances in Neural Information Processing Systems 19, 601--608, MIT Press, 2007.
• 13: J. Jiang and C. Zhai: Instance Weighting for Domain Adaptation in NLP: In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, 264--271, 2007.
• 14: T. Joachims: Making Large-Scale SVM Learning Practical, LS8, Universitaet Dortmund, 1998.
• 15: Taku Kudo , Yuji Matsumoto, Chunking with support vector machines, Second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies 2001, p.1-8, June 01-07, 2001, Pittsburgh, Pennsylvania [doi>10.3115/1073336.1073361]
• 16: H. Liu, V. Teller, and C. Friedman: A Multi-aspect Comparison Study of Supervised Word Sense Disambiguation. Journal of the Americal Medical Informatics Association, 11:320--331, 2004.
• 17: David Martinez , Eneko Agirre, One sense per collocation and genre/topic variations, Proceedings of the 2000 Joint SIGDAT conference on Empirical methods in natural language processing and very large corpora: held in conjunction with the 38th Annual Meeting of the Association for Computational Linguistics, p.207-215, October 07-08, 2000, Hong Kong [doi>10.3115/1117794.1117820]
• 18: R. Milidíu, C. Santos, and J. Duarte: Phrase Chunking Using Entropy Guided Transformation Leanring: In Proceedings of the 46th Annual Meeting of the Association of Computational Linguistics, 647--655, 2008.
• 19: T. Pahikkala, S. Pyysalo, F. Ginter, J. Boberg, J. Jarvinen, and T. Salakoski: Kernels Incorporating Word Positional Information in Natural Language Disambiguation Tasks: In Proceedings of FLAIRS, 2005.
• 20: Seong-Bae Park , Byoung-Tak Zhang, A Boosted Maximum Entropy Model for Learning Text Chunking, Proceedings of the Nineteenth International Conference on Machine Learning, p.482-489, July 08-12, 2002
• 21: Brian Roark , Michiel Bacchiani, Supervised and unsupervised PCFG adaptation to novel domains, Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, p.126-133, May 27-June 01, 2003, Edmonton, Canada [doi>10.3115/1073445.1073472]
• 22: J. Shawe-Taylor and N. Cristianini: Support Vector Machines and Other Kernel-based Learning Methods: Cambridge University Press, 2000.
• 23: B. Schijvenaars, B. Mons, M. Weeber, M. Schumie, E. Mulligen, H. Wain, and J. Kors: Thesaurus-based disambiguation of gene symbols. BMC Bioinformatics, 6, 2005.
• 24: M. Schmidt , H. Gish, Speaker identification via support vector classifiers, Proceedings of the Acoustics, Speech, and Signal Processing, 1996. on Conference Proceedings., 1996 IEEE International Conference, p.105-108, May 07-10, 1996 [doi>10.1109/ICASSP.1996.540301]
• 25: M. Schuemie, J. Kors, and B. Mons: Word Sense Disambiguation in the Biomedical domain: An overview: journal of Computational Biology, 27(3):321--349, 2001.
• 26: H. Shimodaira: Improving Predictive Inference Under Covariate Shift by Weighting the Log-Likelihood Function: Journal of Statistical Planning and Inference, 90(2):227--244, 2000.
• 27: M. Sugiyama, S. Nakajima, H. Kashima, P. Bünau, and M. Kawanabe: Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation: Advances in Neural Information Processing Systems 20, 1433--1440, MIT Press, 2008.
• 28: Ioannis Tsochantaridis , Thorsten Joachims , Thomas Hofmann , Yasemin Altun, Large Margin Methods for Structured and Interdependent Output Variables, The Journal of Machine Learning Research, 6, p.1453-1484, 12/1/2005
• 29: M. Weeber, J. Mork, and A. Aronson: Developing a Test Collection for Biomedical Word Sense Disambiguation: In Proceedings of the AMIA 2001 Symposium, 2001.
• 30: Bianca Zadrozny, Learning and evaluating classifiers under sample selection bias, Proceedings of the twenty-first international conference on Machine learning, p.114, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015425]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Linguistic processing

General Terms:
Algorithms, Performance, Verification

Keywords:
biotext classification, covariate shift, distribution change, kernel method, machine learning