Intra-document structural frequency features for semi-supervised domain adaptation

ABSTRACT

In this work we try to bridge the gap often encountered by researchers who find themselves with few or no labeled examples from their desired target domain, yet still have access to large amounts of labeled data from other related, but distinct source domains, and seemingly no way to transfer knowledge from one to the other. Experimentally, we focus on the problem of extracting protein mentions from academic publications in the field of biology, where the source domain data are abstracts labeled with protein mentions, and the target domain data are wholly unlabeled captions. We mine the large number of such full text articles freely available on the Internet in order to supplement the limited amount of annotated data available. By exploiting the explicit and implicit common structure of the different subsections of these documents, including the unlabeled full text, we are able to generate robust features that are insensitive to changes in marginal and conditional distributions of classes and data across domains. We supplement these domain-insensitive features with automatically obtained high-confidence positive and negative predictions on the target domain to learn extractors that generalize well from one section of a document to another. Finally, lacking labeled target testing data, we employ comparative user preference studies to evaluate the relative performance of the proposed methods with respect to existing baselines.

REFERENCES

• 1: Rie Kubota Ando , Tong Zhang, A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data, The Journal of Machine Learning Research, 6, p.1817-1853, 12/1/2005
• 2: Andrew Arnold , Ramesh Nallapati , William W. Cohen, A Comparative Study of Methods for Transductive Transfer Learning, Proceedings of the Seventh IEEE International Conference on Data Mining Workshops, p.77-82, October 28-31, 2007 [doi>10.1109/ICDMW.2007.2]
• 3: A. Arnold, R. Nallapati, and W. W. Cohen. Exploiting feature hierarchy for transfer learning in named entity recognition. In ACL:HLT '08, 2008.
• 4: Jonathan Baxter, A Bayesian/Information Theoretic Model of Learning to Learn viaMultiple Task Sampling, Machine Learning, v.28 n.1, p.7-39, July 1997 [doi>10.1023/A:1007327622663]
• 5: S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira. Analysis of representations for domain adaptation. In NIPS 20, Cambridge, MA, 2007. MIT Press.
• 6: D. M. Blei, J. A. Bagnell, and A. McCallum. Learning with scope, with application to information extraction and classification. In UAI, pages 53--60, 2002.
• 7: J. Blitzer, R. McDonald, and F. Pereira. Domain adaptation with structural correspondence learning. In EMNLP, Sydney, Australia, 2006.
• 8: Avrim Blum , Tom Mitchell, Combining labeled and unlabeled data with co-training, Proceedings of the eleventh annual conference on Computational learning theory, p.92-100, July 24-26, 1998, Madison, Wisconsin, United States [doi>10.1145/279943.279962]
• 9: Rich Caruana, Multitask Learning, Machine Learning, v.28 n.1, p.41-75, July 1997 [doi>10.1023/A:1007379606734]
• 10: W. W. Cohen. Minorthird: Methods for identifying names and ontological relations in text using heuristics for inducing regularities from data. http://minorthird.sourceforge.net, 2004.
• 11: William W. Cohen , Richard Wang , Robert F. Murphy, Understanding captions in biomedical publications, Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, August 24-27, 2003, Washington, D.C. [doi>10.1145/956750.956809]
• 12: H. Daumé III. Frustratingly easy domain adaptation. In ACL, 2007.
• 13: H. Daumé III and D. Marcu. Domain adaptation for statistical classifiers. In Journal of Artificial Intelligence Research 26, pages 101--126, 2006.
• 14: K. Franzén, G. Eriksson, F. Olsson, L. Asker, P. Lidén, and J. Cöster. Protein names and how to find them. In International Journal of Medical Informatics, 2002.
• 15: William A. Gale , Kenneth W. Church , David Yarowsky, One sense per discourse, Proceedings of the workshop on Speech and Natural Language, February 23-26, 1992, Harriman, New York [doi>10.3115/1075527.1075579]
• 16: Y. Grandvalet and Y. Bengio. Semi-supervised learning by entropy minimization. In CAP, Nice, France, 2005.
• 17: Jing Jiang , ChengXiang Zhai, Exploiting domain structure for named entity recognition, Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, p.74-81, June 04-09, 2006, New York, New York [doi>10.3115/1220835.1220845]
• 18: John D. Lafferty , Andrew McCallum , Fernando C. N. Pereira, Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data, Proceedings of the Eighteenth International Conference on Machine Learning, p.282-289, June 28-July 01, 2001
• 19: R. F. Murphy, Z. Kou, J. Hua, M. Joffe, and W. W. Cohen. Extracting and structuring subcellular location information from on-line journal articles: The subcellular location image finder. In KSCE, 2004.
• 20: National Institues of Health. http://www.pubmedcentral.nih.gov/.
• 21: Tomoko Ohta , Yuka Tateisi , Jin-Dong Kim, The GENIA corpus: an annotated research abstract corpus in molecular biology domain, Proceedings of the second international conference on Human Language Technology Research, March 24-27, 2002, San Diego, California
• 22: Bernhard Schölkopf , Florian Steinke , Volker Blanz, Object correspondence as a machine learning problem, Proceedings of the 22nd international conference on Machine learning, p.776-783, August 07-11, 2005, Bonn, Germany [doi>10.1145/1102351.1102449]
• 23: L. Shi and F. Campagne. Building a protein name dictionary from full text: a machine learning term extraction approach. BMC Bioinformatics, 6(88), 2005.
• 24: Charles Sutton , Andrew McCallum, Composition of conditional random fields for transfer learning, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, p.748-754, October 06-08, 2005, Vancouver, British Columbia, Canada [doi>10.3115/1220575.1220669]
• 25: B. Taskar, M.-F. Wong, and D. Koller. Learning on the test data: Leveraging 'unseen' features. In Proc. Twentieth International Conference on Machine Learning (ICML), 2003.
• 26: S. Thrun. Is learning the n-th thing any easier than learning the first? In NIPS, volume 8, pages 640--646. MIT, 1996.
• 27: J. Zhang, Z. Ghahramani, and Y. Yang. Learning multiple related tasks using latent independent component analysis, 2005.
• 28: X. Zhu. Semi-supervised learning literature survey. In Technical Report 1530. University of Wisconsin, 2005.
• 29: X. Zhu and Z. Ghahramani. Learning from labeled and unlabeled data with label propagation. Technical Report CMU-CALD-02-107, Carnegie Mellon University, 2002

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.m Miscellaneous

Additional Classification:
  J. Computer Applications
  J.3 LIFE AND MEDICAL SCIENCES

General Terms:
Algorithms

Keywords:
ir_content::structured, km_information_extraction, km_statistical_techniques, km_text_mining, meta data, semi structured, social tagging