Using a Hybrid Convolution Tree Kernel for Semantic Role Labeling

ABSTRACT

As a kind of Shallow Semantic Parsing, Semantic Role Labeling (SRL) is gaining more attention as it benefits a wide range of natural language processing applications. Given a sentence, the task of SRL is to recognize semantic arguments (roles) for each predicate (target verb or noun). Feature-based methods have achieved much success in SRL and are regarded as the state-of-the-art methods for SRL. However, these methods are less effective in modeling structured features. As an extension of feature-based methods, kernel-based methods are able to capture structured features more efficiently in a much higher dimension. Application of kernel methods to SRL has been achieved by selecting the tree portion of a predicate and one of its arguments as feature space, which is named as predicate-argument feature (PAF) kernel. The PAF kernel captures the syntactic tree structure features using convolution tree kernel, however, it does not distinguish between the path structure and the constituent structure. In this article, a hybrid convolution tree kernel is proposed to model different linguistic objects. The hybrid convolution tree kernel consists of two individual convolution tree kernels. They are a Path kernel, which captures predicate-argument link features, and a Constituent Structure kernel, which captures the syntactic structure features of arguments. Evaluations on the data sets of the CoNLL-2005 SRL shared task and the Chinese PropBank (CPB) show that our proposed hybrid convolution tree kernel statistically significantly outperforms the previous tree kernels. Moreover, in order to maximize the system performance, we present a composite kernel through combining our hybrid convolution tree kernel method with a feature-based method extended by the polynomial kernel. The experimental results show that the composite kernel achieves better performance than each of the individual methods and outperforms the best reported system on the CoNLL-2005 corpus when only one syntactic parser is used and on the CPB corpus when automated syntactic parse results and correct syntactic parse results are used respectively.

REFERENCES

• 1: Collin F. Baker , Charles J. Fillmore , John B. Lowe, The Berkeley FrameNet Project, Proceedings of the 17th international conference on Computational linguistics, August 10-14, 1998, Montreal, Quebec, Canada
• 2: Adam L. Berger , Vincent J. Della Pietra , Stephen A. Della Pietra, A maximum entropy approach to natural language processing, Computational Linguistics, v.22 n.1, p.39-71, March 1996
• 3: Carreras, X. and Màrquez, L. 2004. Introduction to the CoNLL-2004 shared task: Semantic role labeling. In Proceedings of the 8th Conference on Natural Language Learning (CoNLL’04). 89--97.
• 4: Carreras, X. and Màrquez, L. 2005. Introduction to the CoNLL-2005 shared task: Semantic role labeling. In Proceedings of the 9th Conference on Natural Language Learning (CoNLL’05). 152--164.
• 5: Eugene Charniak, A maximum-entropy-inspired parser, Proceedings of the first conference on North American chapter of the Association for Computational Linguistics, p.132-139, April 29-May 04, 2000, Seattle, Washington
• 6: Wanxiang Che , Min Zhang , Ting Liu , Sheng Li, A hybrid convolution tree kernel for semantic role labeling, Proceedings of the COLING/ACL on Main conference poster sessions, p.73-80, July 17-18, 2006, Sydney, Australia
• 7: Hai Leong Chieu , Hwee Tou Ng, Named entity recognition with a maximum entropy approach, Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003, p.160-163, May 31, 2003, Edmonton, Canada [doi>10.3115/1119176.1119199]
• 8: Michael John Collins , Mitchell P. Marcus, Head-driven statistical models for natural language parsing, University of Pennsylvania, Philadelphia, PA, 1999
• 9: Collins, M. and Duffy, N. 2001. Convolution kernels for natural language. In Proceedings of the 15th Annual Conference on Neutral Information Processing Systems (NIPS’01).
• 10: Nello Cristianini , John Shawe-Taylor, An introduction to support Vector Machines: and other kernel-based learning methods, Cambridge University Press, New York, NY, 1999
• 11: Aron Culotta , Jeffrey Sorensen, Dependency tree kernels for relation extraction, Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p.423-es, July 21-26, 2004, Barcelona, Spain [doi>10.3115/1218955.1219009]
• 12: Daniel Gildea , Daniel Jurafsky, Automatic labeling of semantic roles, Computational Linguistics, v.28 n.3, p.245-288, September 2002 [doi>10.1162/089120102760275983]
• 13: Daniel Gildea , Martha Palmer, The necessity of parsing for predicate argument recognition, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073124]
• 14: Gimenez, J. and Màrquez, L. 2003. Fast and accurate part-of-speech tagging: The svm approach revisited. In Proceedings of the International Conference on Recent Advances in Natural Language (RANLP’03).
• 15: Haussler, D. 1999. Convolution kernels on discrete structures. Tech. Rep. UCSC-CRL-99-10. July.
• 16: Jiang, Z. P., Li, J., and Ng, H. T. 2005. Semantic argument classification exploiting argument interdependence. In Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI’05).
• 17: Thorsten Joachims, Learning to Classify Text Using Support Vector Machines: Methods, Theory and Algorithms, Kluwer Academic Publishers, Norwell, MA, 2002
• 18: Thorsten Joachims , Nello Cristianini , John Shawe-Taylor, Composite Kernels for Hypertext Categorisation, Proceedings of the Eighteenth International Conference on Machine Learning, p.250-257, June 28-July 01, 2001
• 19: Liu, T., Che, W., Li, S., Hu, Y., and Liu, H. 2005. Semantic role labeling system using maximum entropy classifier. In Proceedings of the 9th Conference on Natural Language Learning (CoNLL’05). 189--192.
• 20: Huma Lodhi , Craig Saunders , John Shawe-Taylor , Nello Cristianini , Chris Watkins, Text classification using string kernels, The Journal of Machine Learning Research, 2, p.419-444, 3/1/2002 [doi>10.1162/153244302760200687]
• 21: Mitchell P. Marcus , Mary Ann Marcinkiewicz , Beatrice Santorini, Building a large annotated corpus of English: the penn treebank, Computational Linguistics, v.19 n.2, June 1993
• 22: Alessandro Moschitti, A study on convolution kernels for shallow semantic parsing, Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p.335-es, July 21-26, 2004, Barcelona, Spain [doi>10.3115/1218955.1218998]
• 23: Moschitti, A., Pighin, D., and Basili, R. 2006. Tree kernel engineering in semantic role labeling systems. In Proceedings of the Workshop on Learning Structured Information for Natural Language Applications, 11th International Conference on European Association for Computational Linguistics (EACL’06). Trento, Italy, 49--56.
• 24: Alessandro Moschitti , Daniele Pighin , Roberto Basili, Tree kernels for semantic role labeling, Computational Linguistics, v.34 n.2, p.193-224, June 2008 [doi>10.1162/coli.2008.34.2.193]
• 25: Moschitti, A., Quarteroni, S., Basili, R., and Manandhar, S. 2006. Exploiting syntactic and shallow semantic kernels for question answer classification. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics (ACL’07). Prague, Czech Republic, 776--783.
• 26: Srini Narayanan , Sanda Harabagiu, Question answering based on semantic structures, Proceedings of the 20th international conference on Computational Linguistics, p.693-es, August 23-27, 2004, Geneva, Switzerland [doi>10.3115/1220355.1220455]
• 27: Nielsen, R. D. and Pradhan, S. 2004. Mixing weak learners in semantic parsing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’04).
• 28: Martha Palmer , Daniel Gildea , Paul Kingsbury, The Proposition Bank: An Annotated Corpus of Semantic Roles, Computational Linguistics, v.31 n.1, p.71-106, March 2005 [doi>10.1162/0891201053630264]
• 29: Simone Paolo Ponzetto , Michael Strube, Exploiting semantic role labeling, WordNet and Wikipedia for coreference resolution, Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, p.192-199, June 04-09, 2006, New York, New York [doi>10.3115/1220835.1220860]
• 30: Sameer Pradhan , Kadri Hacioglu , Valerie Krugler , Wayne Ward , James H. Martin , Daniel Jurafsky, Support Vector Learning for Semantic Argument Classification, Machine Learning, v.60 n.1-3, p.11-39, September 2005 [doi>10.1007/s10994-005-0912-2]
• 31: Sameer Pradhan , Wayne Ward , Kadri Hacioglu , James H. Martin , Daniel Jurafsky, Semantic role labeling using different syntactic views, Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, p.581-588, June 25-30, 2005, Ann Arbor, Michigan [doi>10.3115/1219840.1219912]
• 32: Punyakanok, V., Roth, D., and Tau Yih, W. 2005. The necessity of syntactic parsing for semantic role labeling. In Proceedings of 19th International Joint Conference on Artificial Intelligence (IJCAI’05). 1117--1123.
• 33: Vasin Punyakanok , Dan Roth , Wen-tau Yih , Dav Zimak, Semantic role labeling via integer linear programming inference, Proceedings of the 20th international conference on Computational Linguistics, p.1346-es, August 23-27, 2004, Geneva, Switzerland [doi>10.3115/1220355.1220552]
• 34: Ryan Rifkin , Aldebaro Klautau, In Defense of One-Vs-All Classification, The Journal of Machine Learning Research, 5, p.101-141, 12/1/2004
• 35: Robert E. Schapire , Yoram Singer, Improved Boosting Algorithms Using Confidence-rated Predictions, Machine Learning, v.37 n.3, p.297-336, Dec. 1999 [doi>10.1023/A:1007614523901]
• 36: John Shawe-Taylor , Nello Cristianini, Kernel Methods for Pattern Analysis, Cambridge University Press, New York, NY, 2004
• 37: Shen, D. and Lapata, M. 2007. Using semantic roles to improve question answering. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL’07). 12--21.
• 38: Sun, H. and Jurafsky, D. 2004. Shallow semantic parsing of chinese. In Proceedings of the Joint Human Language Technology Conference/Annual Meeting of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL’04).
• 39: Mihai Surdeanu , Sanda Harabagiu , John Williams , Paul Aarseth, Using predicate-argument structures for information extraction, Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, p.8-15, July 07-12, 2003, Sapporo, Japan [doi>10.3115/1075096.1075098]
• 40: Surdeanu, M. and Turmo, J. 2005. Semantic role labeling using complete syntactic analysis. In Proceedings of the 9th Conference on Natural Language Learning (CoNLL’05). Ann Arbor, Michigan.
• 41: Vapnik, V. N. 1998. Statistical learning theory. Wiley.
• 42: Watkins, C. 1999. Dynamic alignment kernels. Tech. Rep. CSD-TR-98-11. January.
• 43: Xue, N. and Kulick, S. 2003. Automatic predicate argument structure analysis of the penn Chinese treebank. In Proceedings of the 10th Machine Translation Summit (MT Summit X).
• 44: Xue, N. and Palmer, M. 2004. Calibrating features for semantic role labeling. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’04).
• 45: Xue, N. and Palmer, M. 2005. Automatic semantic role labeling for Chinese verbs. In Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI’05).
• 46: Naiwen Xue , Fei Xia , Fu-dong Chiou , Marta Palmer, The Penn Chinese TreeBank: Phrase structure annotation of a large corpus, Natural Language Engineering, v.11 n.2, p.207-238, June 2005 [doi>10.1017/S135132490400364X]
• 47: Dmitry Zelenko , Chinatsu Aone , Anthony Richardella, Kernel methods for relation extraction, The Journal of Machine Learning Research, 3, 3/1/2003
• 48: Min Zhang , Jie Zhang , Jian Su, Exploring syntactic features for relation extraction using a convolution tree kernel, Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, p.288-295, June 04-09, 2006, New York, New York [doi>10.3115/1220835.1220872]
• 49: Min Zhang , Jie Zhang , Jian Su , Guodong Zhou, A composite kernel to extract relations between entities with both flat and structured features, Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, p.825-832, July 17-18, 2006, Sydney, Australia [doi>10.3115/1220175.1220279]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.1 Models
      I.5.4 Applications
          Subjects: Text processing

General Terms:
Documentation, Languages

Keywords:
Semantic role labeling, hybrid convolution tree kernel