Hierarchical, perceptron-like learning for ontology-based information extraction

ABSTRACT

Recent work on ontology-based Information Extraction (IE) has tried to make use of knowledge from the target ontology in order to improve semantic annotation results. However, very few approaches exploit the ontology structure itself, and those that do so, have some limitations. This paper introduces a hierarchical learning approach for IE, which uses the target ontology as an essential part of the extraction process, by taking into account the relations between concepts. The approach is evaluated on the largest available semantically annotated corpus. The results demonstrate clearly the benefits of using knowledge from the ontology as input to the information extraction process. We also demonstrate the advantages of our approach over other state-of-the-art learning systems on a commonly used benchmark dataset.

REFERENCES

• 1: Mary Elaine Califf , Raymond J. Mooney, Relational learning techniques for natural language information extraction, The University of Texas at Austin, 1998
• 2: N. Cesa-Bianchi, C. Gentile, A. Tironi, and L. Zaniboni. Incremental Algorithms for Hierarchical Classification. In Neural Information Processing Systems, 2004.
• 3: Hai Leong Chieu , Hwee Tou Ng, A maximum entropy approach to information extraction from semi-structured and free text, Eighteenth national conference on Artificial intelligence, p.786-791, July 28-August 01, 2002, Edmonton, Alberta, Canada
• 4: Nancy Chinchor, MUC-4 evaluation metrics, Proceedings of the 4th conference on Message understanding, June 16-18, 1992, McLean, Virginia [doi>10.3115/1072064.1072067]
• 5: Philipp Cimiano , Siegfried Handschuh , Steffen Staab, Towards the self-annotating web, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988735]
• 6: F. Ciravegna and Y. Wilks. Designing Adaptive Information Extraction for the Semantic Web in Amilcare. In S. Handschuh and S. Staab, editors, Annotation for the Semantic Web. IOS Press, Amsterdam, 2003.
• 7: H. Cunningham, D. Maynard, K. Bontcheva, and V. Tablan. GATE: A Framework and Graphical Development Environment for Robust NLP Tools and Applications. In Proceedings of the 40th Anniversary Meeting of the Association for Computational Linguistics (ACL'02), 2002.
• 8: Ofer Dekel , Joseph Keshet , Yoram Singer, Large margin hierarchical classification, Proceedings of the twenty-first international conference on Machine learning, p.27, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015374]
• 9: Stephen Dill , Nadav Eiron , David Gibson , Daniel Gruhl , R. Guha , Anant Jhingran , Tapas Kanungo , Sridhar Rajagopalan , Andrew Tomkins , John A. Tomlin , Jason Y. Zien, SemTag and seeker: bootstrapping the semantic web via automated semantic annotation, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775178]
• 10: John Domingue , Martin Dzbor, Magpie: supporting browsing and navigation on the semantic web, Proceedings of the 9th international conference on Intelligent user interfaces, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964479]
• 11: D. Freigtag and A. K. McCallum. Information Extraction with HMMs and Shrinkage. In Proceesings of Workshop on Machine Learnig for Information Extraction, pages 31--36, 1999.
• 12: Dayne Freitag, Machine Learning for Information Extraction in Informal Domains, Machine Learning, v.39 n.2-3, p.169-202, May-June 2000 [doi>10.1023/A:1007601113994]
• 13: Dayne Freitag , Nicholas Kushmerick, Boosted Wrapper Induction, Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence, p.577-583, July 30-August 03, 2000
• 14: Siegfried Handschuh , Steffen Staab , Fabio Ciravegna, S-CREAM - Semi-automatic CREAtion of Metadata, Proceedings of the 13th International Conference on Knowledge Engineering and Knowledge Management. Ontologies and the Semantic Web, p.358-372, October 01-04, 2002
• 15: A. Kiryakov, B. Popov, D. Ognyanoff, D. Manov, A. Kirilov, and M. Goranov. Semantic annotation, indexing and retrieval. Journal of Web Semantics, ISWC 2003 Special Issue, 1(2):671--680, 2004.
• 16: P. Kogut and W. Holmes. AeroDAML: Applying Information Extraction to Generate DAML Annotations from Web Pages. In First International Conference on Knowledge Capture (K-CAP 2001), Workshop on Knowledge Markup and Semantic Annotation, Victoria, B.C., 2001.
• 17: Y. Li, K. Bontcheva, and H. Cunningham. Using Uneven Margins SVM and Perceptron for Information Extraction. In Proceedings of Ninth Conference on Computational Natural Language Learning (CoNLL-2005), 2005.
• 18: Yaoyong Li , Hugo Zaragoza , Ralf Herbrich , John Shawe-Taylor , Jaz S. Kandola, The Perceptron Algorithm with Uneven Margins, Proceedings of the Nineteenth International Conference on Machine Learning, p.379-386, July 08-12, 2002
• 19: D. Maynard, W. Peters, and Y. Li. Metrics for evaluation of ontology-based information extraction. In WWW 2006 Workshop on "Evaluation of Ontologies for the Web" (EON), Edinburgh, Scotland, 2006.
• 20: L. K. McDowell and M. Cafarella. Ontology-Driven Information Extraction with OntoSyphon. In 5th Internal Semantic Web Conference (ISWC'06). Springer, 2006.
• 21: Maria Vargas-Vera , Enrico Motta , John Domingue , Mattia Lanzoni , Arthur Stutt , Fabio Ciravegna, MnM: Ontology Driven Semi-automatic and Automatic Support for Semantic Markup, Proceedings of the 13th International Conference on Knowledge Engineering and Knowledge Management. Ontologies and the Semantic Web, p.379-391, October 01-04, 2002
• 22: J. Perna and A. Spector. Introduction to the Special Issue on Unstructured Information Management. IBM Systems Journal, 43(3), 2004.
• 23: P. Resnik. Using information content to evaluate semantic similarity in a taxonomy. In Proc. of 14th International Joint Conference on Artificial Intelligence, pages 448--453, Montreal, Canada, 1995.
• 24: D. Roth and W. T. Yih. Relational Learning via Propositional Algorithms: An Information Extraction Case Study. In Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI), pages 1257--1263, 2001.
• 25: Juho Rousu , Craig Saunders , Sandor Szedmak , John Shawe-Taylor, Kernel-Based Learning of Hierarchical Multilabel Classification Models, The Journal of Machine Learning Research, 7, p.1601-1626, 12/1/2006

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Classifier design and evaluation

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Text analysis

General Terms:
Algorithms, Performance

Keywords:
hierarchical learning, ontology-based information extraction, semantic annotation