ACM Portal
  Subscribe (Full Service)    Register (Limited Service, Free)    Login
  Search:   The ACM Digital Library   The Guide
  
ACM Home Page
Please provide us with feedback. Feedback
Digital Library logoTake a look at the new version of this page: [ beta version ]. Tell us what you think.
A model-theoretic coreference scoring scheme
Full text Publisher SitePublisher Site ,  PdfPdf (369 KB)
Source Message Understanding Conference archive
Proceedings of the 6th conference on Message understanding table of contents
Columbia, Maryland
SESSION: Evaluation table of contents
Pages: 45 - 52  
Year of Publication: 1995
ISBN:1-55860-402-2
Authors
Marc Vilain  The MITRE Corporation, Bedford, MA
John Burger  The MITRE Corporation, Bedford, MA
John Aberdeen  The MITRE Corporation, Bedford, MA
Dennis Connolly  The MITRE Corporation, Bedford, MA
Lynette Hirschman  The MITRE Corporation, Bedford, MA
Publisher
Association for Computational Linguistics  Morristown, NJ, USA
Bibliometrics
Downloads (6 Weeks): 2,   Downloads (12 Months): 36,   Citation Count: 61
Additional Information:

abstract   cited by   collaborative colleagues  

Tools and Actions: Review this Article  
DOI Bookmark: 10.3115/1072399.1072405

ABSTRACT

This note describes a scoring scheme for the coreference task in MUC6. It improves on the original approach by: (1) grounding the scoring scheme in terms of a model; (2) producing more intuitive recall and precision scores; and (3) not requiring explicit computation of the transitive closure of coreference. The principal conceptual difference is that we have moved from a syntactic scoring model based on following coreference links to an approach defined by the model theory of those links.


CITED BY  61
Ralph Grishman , Beth Sundheim, Design of the MUC-6 evaluation, Proceedings of a workshop on held at Vienna, Virginia: May 6-8, 1996, May 06-08, 1996, Vienna, Virginia
Saliha Azzam , Kevin Humphreys , Robert Gaizauskas, Evaluating a focus-based approach to anaphora resolution, Proceedings of the 36th annual meeting on Association for Computational Linguistics, August 10-14, 1998, Montreal, Quebec, Canada
Andrei Popescu-Belis , Isabelle Robba , Gérard Sabah, Reference resolution beyond coreference: a conceptual frame and its application, Proceedings of the 17th international conference on Computational linguistics, p.1046-1052, August 10-14, 1998, Montreal, Quebec, Canada
Renata Vieira , Massimo Poesio, An empirically based system for processing definite descriptions, Computational Linguistics, v.26 n.4, p.539-593, December 2000
Amit Bagga , Breck Baldwin, Entity-based cross-document coreferencing using the Vector Space Model, Proceedings of the 17th international conference on Computational linguistics, August 10-14, 1998, Montreal, Quebec, Canada
Roland Stuckardt, Design and enhanced evaluation of a robust anaphor resolution algorithm, Computational Linguistics, v.27 n.4, December 2001
Lynette Hirschman , Marc Light , Eric Breck , John D. Burger, Deep Read: a reading comprehension system, Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, p.325-332, June 20-26, 1999, College Park, Maryland
Ralph Grishman , Beth Sundheim, Design of the MUC-6 evaluation, Proceedings of the 6th conference on Message understanding, November 06-08, 1995, Columbia, Maryland
Sabine Schulte im Walde , Chris Brew, Inducing German semantic verb classes from purely syntactic subcategorisation information, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania
Sanda M. Harabagiu , Rǎzvan C. Bunescu , Steven J. Maiorano, Text and knowledge mining for coreference resolution, Second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies 2001, p.1-8, June 01-07, 2001, Pittsburgh, Pennsylvania
Vincent Ng , Claire Cardie, Weakly supervised natural language learning without redundant views, Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, p.94-101, May 27-June 01, 2003, Edmonton, Canada
Thomas Finley , Thorsten Joachims, Supervised clustering with support vector machines, Proceedings of the 22nd international conference on Machine learning, p.217-224, August 07-11, 2005, Bonn, Germany
Christoph Müller , Michael Strube, Annotating anaphoric and bridging relations with MMAX, Proceedings of the Second SIGdial Workshop on Discourse and Dialogue, p.1-6, September 01-02, 2001, Aalborg, Denmark
Vincent Ng , Claire Cardie, Combining sample selection and error-driven pruning for machine learning of coreference rules, Proceedings of the ACL-02 conference on Empirical methods in natural language processing, p.55-62, July 06, 2002
Michael Strube , Christoph Müller, A machine learning approach to pronoun resolution in spoken dialogue, Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, p.168-175, July 07-12, 2003, Sapporo, Japan
Thomas S. Morton, Coreference for NLP applications, Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, p.173-180, October 03-06, 2000, Hong Kong
Andrea Setzer , Robert Gaizauskas, A pilot study on annotating temporal relations in text, Proceedings of the workshop on Temporal and spatial information processing, p.1-8, July 07-07, 2001
Vincent Ng , Claire Cardie, Improving machine learning approaches to coreference resolution, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania
Christoph Müller , Stefan Rapp , Michael Strube, Applying Co-Training to reference resolution, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania
Vincent Ng , Claire Cardie, Bootstrapping coreference classifiers with multiple machine learning algorithms, Proceedings of the 2003 conference on Empirical methods in natural language processing, p.113-120, July 11, 2003
Sven Hartrumpf, Coreference resolution with syntactico-semantic rules and corpus statistics, Proceedings of the 2001 workshop on Computational Natural Language Learning, p.1-8, July 06-07, 2001, Toulouse, France
Michael Strube , Stefan Rapp , Christoph Müller, The influence of minimum edit distance on reference resolution, Proceedings of the ACL-02 conference on Empirical methods in natural language processing, p.312-319, July 06, 2002
Vincent Ng , Claire Cardie, Identifying anaphoric and non-anaphoric noun phrases to improve coreference resolution, Proceedings of the 19th international conference on Computational linguistics, p.1-7, August 24-September 01, 2002, Taipei, Taiwan
Nancy A. Chinchor, MUC/MET evaluation trends, Proceedings of a workshop on held at Baltimore, Maryland: October 13-15, 1998, October 13-15, 1998, Baltimore, Maryland
Xiaofeng Yang , Guodong Zhou , Jian Su , Chew Lim Tan, Coreference resolution using competition learning approach, Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, p.176-183, July 07-12, 2003, Sapporo, Japan
Regina Barzilay , Mirella Lapata, Modeling local coherence: An entity-based approach, Computational Linguistics, v.34 n.1, p.1-34, March 2008
Indra Budi , Stephane Bressan, Application of association rules mining to Named Entity Recognition and co-reference resolution for the Indonesian language, International Journal of Business Intelligence and Data Mining, v.2 n.4, p.426-446, December 2007
Vincent Ng, Learning noun phrase anaphoricity to improve coreference resolution: issues in representation and optimization, Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p.151-es, July 21-26, 2004, Barcelona, Spain
Xiaofeng Yangy , Jian Su , Guodong Zhou , Chew Lim Tan, An NP-cluster based approach to coreference resolution, Proceedings of the 20th international conference on Computational Linguistics, p.226-es, August 23-27, 2004, Geneva, Switzerland
Xiaoqiang Luo , Abe Ittycheriah , Hongyan Jing , Nanda Kambhatla , Salim Roukos, A mention-synchronous coreference resolution algorithm based on the Bell tree, Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p.135-es, July 21-26, 2004, Barcelona, Spain
Vincent Ng, Machine learning for coreference resolution: from local classification to global ranking, Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, p.157-164, June 25-30, 2005, Ann Arbor, Michigan
Staff, Book Reviews, Computational Linguistics, v.29 n.4, p.655-664, December 2003
Xiaoqiang Luo, On coreference resolution performance metrics, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, p.25-32, October 06-08, 2005, Vancouver, British Columbia, Canada
Xiaoqiang Luo , Imed Zitouni, Multi-lingual coreference resolution with syntactic features, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, p.660-667, October 06-08, 2005, Vancouver, British Columbia, Canada
Xiaofeng Yang , Jian Su , Guodong Zhou , Chew Lim Tan, Improving pronoun resolution by incorporating coreferential information of candidates, Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p.127-es, July 21-26, 2004, Barcelona, Spain
Heng Ji , David Westbrook , Ralph Grishman, Using semantic relations to refine coreference decisions, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, p.17-24, October 06-08, 2005, Vancouver, British Columbia, Canada
Art Munson , Claire Cardie , Rich Caruana, Optimizing to arbitrary NLP metrics using ensemble selection, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, p.539-546, October 06-08, 2005, Vancouver, British Columbia, Canada
Simone Paolo Ponzetto , Michael Strube, Exploiting semantic role labeling, WordNet and Wikipedia for coreference resolution, Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, p.192-199, June 04-09, 2006, New York, New York
Andrei Popescu-Belis, Evaluation-driven design of a robust coreference resolution system, Natural Language Engineering, v.9 n.3, p.281-306, September 2003
Xiaofeng Yang , Jian Su , Chew Lim Tan, A twin-candidate model for learning-based anaphora resolution, Computational Linguistics, v.34 n.3, p.327-356, September 2008
Ron Artstein , Massimo Poesio, Inter-coder agreement for computational linguistics, Computational Linguistics, v.34 n.4, p.555-596, December 2008
Simone Paolo Ponzetto , Michael Strube, Semantic role labeling for coreference resolution, Proceedings of the Eleventh Conference of the European Chapter of the Association for Computational Linguistics: Posters & Demonstrations, April 05-06, 2006, Trento, Italy
Jun Zhao , Fei-Fan Liu, Linguistic theory based contextual evidence mining for statistical chinese co-reference resolution, Journal of Computer Science and Technology, v.22 n.4, p.608-617, July 2007
Michael Strube , Simone Paolo Ponzetto, WikiRelate! computing semantic relatedness using wikipedia, proceedings of the 21st national conference on Artificial intelligence, p.1419-1424, July 16-20, 2006, Boston, Massachusetts
Jenny Rose Finkel , Christopher D. Manning, Enforcing transitivity in coreference resolution, Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers, June 16-17, 2008, Columbus, Ohio
Chun-Nam John Yu , Thorsten Joachims, Learning structural SVMs with latent variables, Proceedings of the 26th Annual International Conference on Machine Learning, p.1169-1176, June 14-18, 2009, Montreal, Quebec, Canada
Simone Paolo Ponzetto , Michael Strube, Knowledge derived from wikipedia for computing semantic relatedness, Journal of Artificial Intelligence Research, v.30 n.1, p.181-212, September 2007
Jacob Eisenstein , Regina Barzilay , Randall Davis, Gesture salience as a hidden variable for coreference resolution and keyframe extraction, Journal of Artificial Intelligence Research, v.31 n.1, p.353-398, January 2008
Vincent Ng, Shallow semantics for coreference resolution, Proceedings of the 20th international joint conference on Artifical intelligence, p.1689-1694, January 06-12, 2007, Hyderabad, India
Veselin Stoyanov , Claire Cardie, Toward opinion summarization: linking the sources, Proceedings of the Workshop on Sentiment and Subjectivity in Text, p.9-14, July 22-22, 2006, Sydney, Australia
Pascal Denis , Jason Baldridge, Specialized models and ranking for coreference resolution, Proceedings of the Conference on Empirical Methods in Natural Language Processing, October 25-27, 2008, Honolulu, Hawaii
Ryu Iida , Mamoru Komachi , Kentaro Inui , Yuji Matsumoto, Annotating a Japanese text corpus with predicate-argument and coreference relations, Proceedings of the Linguistic Annotation Workshop, p.132-139, June 28-29, 2007, Prague, Czech Republic
Eric Bengtson , Dan Roth, Understanding the value of features for coreference resolution, Proceedings of the Conference on Empirical Methods in Natural Language Processing, October 25-27, 2008, Honolulu, Hawaii
Vincent Ng, Graph-cut-based anaphoricity determination for coreference resolution, Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, May 31-June 05, 2009, Boulder, Colorado
Marta Recasens , Toni Martí , Mariona Taulé , Lluís Màrquez , Emili Sapena, SemEval-2010 task 1: coreference resolution in multiple languages, Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions, June 04-04, 2009, Boulder, Colorado
Vincent Ng, Unsupervised models for coreference resolution, Proceedings of the Conference on Empirical Methods in Natural Language Processing, October 25-27, 2008, Honolulu, Hawaii
Hoifung Poon , Pedro Domingos, Joint unsupervised coreference resolution with Markov logic, Proceedings of the Conference on Empirical Methods in Natural Language Processing, October 25-27, 2008, Honolulu, Hawaii
Veselin Stoyanov , Claire Cardie, Topic identification for fine-grained opinion analysis, Proceedings of the 22nd International Conference on Computational Linguistics, p.817-824, August 18-22, 2008, Manchester, United Kingdom
Jörg Tiedemann , Jori Mur, Simple is best: experiments with different document segmentation strategies for passage retrieval, Coling 2008: Proceedings of the 2nd workshop on Information Retrieval for Question Answering, p.17-25, August 24-24, 2008, Manchester, UK
Cristina Nicolae , Gabriel Nicolae, BestCut: a graph algorithm for coreference resolution, Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, July 22-23, 2006, Sydney, Australia
Cláudia Freitas , Diana Santos , Cristina Mota , Hugo Gonçalo Oliveira , Paula Carvalho, Relation detection between named entities: report of a shared task, Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions, June 04-04, 2009, Boulder, Colorado
Collaborative Colleagues:
Marc Vilain: colleagues
John Burger: colleagues
John Aberdeen: colleagues
Dennis Connolly: colleagues
Lynette Hirschman: colleagues

The ACM Portal is published by the Association for Computing Machinery. Copyright © 2010 ACM, Inc.
Terms of Usage   Privacy Policy   Code of Ethics   Contact Us

Useful downloads: Adobe Acrobat    QuickTime    Windows Media Player    Real Player