MUC-5 evaluation metrics

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

MUC-5 evaluation metrics

Full text

Publisher Site , Pdf

Pdf (802 KB)

Source

Message Understanding Conference archive
Proceedings of the 5th conference on Message understanding table of contents

Baltimore, Maryland

SESSION: Evaluation table of contents

Pages: 69 - 78

Year of Publication: 1993

ISBN:1-55860-336-0

Authors

Nancy Chinchor	Science Applications International Corporation, San Diego, CA
Beth Sundheim	Naval Command, Control, and Ocean Surveillance Center, San Diego, CA

Publisher

Association for Computational Linguistics Morristown, NJ, USA

Bibliometrics

Downloads (6 Weeks): 4, Downloads (12 Months): 27, Citation Count: 13

Additional Information:

abstract references cited by collaborative colleagues

Tools and Actions:	Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	10.3115/1072017.1072026

ABSTRACT

The metrics used for the Fifth Message Understanding Conference (MUC-5) evaluation are a major update to those used for MUC-4 in 1992. The official MUC-5 metrics express error rates while the official MUC-4 metrics express performance in terms of recall and precision (used for MUC-5 only as "unofficial" metrics). This paper discusses the current metrics and the reasons for their adoption.

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	William B. Frakes , Ricardo Baeza-Yates, Information retrieval: data structures and algorithms, Prentice-Hall, Inc., Upper Saddle River, NJ, 1992
	2	C. J. Van Rijsbergen, Information Retrieval, Butterworth-Heinemann, Newton, MA, 1979
	3	Oscar Nierstrasz, A survey of object-oriented concepts, Object-oriented concepts, databases, and applications, ACM Press, New York, NY, 1989 [doi>10.1145/63320.66468]

CITED BY 13

	Scott B. Huffman, Learning to extract information from text based on user-provided examples, Proceedings of the fifth international conference on Information and knowledge management, p.154-163, November 12-16, 1996, Rockville, Maryland, United States
	David D. Palmer , David S. Day, A statistical profile of the Named Entity task, Proceedings of the fifth conference on Applied natural language processing, p.190-193, March 31-April 03, 1997, Washington, DC
	Kamal Kumar , Mathew J. Palakal , Snehasis Mukhopadhyay , Mathew J. Stephens , Huian Li, BioMap: toward the development of a knowledge base of biomedical literature, Proceedings of the 2004 ACM symposium on Applied computing, March 14-17, 2004, Nicosia, Cyprus
	Tsuyoshi Kitani , Yoshio Eriguchi , Masami Hara, Pattern matching in the TEXTRACT information extraction system, Proceedings of the 15th conference on Computational linguistics, August 05-09, 1994, Kyoto, Japan
	John D. Burger , David Palmer , Lynette Hirschman, Named Entity scoring for speech input, Proceedings of the 17th international conference on Computational linguistics, August 10-14, 1998, Montreal, Quebec, Canada
	Lynette Hirschman , Marc Light , Eric Breck , John D. Burger, Deep Read: a reading comprehension system, Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, p.325-332, June 20-26, 1999, College Park, Maryland
	Craig A. Will, Comparing human and machine performance for natural language information extraction: results for English microelectronics from the MUC-5 evaluation, Proceedings of the 5th conference on Message understanding, August 25-27, 1993, Baltimore, Maryland
	Beth M. Sundheim, Tipster/MUC-5 information extraction system evaluation, Proceedings of a workshop on held at Fredericksburg, Virginia: September 19-23, 1993, September 19-23, 1993, Fredericksburg, Virginia
	Chikashi Nobata , Nigel Collier , Jun'ichi Tsujii, Comparison between tagged corpora for the named entity task, Proceedings of the workshop on Comparing corpora, p.20-27, October 07-07, 2000, Hong Kong
	Craig A. Will, Comparing human and machine performance for natural language information extraction: results from the Tipster text evaluation, Proceedings of a workshop on held at Fredericksburg, Virginia: September 19-23, 1993, September 19-23, 1993, Fredericksburg, Virginia
	Marc Vilain , David Day, Finite-state phrase parsing by rule sequences, Proceedings of the 16th conference on Computational linguistics, August 05-09, 1996, Copenhagen, Denmark
	Tsuyoshi Kitani , Yoshio Eriguchi , Masami Hara, Pattern matching and discourse processing in information extraction from Japanese text, Journal of Artificial Intelligence Research, v.2 n.1, p.89-110, August 1994
	Joseph F. McCarthy , Wendy G. Lehnert, Using decision trees for conference resolution, Proceedings of the 14th international joint conference on Artificial intelligence, p.1050-1055, August 20-25, 1995, Montreal, Quebec, Canada

Collaborative Colleagues:

Nancy Chinchor: colleagues

Beth Sundheim: colleagues

Adobe Acrobat

QuickTime

Windows Media Player

Real Player