Unsupervised learning of soft patterns for generating definitions from online news

ABSTRACT

Breaking news often contains timely definitions and descriptions of current terms, organizations and personalities. We utilize such web sources to construct definitions for such terms. Previous work has identified definitions using hand-crafted rules or supervised learning that constructs rigid, hard text patterns. In contrast, we demonstrate a new approach that uses flexible, soft matching patterns to characterize definition sentences. Our soft patterns are able to effectively accommodate the diversity of definition sentence structure exhibited in news. We use pseudo-relevance feedback to automatically label sentences for use in soft pattern generation. The application of our unsupervised method significantly improves baseline systems on both the standardized TREC corpus as well as crawled online news articles by 27% and 30%, respectively, in terms of F measure. When applied to a state-of-art definition generation system recently fielded in the TREC 2003 definitional question answering task, it improves the performance by 14%.

REFERENCES

• 1: Eugene Agichtein , Luis Gravano, Snowball: extracting relations from large plain-text collections, Proceedings of the fifth ACM conference on Digital libraries, p.85-94, June 02-07, 2000, San Antonio, Texas, United States [doi>10.1145/336597.336644]
• 2: S. Blair-Goldensohn, K. R. McKeown and A. H. Schlaikjer. A Hybrid Approach for Answering Definitional Questions. Technical Report CUCS-006-03. Columbia University, 2003.
• 3: C. Buckley, G. Salton, J. Allan, and A. Singhal, Automatic query expansion using SMART, NIST Special Publication 500-225: The Third Text Retrieval conference (TREC 3), 1995, pp. 69--80.
• 4: Jaime Carbonell , Jade Goldstein, The use of MMR, diversity-based reranking for reordering documents and producing summaries, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.335-336, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291025]
• 5: S. Harabagiu, D. Moldovan, R. Mihalcea M. Pasca, R. Bunescu, M. Surdeanu, R. Girju, V. Rus, and P. Morarescu, Falcon: Boosting knowledge for answer engines, Proc. of Ninth Text Retrieval Conference (TREC 9), pp. 479--488, 2000.
• 6: J. L. Klavans, S. Popper and R. Passonneau, Tackling the Internet Glossary Glut: Automatic extraction and Evaluation of Genus Phrases, In Proceedings of Semantic Web Workshop, SIGIR 2003, July 28 - Aug. 1, 2003, Toronto, Canada.
• 7: J. Lannon, 1991, Technical Writing, Ch 5, HarperCollins Publishers Inc., 1991.
• 8: Chin-Yew Lin , Eduard Hovy, The automated acquisition of topic signatures for text summarization, Proceedings of the 18th conference on Computational linguistics, p.495-501, July 31-August 04, 2000, Saarbrücken, Germany [doi>10.3115/990820.990892]
• 9: Bing Liu , Chee Wee Chin , Hwee Tou Ng, Mining topic-specific concepts and definitions on the web, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775188]
• 10: I. Muslea. Extraction patterns for information extraction tasks: A survey. In AAAI-99 Workshop on Machine Learning for Information Extraction, 1999, pp. 1--6.
• 11: U. Y. Nahm and R. J. Mooney, 2001, Mining softmatching rules from textual data. In Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI-2001), pp. 979--984.
• 12: John Prager , Dragomir Radev , Krzysztof Czuba, Answering what-is questions by Virtual Annotation, Proceedings of the first international conference on Human language technology research, p.1-5, March 18-21, 2001, San Diego [doi>10.3115/1072133.1072139]
• 13: Dragomir R. Radev , Hongyan Jing , Malgorzata Budzikowska, Centroid-based summarization of multiple documents: sentence extraction, utility-based evaluation, and user studies, NAACL-ANLP 2000 Workshop on Automatic summarization, p.21-30, April 30-30, 2000, Seattle, Washington [doi>10.3115/1117575.1117578]
• 14: Deepak Ravichandran , Eduard Hovy, Learning surface text patterns for a Question Answering system, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073092]
• 15: E. Riloff. Automatically generating extraction patterns from untagged text. In Proceedings of the 13th National Conference on Artificial Intelligence (AAAI-96), AAAI Press, 1996, pp. 1044--1049.
• 16: Gerald Salton, Automatic text processing, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1988
• 17: Barry Schiffman , Inderjeet Mani , Kristian J. Concepcion, Producing biographical summaries: combining linguistic knowledge with corpus statistics, Proceedings of the 39th Annual Meeting on Association for Computational Linguistics, p.458-465, July 06-11, 2001, Toulouse, France [doi>10.3115/1073012.1073071]
• 18: Stephen Soderland, Learning Information Extraction Rules for Semi-Structured and Free Text, Machine Learning, v.34 n.1-3, p.233-272, Feb. 1999 [doi>10.1023/A:1007562322031]
• 19: Kiyoshi Sudo , Satoshi Sekine , Ralph Grishman, Automatic pattern acquisition for Japanese information extraction, Proceedings of the first international conference on Human language technology research, p.1-7, March 18-21, 2001, San Diego [doi>10.3115/1072133.1072142]
• 20: Ellen M. Voorhees, Evaluating answers to definition questions, Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: companion volume of the Proceedings of HLT-NAACL 2003--short papers, p.109-111, May 27-June 01, 2003, Edmonton, Canada [doi>10.3115/1073483.1073520]
• 21: Roman Yangarber , Ralph Grishman , Pasi Tapanainen , Silja Huttunen, Automatic acquisition of domain knowledge for Information Extraction, Proceedings of the 18th conference on Computational linguistics, July 31-August 04, 2000, Saarbrücken, Germany [doi>10.3115/992730.992782]
• 22: H. Yang, H. Cui, M.-Y. Kan, M. Maslennikov, L. Qiu, T.-S. Chua, QUALIFIER in TREC-12 QA Main Task, In Proceedings of the Twelfth Annual Text Retrieval Conference (TREC- 12), NIST, November, 2003.
• 23: Yiming Yang, An Evaluation of Statistical Approaches to Text Categorization, Information Retrieval, v.1 n.1-2, p.69-90, 1999 [doi>10.1023/A:1009982220290]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Experimentation

Keywords:
definition generation, definitional question answering, pseudo-relevance feedback, soft patterns, unsupervised learning