Improving summarization performance by sentence compression: a pilot study

ABSTRACT

In this paper we study the effectiveness of applying sentence compression on an extraction based multi-document summarization system. Our results show that pure syntactic-based compression does not improve system performance. Topic signature-based reranking of compressed sentences does not help much either. However reranking using an oracle showed a significant improvement remains possible.

REFERENCES

• 1: Orkut Buyukkokten , Hector Garcia-Molina , Andreas Paepcke, Seeing the whole in parts: text summarization for web browsing on handheld devices, Proceedings of the 10th international conference on World Wide Web, p.652-662, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.372178]
• 2: J. Carroll, G. Minnen, Y. Canning, S. Devlin, and J. Tait. 1998. Practical Simplification of English Newspaper Text to Assist Aphasic Readers. In Proceedings of AAAI-98 Workshop on Integrating Artificial Intelligence and Assistive Technology, Madison, WI, USA.
• 3: H. H. Chen, J. J. Kuo, and T. C. Su 2003. Clustering and Visualization in a Multi-Lingual MultiDocument Summarization System. In Proceedings of 25th European Conference on Information Retrieval Research, Lecture Note in Computer Science, April 14-16, Pisa, Italy.
• 4: S. Corston-Oliver. 2001. Text Compaction for Display on Very Small Screens. In Proceedings of the Workshop on Automatic Summarization (WAS 2001), Pittsburgh, PA, USA.
• 5: DUC. 2002. The Document Understanding Conference. http://duc.nist.gov.
• 6: Ted Dunning, Accurate methods for the statistics of surprise and coincidence, Computational Linguistics, v.19 n.1, March 1993
• 7: H. P. Edmundson, New Methods in Automatic Extracting, Journal of the ACM (JACM), v.16 n.2, p.264-285, April 1969 [doi>10.1145/321510.321519]
• 8: T. Fukusima and M. Okumura. 2001. Text Summarization Challenge Text Summarization Evaluation in Japan. In Proceedings of the Workshop on Automatic Summarization (WAS 2001), Pittsburgh, PA, USA.
• 9: Jade Goldstein , Mark Kantrowitz , Vibhu Mittal , Jaime Carbonell, Summarizing text documents: sentence selection and evaluation metrics, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.121-128, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312665]
• 10: G. Grefenstette. 1998. Producing Intelligent Telegraphic Text Reduction to Provide an Audio Scanning Service for the Blind. In Working Notes of the AAAI Spring Symposium on Intelligent Text Summarization, Stanford University, CA, USA, 111--118.
• 11: E. Hovy and C.-Y. Lin. 1999. Automatic Text Summarization in SUMMARIST. In I. Mani and M. Maybury (eds), Advances in Automatic Text Summarization, 81--94. MIT Press.
• 12: Hongyan Jing, Sentence reduction for automatic text summarization, Proceedings of the sixth conference on Applied natural language processing, p.310-315, April 29-May 04, 2000, Seattle, Washington [doi>10.3115/974147.974190]
• 13: Kevin Knight , Daniel Marcu, Statistics-Based Summarization - Step One: Sentence Compression, Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence, p.703-710, July 30-August 03, 2000
• 14: Julian Kupiec , Jan Pedersen , Francine Chen, A trainable document summarizer, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.68-73, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215333]
• 15: Chin-Yew Lin , Eduard Hovy, The automated acquisition of topic signatures for text summarization, Proceedings of the 18th conference on Computational linguistics, p.495-501, July 31-August 04, 2000, Saarbrücken, Germany [doi>10.3115/990820.990892]
• 16: Chin-Yew Lin , Eduard Hovy, From single to multi-document summarization: a prototype system and its evaluation, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073160]
• 17: Chin-Yew Lin , Eduard Hovy, Manual and automatic evaluation of summaries, Proceedings of the ACL-02 Workshop on Automatic Summarization, p.45-51, July 11-12, 2002, Phildadelphia, Pennsylvania [doi>10.3115/1118162.1118168]
• 18: Chin-Yew Lin , Eduard Hovy, Automatic evaluation of summaries using N-gram co-occurrence statistics, Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, p.71-78, May 27-June 01, 2003, Edmonton, Canada [doi>10.3115/1073445.1073465]
• 19: Chin-Yew Lin , Eduard Hovy, The potential and limitations of automatic sentence extraction for summarization, Proceedings of the HLT-NAACL 03 on Text summarization workshop, p.73-80, May 31, 2003 [doi>10.3115/1119467.1119477]
• 20: H. P. Luhn. 1969. The Automatic Creation of Literature Abstracts. IBM Journal of Research and Development. 2(2).
• 21: D. Marcu. 1999. Discourse trees are good indicators of importance in text. In I. Mani and M. Maybury (eds), Advances in Automatic Text Summarization, 123--136. MIT Press.
• 22: K. McKeown, Barzilay, D. Evans, V. Hatzivassiloglou, J. L. Klavans, A. Nenkova, C. Sable, B. Schiffman, S. Sigelman. 2002. Tracking and Summarizing News on a Daily Basis with Columbia's Newsblaster. In Proceedings of Human Language Technology Conference 2002 (HLT 2002). San Diego, CA, USA.
• 23: NIST. 2002. Automatic Evaluation of Machine Translation Quality using N-gram Co-Occurrence Statistics.
• 24: P. Over and W. Liggett. 2002. Introduction to DUC-2002: an Intrinsic Evaluation of Generic News Text Summarization Systems. In Proceedings of Workshop on Automatic Summarization (DUC 2002), Philadelphia, PA, USA. http://www-nlpir.nist.gov/projects/duc/pubs/2002slides/overview.02.pdf
• 25: K. Papineni, S. Roukos, T. Ward, W.-J. Zhu. 2001. Bleu: a Method for Automatic Evaluation of Machine Translation. IBM Research Report RC22176 (W0 109--022).
• 26: Dragomir R. Radev , Kathleen R. McKeown, Generating natural language summaries from multiple on-line sources, Computational Linguistics, v.24 n.3, September 1998
• 27: T. Strzalkowski, G. Stein, J. Wang, and B, Wise. A Robust Practical Text Summarizer. 1999. In I. Mani and M. Maybury (eds), Advances in Automatic Text Summarization, 137--154. MIT Press.
• 28: Michael White , Tanya Korelsky , Claire Cardie , Vincent Ng , David Pierce , Kiri Wagstaff, Multidocument summarization via information extraction, Proceedings of the first international conference on Human language technology research, p.1-7, March 18-21, 2001, San Diego [doi>10.3115/1072133.1072206]

INDEX TERMS

Keywords:
evaluation, sentence compression, sentence extraction, text summarization