Temporal summaries of new topics

ABSTRACT

We discuss technology to help a person monitor changes in news coverage over time. We define temporal summaries of news stories as extracting a single sentence from each event within a news topic, where the stories are presented one at a time and sentences from a story must be ranked before the next story can be considered. We explain a method for evaluation, and describe an evaluation corpus that we have built. We also propose several methods for constructing temporal summaries and evaluate their effectiveness in comparison to degenerate cases. We show that simple approaches are effective, but that the problem is far from solved.

REFERENCES

• 1: J. Allan, J. Carbonell, G. Doddington, J. Yamron, and Y. Yang. Topic detection and tracking pilot study: Final report. In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop, pages 194-218, 1998.
• 2: J. Allan, H. Jin, M. Rajman, C. Wayne, D. Gildea, V. L. vrenko, R. Hoberman, and D. Caputo. Topic-based novelty detection: 1999 summer workshop at CLSP, Final report. Available at http://www.clsp.jhu.edu/ws99/tdt, 1999.
• 3: C. Aone, M. Okurowski, J. Gorlinsky, and B. Larsen. A trainable summarizer with knowledge acquired from robust nlp techniques. In Mani and Maybury {21}.
• 4: Adam L. Berger , Vibhu O. Mittal, OCELOT: a system for summarizing Web pages, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.144-151, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345565]
• 5: Ronald Brandow , Karl Mitze , Lisa F. Rau, Automatic condensation of electronic publications by sentence selection, Information Processing and Management: an International Journal, v.31 n.5, p.675-685, Sept. 1995 [doi>10.1016/0306-4573(95)00052-I]
• 6: Jaime Carbonell , Jade Goldstein, The use of MMR, diversity-based reranking for reordering documents and producing summaries, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.335-336, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291025]
• 7: DARPA, editor. Proceedings of the DARPA Broadcast news Workshop, Herndon, Virginia, Feb. 1999.
• 8: DARPA, editor. The TDT-2 Text and Speech Corpus, Herndon, Virginia, Feb. 1999.
• 9: Robert L. Donaway , Kevin W. Drummey , Laura A. Mather, A comparison of rankings produced by summarization evaluation measures, NAACL-ANLP 2000 Workshop on Automatic summarization, p.69-78, April 30-30, 2000, Seattle, Washington [doi>10.3115/1117575.1117583]
• 10: H. P. Edmundson, New Methods in Automatic Extracting, Journal of the ACM (JACM), v.16 n.2, p.264-285, April 1969 [doi>10.1145/321510.321519]
• 11: T. Firmin and M. Chrzanowski. An evaluation of automatic text summarization systems. In Mani and Maybury {21}.
• 12: Fumiyo Fukumoto , Yoshimi Suzuki, Extracting key paragraph based on topic and event detection: towards multi-document summarization, NAACL-ANLP 2000 Workshop on Automatic summarization, p.31-39, April 30-30, 2000, Seattle, Washington [doi>10.3115/1117575.1117579]
• 13: Jade Goldstein , Mark Kantrowitz , Vibhu Mittal , Jaime Carbonell, Summarizing text documents: sentence selection and evaluation metrics, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.121-128, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312665]
• 14: Jade Goldstein , Vibhu Mittal , Jaime Carbonell , Mark Kantrowitz, Multi-document summarization by sentence extraction, NAACL-ANLP 2000 Workshop on Automatic summarization, p.40-48, April 30-30, 2000, Seattle, Washington [doi>10.3115/1117575.1117580]
• 15: U. Hahn, C. Lin, I. Mani, and D. Radev, editors. Automatic Summarization: ANLP/NAACL 2000 Workshop,New Brunswick, NJ, 2000. Association for Computational Linguistics.
• 16: E. Hovy and C. Lin. Automated text summarization in SUMMARIST. In Mani and Maybury {21}.
• 17: V. Khandelwal, R. Gupta, and J. Allan. An evaluation scheme for summarizing topic shifts in news streams, 2001. Submitted to Human Language Technology Conference.
• 18: Julian Kupiec , Jan Pedersen , Francine Chen, A trainable document summarizer, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.68-73, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215333]
• 19: H. Luhn. The automatic creation of literature abstracts. In Mani and Maybury {21}. Originally published in IBM Journal of R&D.
• 20: Inderjeet Mani , Eric Bloedorn, Summarizing Similarities and Differences Among Related Documents, Information Retrieval, v.1 n.1-2, p.35-67, 1999 [doi>10.1023/A:1009930203452]
• 21: Mark T. Maybury , Inderjeet Mani, Advances in Automatic Text Summarization, MIT Press, Cambridge, MA, 1999
• 22: Inderjeet Mani , George Wilson, Robust temporal processing of news, Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, p.69-76, October 03-06, 2000, Hong Kong [doi>10.3115/1075218.1075228]
• 23: Christopher D. Manning , Hinrich Schütze, Foundations of statistical natural language processing, MIT Press, Cambridge, MA, 1999
• 24: Mark T. Maybury, Generating summaries from event data, Information Processing and Management: an International Journal, v.31 n.5, p.735-751, Sept. 1995 [doi>10.1016/0306-4573(95)00025-C]
• 25: K. McKeown and D. Radev. Generating summaries of multiple news articles. In Mani and Maybury {21}.
• 26: A. Morris, G. Kasper, and D. Adams. The effects and limitations of automated text condensing on reading comprehension preformance. In Mani and Maybury {21}. Originally published in Information Systems Research.
• 27: S. Myaeng and D. Jang. Development and evaluation of a statistically based document summarization system. In Mani and Maybury {21}.
• 28: NIST. Proceedings of the tdt 1999 workshop. Notebook publication for participants only, Mar. 2000.
• 29: NIST. Proceedings of the tdt 2000 workshop. Notebook publication for participants only, Nov. 2000.
• 30: Mamiko Oka , Yoshihiro Ueda, Evaluation of phrase-representation summarization based on information retrieval task, NAACL-ANLP 2000 Workshop on Automatic summarization, p.59-68, April 30-30, 2000, Seattle, Washington [doi>10.3115/1117575.1117582]
• 31: Mary Ellen Okurowski , Harold Wilson , Joaquin Urbina , Tony Taylor , Ruth Colvin Clark , Frank Krapcho, Text summarizer in use: lessons learned from real world deployment and evaluation, NAACL-ANLP 2000 Workshop on Automatic summarization, p.49-58, April 30-30, 2000, Seattle, Washington [doi>10.3115/1117575.1117581]
• 32: J. Pollock and A. Zamora. Automatic abstracting research at chemical abstracts service. In Mani and Maybury {21}. Originally published in Journal of Chemical Information and Computer Science.
• 33: Jay M. Ponte , W. Bruce Croft, A language modeling approach to information retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.275-281, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291008]
• 34: Dragomir R. Radev , Hongyan Jing , Malgorzata Budzikowska, Centroid-based summarization of multiple documents: sentence extraction, utility-based evaluation, and user studies, NAACL-ANLP 2000 Workshop on Automatic summarization, p.21-30, April 30-30, 2000, Seattle, Washington [doi>10.3115/1117575.1117578]
• 35: G. Rath, A. Resnick, and T. Savage. The formation of abstracts by the selection of sentences. In Mani and Maybury {21}. Originally published in American Documentation (now JASIS).
• 36: Horacio Saggion , Guy Lapalme, Concept identification and presentation in the context of technical text summarization, NAACL-ANLP 2000 Workshop on Automatic summarization, p.1-10, April 30-30, 2000, Seattle, Washington [doi>10.3115/1117575.1117576]
• 37: Gerard Salton , Amit Singhal , Mandar Mitra , Chris Buckley, Automatic text structuring and summarization, Information Processing and Management: an International Journal, v.33 n.2, p.193-207, March 1997 [doi>10.1016/S0306-4573(96)00062-3]
• 38: Russell Swan , James Allan, Automatic generation of overview timelines, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.49-56, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345546]
• 39: Michael J. Witbrock , Vibhu O. Mittal, Ultra-summarization (poster abstract): a statistical approach to generating highly condensed non-extractive summaries, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.315-316, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312748]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.4 Knowledge Representation Formalisms and Methods
          Subjects: Temporal logic

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process; Retrieval models

General Terms:
Design, Experimentation, Performance

Keywords:
experimental design, metrics, summarization