Automatic generation of overview timelines

ABSTRACT

We present a statistical model of feature occurrence over time, and develop tests based on classical hypothesis testing for significance of term appearance on a given date. Using additional classical hypothesis testing we are able to combine these terms to generate “topics” as defined by the Topic Detection and Tracking study. The groupings of terms obtained can be used to automatically generate an interactive timeline displaying the major events and topics covered by the corpus. To test the validity of our technique we extracted a large number of these topics from a test corpus and had human evaluators judge how well the selected features captured the gist of the topics, and how they overlapped with a set of known topics from the corpus. The resulting topics were highly rated by evaluators who compared them to known topics.

REFERENCES

• 1: James Allan , Ron Papka , Victor Lavrenko, On-line new event detection and tracking, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.37-45, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290954]
• 2: R. B. Allen. Timelines as information system interfaces. In Proceedings International Symposium on Digital Libraries, pages 175-180, Tsukuba, Japan, 1995.
• 3: Daniel M. Bikel , Scott Miller , Richard Schwartz , Ralph Weischedel, Nymble: a high-performance learning name-finder, Proceedings of the fifth conference on Applied natural language processing, p.194-201, March 31-April 03, 1997, Washington, DC [doi>10.3115/974557.974586]
• 4: Jack G. Conrad , Mary Hunter Utt, A system for discovering relationships by feature extraction from text databases, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.260-270, July 03-06, 1994, Dublin, Ireland
• 5: Ido Dagan and Ronen Feldman. Keyword-based browsing and analysis of large document sets. In Proceedings of the Symposium on Document Analysis and Information Retrieval (SDAIR-96), Las Vegas, Nevada, 1996.
• 6: DARPA, editor. Proceedings of the DARPA Broadcast news Workshop, Herndon, Virginia, February 1999.
• 7: B. S. Everitt. The Analysis of Contingency Tables. Chapman and Hall, London, 1977.
• 8: Ronen Feldman and Ido Dagan. Knowledge discovery in textual databases (kdt). In Proceedings of the First International Conference on Knowledge Discovery (KDD-95). ACM, August 1995.
• 9: Scott Fertig , Eric Freeman , David Gelernter, Lifestreams: an alternative to the desktop metaphor, Conference companion on Human factors in computing systems: common ground, p.410-411, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/257089.257404]
• 10: Dan Ingalls , Ted Kaehler , John Maloney , Scott Wallace , Alan Kay, Back to the future: the story of Squeak, a practical Smalltalk written in itself, Proceedings of the 12th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, p.318-326, October 05-09, 1997, Atlanta, Georgia, United States
• 11: Vijay Kumar , Richard Furuta , Robert B. Allen, Metadata visualization for digital libraries: interactive timeline editing and review, Proceedings of the third ACM conference on Digital libraries, p.126-133, June 23-26, 1998, Pittsburgh, Pennsylvania, United States [doi>10.1145/276675.276689]
• 12: Catherine Plaisant , Brett Milash , Anne Rose , Seth Widoff , Ben Shneiderman, LifeLines: visualizing personal histories, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.221-ff., April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238493]
• 13: Earl Rennison, Galaxy of news: an approach to visualizing and understanding expansive news landscapes, Proceedings of the 7th annual ACM symposium on User interface software and technology, p.3-12, November 02-04, 1994, Marina del Rey, California, United States [doi>10.1145/192426.192429]
• 14: Russell Swan , James Allan, Extracting significant time varying features from text, Proceedings of the eighth international conference on Information and knowledge management, p.38-45, November 02-06, 1999, Kansas City, Missouri, United States [doi>10.1145/319950.319956]
• 15: Russell Swan , James Allan, TimeMine (demonstration session): visualizing automatically constructed timelines, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.393, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345674]
• 16: R. H. Thompson , W. B. Croft, Support for browsing in an intelligent text retrieval system, International Journal of Man-Machine Studies, v.30 n.6, p.639-668, June 1989
• 17: James A. Wise , James J. Thomas , Kelly Pennock , David Lantrip , Marc Pottier , Anne Schur , Vern Crow, Visualizing the non-visual: spatial analysis and interaction with information for text documents, Readings in information visualization: using vision to think, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1999
• 18: Xia Lin , Dagobert Soergel , Gary Marchionini, A self-organizing semantic map for information retrieval, Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, p.262-269, October 13-16, 1991, Chicago, Illinois, United States [doi>10.1145/122860.122887]
• 19: J. Xu , J. Broglio , B. Croft, The Design and Implementation of a Part of Speech Tagger for English, University of Massachusetts, Amherst, MA, 1994
• 20: Yiming Yang , Tom Pierce , Jaime Carbonell, A study of retrospective and on-line event detection, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.28-36, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290953]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

General Terms:
Experimentation, Human Factors, Performance, Reliability

Keywords:
UIs&slash;visualization for collection overviews, event detection and tracking, statistical&slash;probabilistic models, text data mining