Event tracking based on domain dependency

ABSTRACT

This paper proposes a method for event tracking on broadcast news stories based on distinction between a topic and an event. A topic and an event are identified using a simple criterion called domain dependency of words: how greatly a word features a given set of data. The method was tested on the TDT corpus which has been developed by the TDT Pilot Study and the result can be regarded as promising the usefulness of the method.

REFERENCES

• 1: J. Allan. J. Carbonell, G. Doddington. J. Yamron, and Y. Yang. Topic Detection and Tracking Pilot Study: Final Report. In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop, 1998.
• 2: J. Allan et al. The Topic Detection and Tracking (TDT) Pilot Study Evaluation Plan, Version 2.8. pages 1-8, 1997.
• 3: James Allan , Ron Papka , Victor Lavrenko, On-line new event detection and tracking, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.37-45, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290954]
• 4: L. Douglas Baker , Andrew Kachites McCallum, Distributional clustering of words for text classification, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.96-103, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290970]
• 5: Eric Brill, A simple rule-based part of speech tagger, Proceedings of the third conference on Applied natural language processing, March 31-April 03, 1992, Trento, Italy [doi>10.3115/974499.974526]
• 6: J. Carbonell, Y. Yang, J. Lafferty, R. D. Brown, T. Pierce, and X. Liu. CMU Report on TDT-2: Segmentation, Detection and Tracking. In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop, 1999.
• 7: Douglass R. Cutting , David R. Karger , Jan O. Pedersen , John W. Tukey, Scatter/Gather: a cluster-based approach to browsing large document collections, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.318-329, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133214]
• 8: Daphne Koller , Mehran Sahami, Hierarchically Classifying Documents Using Very Few Words, Proceedings of the Fourteenth International Conference on Machine Learning, p.170-178, July 08-12, 1997
• 9: Julian Kupiec , Jan Pedersen , Francine Chen, A trainable document summarizer, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.68-73, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215333]
• 10: W. Lam , S. Mukhopadhyay , J. Mostafa , M. Palakal, Detection of shifts in user interests for personalized information filtering, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.317-325, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243279]
• 11: C. Leacock and M. Chodorow. Combining Local Context and WordNet Similarity for Word Sense Identification, WordNet. An Electronic Lezical Database. MIT Press, 1988.
• 12: David D. Lewis, An evaluation of phrasal and clustered representations on a text categorization task, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.37-50, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133172]
• 13: David D. Lewis , Robert E. Schapire , James P. Callan , Ron Papka, Training algorithms for linear text classifiers, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.298-306, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243277]
• 14: S. A. Lowe. The BetaBinomial Mixture Model and its Application to TDT Tracking and Detection. In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop, 1999.
• 15: H. P. Luhn. A Statistical Approach to Mechanized Encoding and Searching of Literary Information. IBM journal, 1(4):307-319, 1957.
• 16: II. P. Luhn. The Automatic Creation of Literature Abstracts. IBM journal, 2(1):159-165, 1958.
• 17: Rila Mandala , Takenobu Tokunaga , Hozumi Tanaka, Complementing WordNet with Roget's and corpus-based thesauri for information retrieval, Proceedings of the ninth conference on European chapter of the Association for Computational Linguistics, June 08-12, 1999, Bergen, Norway [doi>10.3115/977035.977049]
• 18: G. A. Miller. Nouns in WordNet, An Electronic Lexical Database. MIT Press, 1998.
• 19: R. Papka, J. Allan, and V. Lavrenko. UMASS Approaches to Detection and Tracking at TDT2. In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop, 1999.
• 20: P. Resnik. Using Information Content to Evaluate Semantic Similarity in a Taxonomy. In Proceedings of 15th International Joint Conference on Artificial Intelligence, pages 448-453, 1995.
• 21: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• 22: Robert E. Schapire , Yoram Singer, BoosTexter: A Boosting-based Systemfor Text Categorization, Machine Learning, v.39 n.2-3, p.135-168, May-June 2000 [doi>10.1023/A:1007649029923]
• 23: S. Sekine and R. Grishman. A Corpus-based Probabilistic Grammar with Only Two Non-Terminals. In Proceedings of the Fourth International Workshop on Parsing Technology, pages 216-223, 1995.
• 24: T. Strzalkowski, G. C. Stein. and G. B. Wise. GE.Tracker: A Robust, Lightweight Topic Tracking System. In Proceedings of the DARPA Broadcast News Transcmption and Understanding Workshop, 1999.
• 25: Y. Watanabe, Y. Okaxta, K. Kaneji, and Y. Sakamoto. Multiple Media Database System for TV Newscasts and Newspapers(in 3apanese). In Technical Report of IEIGE. NLG*97-69, PRMU97- 257, pages 47-54. 1998.
• 26: J. P. Yamron, I. Carp, L. Gillick. S. Lowe, and P. van Mulbregt. Topic Tracking in a News Stream. In Proceedmgs of the DARPA Broadcast News Transcmplion and Understandtng Workshop, 1999.
• 27: Yiming Yang, Expert network: effective and efficient learning from human decisions in text categorization and retrieval, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.13-22, July 03-06, 1994, Dublin, Ireland
• 28: Y. Yang J. G. Carbonell, J. Allan, and J. Ymron. Topic Detection and Tracking: Detection-Task. In Proceedings of the Workshop of Topic Detection and Tracking, 1997.
• 29: Yiming Yang , Tom Pierce , Jaime Carbonell, A study of retrospective and on-line event detection, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.28-36, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290953]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

General Terms:
Design, Experimentation, Performance, Theory