A sentence level probabilistic model for evolutionary theme pattern mining from news corpora

ABSTRACT

Some recent topic model-based methods have been proposed to discover and summarize the evolutionary patterns of themes in temporal text collections. However, the theme patterns extracted by these methods are hard to interpret and evaluate. To produce a more descriptive representation of the theme pattern, we not only give new representations of sentences and themes with named entities, but we also propose a sentence-level probabilistic model based on the new representation pattern. Compared with other topic model methods, our approach not only gets each topic's distribution per term, but also generates candidate summary sentences of the themes as well. Consequently, the results are easier to understand and can be evaluated using the top sentences produced by our probabilistic model. Experimentation with the proposed methods on the Tsunami dataset shows that the proposed methods are useful in the discovery of evolutionary theme patterns.

REFERENCES

• 1: James Allan , Rahul Gupta , Vikas Khandelwal, Temporal summaries of new topics, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.10-18, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383954]
• 2: David M. Blei , Andrew Y. Ng , Michael I. Jordan, Latent dirichlet allocation, The Journal of Machine Learning Research, 3, p.993-1022, 3/1/2003
• 3: Thomas Hofmann, Probabilistic latent semantic indexing, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.50-57, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312649]
• 4: Thomas M. Cover , Joy A. Thomas, Elements of information theory, Wiley-Interscience, New York, NY, 1991
• 5: ChengXiang Zhai , Atulya Velivelli , Bei Yu, A cross-collection mixture model for comparative text mining, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1014150]
• 6: David M. Blei , John D. Lafferty, Dynamic topic models, Proceedings of the 23rd international conference on Machine learning, p.113-120, June 25-29, 2006, Pittsburgh, Pennsylvania [doi>10.1145/1143844.1143859]
• 7: T. Griffiths and M. Steyvers. Finding scientific topics. Proceedings of the National Academy of Sciences, 101(suppl. 1):5228--5235, 2004.
• 8: Kamal Nigam , Andrew Kachites McCallum , Sebastian Thrun , Tom Mitchell, Text Classification from Labeled and Unlabeled Documents using EM, Machine Learning, v.39 n.2-3, p.103-134, May-June 2000 [doi>10.1023/A:1007692713085]
• 9: Qiaozhu Mei , ChengXiang Zhai, Discovering evolutionary theme patterns from text: an exploration of temporal text mining, Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, August 21-24, 2005, Chicago, Illinois, USA [doi>10.1145/1081870.1081895]
• 10: Xuerui Wang , Andrew McCallum, Topics over time: a non-Markov continuous-time model of topical trends, Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, August 20-23, 2006, Philadelphia, PA, USA [doi>10.1145/1150402.1150450]
• 11: Satoshi Morinaga , Kenji Yamanishi, Tracking dynamics of topic trends using a finite mixture model, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1016919]
• 12: Ravi Kumar , Uma Mahadevan , D. Sivakumar, A graph-theoretic approach to extract storylines from search results, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1014078]
• 13: Zhiwei Li , Bin Wang , Mingjing Li , Wei-Ying Ma, A probabilistic model for retrospective news event detection, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076055]
• 14: Yiming Yang , Tom Pierce , Jaime Carbonell, A study of retrospective and on-line event detection, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.28-36, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290953]
• 15: R. Feldman and I. Dagan. Knowledge discovery in textual databases (kdt). In KDD, pages 112--117, 1995.
• 16: Marti A. Hearst, Untangling text data mining, Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, p.3-10, June 20-26, 1999, College Park, Maryland [doi>10.3115/1034678.1034679]
• 17: Jon Kleinberg, Bursty and hierarchical structure in streams, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, July 23-26, 2002, Edmonton, Alberta, Canada [doi>10.1145/775047.775061]
• 18: A. Kontostathis, L. Galitsky, W. M. Pottenger, S. Roy, and D. J. Phelps. A survey of emerging trend detection in textual data mining. Survey of Text Mining, pages 185--224, 2003.
• 19: Satoshi Morinaga , Kenji Yamanishi, Tracking dynamics of topic trends using a finite mixture model, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1016919]
• 20: S. Roy, D. Gevry, and W. M. Pottenger. Methodologies for trend detection in textual data mining. In the Textmine '02 Workshop, Second SIAM International Conference on Data Mining, 2002.
• 21: Alias-I, "LingPipe," Website, 9 2006. {Online}. Available: http://www.alias-i.com/lingpipe/index.html
• 22: Topic detection and tracking (tdt) project. homepage: http://www.nist.gov/speech/tests/tdt/.
• 23: James Allan , Ron Papka , Victor Lavrenko, On-line new event detection and tracking, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.37-45, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290954]
• 24: Giridhar Kumaran , James Allan, Text classification and named entities for new event detection, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009044]
• 25: Gabriel Pui Cheong Fung , Jeffrey Xu Yu , Huan Liu , Philip S. Yu, Time-dependent event hierarchy construction, Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, August 12-15, 2007, San Jose, California, USA [doi>10.1145/1281192.1281227]
• 26: Nadeau D. and Sekine S. A Survey of Named Entity Recognition and Classification. In: Sekine, S. and Ranchhod, E. Named Entities: Recognition, classification and use. Special issue of Linguistics Investigationes. 30(1) pp. 3--26.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Experimentation

Keywords:
evolutionary theme patterns, named entities, temporal text mining, topic model