Data association for topic intensity tracking

ABSTRACT

We present a unified model of what was traditionally viewed as two separate tasks: data association and intensity tracking of multiple topics over time. In the data association part, the task is to assign a topic (a class) to each data point, and the intensity tracking part models the bursts and changes in intensities of topics over time. Our approach to this problem combines an extension of Factorial Hidden Markov models for topic intensity tracking with exponential order statistics for implicit data association. Experiments on text and email datasets show that the interplay of classification and topic intensity tracking improves the accuracy of both classification and intensity tracking. Even a little noise in topic assignments can mislead the traditional algorithms. However, our approach detects correct topic intensities even with 30% topic noise.

REFERENCES

• 1: Aizen, J., Huttenlocher, D., Kleinberg, J., & Novak, A. (2004). Traffic-based feedback on the web. Proc. Natl. Acad. Sci., 101, 5254--5260.
• 2: James Allan , Ron Papka , Victor Lavrenko, On-line new event detection and tracking, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.37-45, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290954]
• 3: Blei, D., & Lafferty, J. (2005). Correlated topic models. NIPS '05.
• 4: David M. Blei , Andrew Y. Ng , Michael I. Jordan, Latent dirichlet allocation, The Journal of Machine Learning Research, 3, p.993-1022, 3/1/2003
• 5: Deerwester, S. C., Dumais, S. T., Landauer, T. K., Furnas, G. W., & Harshman, R. A. (1990). Indexing by latent semantic analysis. J. of the Am. Soc. of Inf. Sci., 41.
• 6: Nir Friedman , Dan Geiger , Moises Goldszmidt, Bayesian Network Classifiers, Machine Learning, v.29 n.2-3, p.131-163, Nov./Dec. 1997 [doi>10.1023/A:1007465528199]
• 7: Ghahramani, Z., & Jordan, M. I. (1995). Factorial hidden Markov models. NIPS '95.
• 8: Kleinberg, J. (2003). Bursty and hierarchical structure in streams. KDD '03.
• 9: Krause, A., Leskovec, J., & Guestrin, C. (2006). Data association for topic intensity tracking (Technical Report CMU-ML-06-100). Carnegie Mellon University.
• 10: Lerner, U. (2002). Hybrid bayesian networks for reasoning about complex systems. Ph.d. thesis, Stanford University.
• 11: Uri Lerner , Ronald Parr, Inference in Hybrid Networks: Theoretical Limits and Practical Algorithms, Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence, p.310-318, August 02-05, 2001
• 12: Ng, B., Pfeffer, A., & Dearden, R. (2005). Continuous time particle filtering. IJCAI.
• 13: Nodelman, U., Shelton, C., & Koller, D. (2003). Learning continuous time bayesian networks. UAI.
• 14: Richard B. Segal , Jeffrey O. Kephart, MailCat: an intelligent assistant for organizing e-mail, Proceedings of the third annual conference on Autonomous Agents, p.276-282, April 1999, Seattle, Washington, United States [doi>10.1145/301136.301209]
• 15: Russell Swan , James Allan, Automatic generation of overview timelines, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.49-56, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345546]
• 16: Kishor S. Trivedi, Probability and statistics with reliability, queuing and computer science applications, John Wiley and Sons Ltd., Chichester, UK, 2001
• 17: Yiming Yang , Tom Ault , Thomas Pierce , Charles W. Lattimer, Improving text categorization methods for event tracking, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.65-72, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345550]

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Markov processes

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Statistical computing

  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology

General Terms:
Algorithms, Theory