Tracking dynamics of topic trends using a finite mixture model

ABSTRACT

In a wide range of business areas dealing with text data streams, including CRM, knowledge management, and Web monitoring services, it is an important issue to discover topic trends and analyze their dynamics in real-time. Specifically we consider the following three tasks in topic trend analysis: 1)Topic Structure Identification; identifying what kinds of main topics exist and how important they are, 2)Topic Emergence Detection; detecting the emergence of a new topic and recognizing how it grows, 3)Topic Characterization; identifying the characteristics for each of main topics. For real topic analysis systems, we may require that these three tasks be performed in an on-line fashion rather than in a retrospective way, and be dealt with in a single framework. This paper proposes a new topic analysis framework which satisfies this requirement from a unifying viewpoint that a topic structure is modeled using a finite mixture model and that any change of a topic trend is tracked by learning the finite mixture model dynamically. In this framework we propose the usage of a time-stamp based discounting learning algorithm in order to realize real-time topic structure identification. This enables tracking the topic structure adaptively by forgetting out-of-date statistics. Further we apply the theory of dynamic model selection to detecting changes of main components in the finite mixture model in order to realize topic emergence detection. We demonstrate the effectiveness of our framework using real data collected at a help desk to show that we are able to track dynamics of topic trends in a timely fashion.

REFERENCES

• 1: James Allan , Ron Papka , Victor Lavrenko, On-line new event detection and tracking, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.37-45, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290954]
• 2: Xin Liu , Yihong Gong , Wei Xu , Shenghuo Zhu, Document clustering with cluster refinement and model selection capabilities, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564411]
• 3: Susan Havre , Beth Hetzler , Lucy Nowell, ThemeRiver: Visualizing Theme Changes over Time, Proceedings of the IEEE Symposium on Information Vizualization 2000, p.115, October 09-10, 2000
• 4: Jon Kleinberg, Bursty and hierarchical structure in streams, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, July 23-26, 2002, Edmonton, Alberta, Canada [doi>10.1145/775047.775061]
• 5: Hang Li , Kenji Yamanishi, Text classification using ESC-based stochastic decision lists, Information Processing and Management: an International Journal, v.38 n.3, p.343-361, May 2002 [doi>10.1016/S0306-4573(01)00038-3]
• 6: Hang Li , Kenji Yamanishi, Topic analysis using a finite mixture model, Information Processing and Management: an International Journal, v.39 n.4, p.521-541, July 2003 [doi>10.1016/S0306-4573(02)00035-3]
• 7: Y.Matsunaga and K.Yamanishi: An information-theoretic approach to detecting anomalous behaviors, in Information Technology Letters vol.2 (Proc. of the 2nd Forum on Information Technologies), pp:123--124, (in Japanese) 2003.
• 8: G.McLahlan and D.Peel: Finite Mixture Models, Wiley Series in Probability and Statistics, John Wiley and Sons, 2000.
• 9: Radford M. Neal , Geoffrey E. Hinton, A view of the EM algorithm that justifies incremental, sparse, and other variants, Learning in graphical models, MIT Press, Cambridge, MA, 1999
• 10: J.Rissanen: Universal coding, information, and estimation, IEEE Trans. on Inform. Theory, 30:629--636, 1984.
• 11: Russell Swan , James Allan, Extracting significant time varying features from text, Proceedings of the eighth international conference on Information and knowledge management, p.38-45, November 02-06, 1999, Kansas City, Missouri, United States [doi>10.1145/319950.319956]
• 12: Russell Swan , James Allan, Automatic generation of overview timelines, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.49-56, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345546]
• 13: K.Yamanishi: A Decision-theoretic Extension of Stochastic Complexity and Its Applications to Learning, IEEE Trans. on Inform. Theory, vol.44/4, pp:1424--1439, 1998.
• 14: Kenji Yamanishi , Jun-Ichi Takeuchi , Graham Williams , Peter Milne, On-line unsupervised outlier detection using finite mixtures with discounting learning algorithms, Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, p.320-324, August 20-23, 2000, Boston, Massachusetts, United States [doi>10.1145/347090.347160]
• 15: Yiming Yang , Tom Pierce , Jaime Carbonell, A study of retrospective and on-line event detection, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.28-36, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290953]
• 16: Yiming Yang , Jian Zhang , Jaime Carbonell , Chun Jin, Topic-conditioned novelty detection, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, July 23-26, 2002, Edmonton, Alberta, Canada [doi>10.1145/775047.775150]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

General Terms:
Algorithms, Design

Keywords:
CRM, model selection, text mining, topic analysis