Finding topic trends in digital libraries

ABSTRACT

We propose a generative model based on latent Dirichlet allocation for mining distinct topics in document collections by integrating the temporal ordering of documents into the generative process. The document collection is divided into time segments where the discovered topics in each segment is propagated to influence the topic discovery in the subsequent time segments. We conduct experiments on the collection of academic papers from CiteSeer repository. We augment the text corpus with the addition of user queries and tags and integrate the citation graph to boost the weight of the topical terms. The experiment results show that segmented topic model can effectively detect distinct topics and their evolution over time.

REFERENCES

• 1: G. Almpanidis , C. Kotropoulos , I. Pitas, Combining text and link analysis for focused crawling-An application for vertical search engines, Information Systems, v.32 n.6, p.886-908, September, 2007 [doi>10.1016/j.is.2006.09.004]
• 2: L. Bolelli, S. Ertekin, and C. L. Giles. Clustering scientific literature using sparse citation graph analysis. In PKDD'06, pages 30--41, 2006.
• 3: Mark Steyvers , Padhraic Smyth , Michal Rosen-Zvi , Thomas Griffiths, Probabilistic author-topic models for information discovery, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1014087]
• 4: C. D. X. He, H. Zha and H. Simon. Web document clustering using hyperlink structures. Computational Statistics and Data Analysis, 41:19--45, 2002.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

General Terms:
Algorithms, Design, Experimentation

Keywords:
latent dirichlet allocation, topic detection, trend analysis