Fast Extraction of Semantic Features from a Latent Semantic Indexed Text Corpus

ABSTRACT

This paper proposes a projection-based symmetrical factorisation method for extracting semantic features from collections of text documents stored in a Latent Semantic space. Preliminary experimental results demonstrate this yields a comparable representation to that provided by a novel probabilistic approach which reconsiders the entire indexing problem of text documents and works directly in the original high dimensional vector-space representation of text. The employed projection index is derived here from the a priori constraints on the problem. The principal advantage of this approach is computational efficiency and is obtained by the exploitation of the Latent Semantic Indexing as a preprocessing stage. Simulation results on subsets of the 20-Newsgroups text corpus in various settings are provided.

REFERENCES

• 1: 1. Berry, M. W: Large-scale sparse singular value computations, The International Journal of Super-computer Applications6(1) (1992), 13-49.
• 2: 2. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K. and Harshman, R.: Indexing by latent semantic analysis. J. Amer. Soc. Inf. Sci.41(6) (1990), 391-407.
• 3: 3. Kolenda, T., Hansen, L.-L. and Sigurdsson, S.: Independent components in text, In: M. Girolami (ed.), Advances in Independent Component Analysis (Springer-Verlag, 2000) pp. 241-262.
• 4: 4. Hofmann, T.: Probabilistic latent semantic analysis, Proceedings of the 15th Annual Conference on Uncertainty in Artificial Intelligence (UAI'99), San Francisco, CA, 1999, pp. 289-296.
• 5: Aapo Hyvärinen , Erkki Oja, A fast fixed-point algorithm for independent component analysis, Neural Computation, v.9 n.7, p.1483-1492, Oct. 1, 1997 [doi>10.1162/neco.1997.9.7.1483]
• 6: Slava M. Katz, Distribution of content words and phrases in text and language modelling, Natural Language Engineering, v.2 n.1, p.15-59, March 1996 [doi>10.1017/S1351324996001246]
• 7: 7. Kabán, A. and Girolami, M.: Unsupervised topic separation and keyword identification in document collections: A projection approach, Technical Report, 10, University of Paisley.
• 8: 8. Lee, D., Seung, S.: Learning the parts of objects by non-negative matrix factorization, Nature401 (1999), 788-791.
• 9: Christos H. Papadimitriou , Hisao Tamaki , Prabhakar Raghavan , Santosh Vempala, Latent semantic indexing: a probabilistic analysis, Proceedings of the seventeenth ACM SIGACT-SIGMOD-SIGART symposium on Principles of database systems, p.159-168, June 01-04, 1998, Seattle, Washington, United States [doi>10.1145/275487.275505]
• 10: Mehran Sahami , Daphne Koller, Using machine learning to improve information access, 1999

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.4 Applications
          Subjects: Text processing

Additional Classification:
  F. Theory of Computation
  F.2 ANALYSIS OF ALGORITHMS AND PROBLEM COMPLEXITY
      F.2.1 Numerical Algorithms and Problems
          Subjects: Computations on matrices
      F.2.2 Nonnumerical Algorithms and Problems
          Subjects: Complexity of proof procedures

  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Probabilistic algorithms (including Monte Carlo)

General Terms:
Algorithms, Experimentation

Keywords:
latent semantic indexing, probabilistic latent semantic analysis, projection pursuit, semantic feature extraction, text analysis