Mining concept associations for knowledge discovery in large textual databases

ABSTRACT

In this paper, we describe a new approach for mining concept associations from large text collections. The concepts are short sequences of words that occur frequently together across the text collections. It is these concepts that convey most of the meaning in any language. Our goal is to extract interesting associations among concepts that co-occur within the text collections. Interesting association between the concepts is mined using association rule mining algorithm. Finally we construct directed graph from current rules. The experimental result shows that our approach can efficiently find interesting concept associations in large text collections.

REFERENCES

• 1: Fayyad U., Piatetsky-Shapiro G., and Smyth P.: "Knowledge Discovery and Data Mining: Towards a Unifying Framework", Proc. 2nd Int. Conf. on Knowledge Discovery and Data Mining, Portland, OR, 1996, pp. 82--88
• 2: Rakesh Agrawal , Ramakrishnan Srikant, Fast Algorithms for Mining Association Rules in Large Databases, Proceedings of the 20th International Conference on Very Large Data Bases, p.487-499, September 12-15, 1994
• 3: Edward R. Omiecinski, Alternative Interest Measures for Mining Associations in Databases, IEEE Transactions on Knowledge and Data Engineering, v.15 n.1, p.57-69, January 2003 [doi>10.1109/TKDE.2003.1161582]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.6 Library Automation

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing

General Terms:
Algorithms, Management, Measurement, Performance

Keywords:
association rule mining, concept extraction, graphical model of concept associations, text data mining