Mining from open answers in questionnaire data

ABSTRACT

Surveys are an important part of marketing and customer relationship management, and open answers (i.e., answers to open questions) in particular may contain valuable information and provide an important basis for making business decisions. We have developed a text mining system that provides a new way for analyzing open answers in questionnaire data. The product is able to perform the following two functions: (A) accurate extraction of characteristics for individual analysis targets, (B) accurate extraction of the relationships among characteristics of analysis targets. In this paper, we describe the working of our text mining system. It employs two statistical learning techniques: rule analysis and Correspondence Analysis for performing the two functions. Our text mining system has already been put into use by a number of large corporations in Japan in the performance of text mining on various types of survey data, including open answers about brand images, open answers about company images, complaints about products, comments written on home pages, business reports, and help desk records. In this it has been found to be useful in forming a basis for effective business decisions.

REFERENCES

• 1: M.R. Anderberg, Cluster Analysis for Applications, Academic Press, 1973.
• 2: J.P. Benzecri, Correspondence Analysis Handbook. Mercel Dekker, i 992.
• 3: Jochen Dörre , Peter Gerstl , Roland Seiffert, Text mining: finding nuggets in mountains of textual data, Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining, p.398-401, August 15-18, 1999, San Diego, California, United States [doi>10.1145/312129.312299]
• 4: Ronen Feldman and Ido Dagan, Knowledge discovery in textual databases (KDT), Proceedings of First International Conference on Knowledge Discovery and Data Mining, 1995.
• 5: Fujitsu, Symfoware World http://www.fuiitsu.co.ip/ip/soft/symfoware/index.html, 2001.
• 6: Marko Grobelnik, Dunja Mladenic, and Natasa Milic-Fraling (Ed.) Proceedings of KDD-2000 Workshop on Text Mining, 2000.
• 7: Marti A. Hearst, Untangling text data mining, Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, p.3-10, June 20-26, 1999, College Park, Maryland [doi>10.3115/1034678.1034679]
• 8: Komatsu Soft, Information Mining Tool VextSearch (in Japanese) http://www.komatsusoft.co.jp/develp/vxtsc/index.html, 2001.
• 9: Brian Lent and Rakesh Agrawal and Ramakrishnan Srikant, Discovering trends in text databases, Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, 227-230, 1997.
• 10: Hang Li , Kenji Yamanishi, Text classification using ESC-based stochastic decision lists, Proceedings of the eighth international conference on Information and knowledge management, p.122-130, November 02-06, 1999, Kansas City, Missouri, United States [doi>10.1145/319950.319966]
• 11: Hang Li , Kenji Yamanishi, Topic analysis using a finite mixture model, Proceedings of the 2000 Joint SIGDAT conference on Empirical methods in natural language processing and very large corpora: held in conjunction with the 38th Annual Meeting of the Association for Computational Linguistics, p.35-44, October 07-08, 2000, Hong Kong [doi>10.3115/1117794.1117799]
• 12: Jorma Rissanen, Fisher information and stochastic complexity, IEEE Transaction on Information Theory, 42(1):40- 47, 1996.
• 13: Russell Swan , James Allan, Extracting significant time varying features from text, Proceedings of the eighth international conference on Information and knowledge management, p.38-45, November 02-06, 1999, Kansas City, Missouri, United States [doi>10.1145/319950.319956]
• 14: Mark Shewhart , Mark Wasson, Monitoring a newsfeed for hot topics, Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining, p.402-404, August 15-18, 1999, San Diego, California, United States [doi>10.1145/312129.312301]
• 15: Kenji Yamanishi, A Learning Criterion for Stochastic Rules, Machine Learning, v.9 n.2-3, p.165-203, July 1992 [doi>10.1007/BF00992676]
• 16: Kenji Yamanishi, A decision-theoretic extension of stochastic complexity and its applications to learning, IEEE Transaction on Infortmation Theory.,44(4): 1424-1439, 1998.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems
          Subjects: Rule-based databases; Textual databases

General Terms:
Algorithms, Design, Performance

Keywords:
Association Rules, Classification Rules, Correspondence Analysis, Open Question, Questionnaire Data, Survey, Text Mining