Opinion mining from noisy text data

ABSTRACT

The proliferation of Internet has not only generated huge volumes of unstructured information in the form of web documents, but a large amount of text is also generated in the form of emails, blogs, and feedbacks etc. The data generated from online communication acts as potential gold mines for discovering knowledge. Text analytics has matured and is being successfully employed to mine important information from unstructured text documents. Most of these techniques use Natural Language Processing techniques which assume that the underlying text is clean and correct. Statistical techniques, though not as accurate as linguistic mechanisms, are also employed for the purpose to overcome the dependence on clean text. The chief bottleneck for designing statistical mechanisms is however its dependence on appropriately annotated training data. None of these methodologies are suitable for mining information from online communication text data due to the fact that they are often noisy. These texts are informally written. They suffer from spelling mistakes, grammatical errors, improper punctuation and irrational capitalization. This paper focuses on opinion extraction from noisy text data. It is aimed at extracting and consolidating opinions of customers from blogs and feedbacks, at multiple levels of granularity. Ours is a hybrid approach, in which we initially employ a semi-supervised method to learn domain knowledge from a training repository which contains both noisy and clean text. Thereafter we employ localized linguistic techniques to extract opinion expressions from noisy text. We have developed a system based on this approach, which provides the user with a platform to analyze opinion expressions extracted from a repository.

REFERENCES

• 1: Pavel Smrž, "Using WordNet for Opinion Mining", GWC 2006 Proceedings, pp. 333--335, 2006
• 2: Tetsuya Nasukawa, Diwakar Punjani, Shourya Roy, L. Venkata Subramaniam, Hironori Takeuchi, "Adding Sentence Boundaries to Conversational Speech Transcriptions using Noisily Labelled Examples", Proc. IJCAI-2007 Workshop on Analytics for Noisy Unstructured Text, pp. 71--78, India, 2007
• 3: Wilson Wong , Wei Liu , Mohammed Bennamoun, Integrated scoring for spelling error correction, abbreviation expansion and case restoration in dirty text, Proceedings of the fifth Australasian conference on Data mining and analystics, p.83-89, November 29-30, 2006, Sydney, Australia
• 4: Wilson Wong, Wei Liu and Mohammed Bennamoun, "Enhanced Integrated Scoring for Cleaning Dirty Texts", Proc. IJCAI-2007 Workshop on Analytics for Noisy Unstructured Text, pp. 55--62, India, 2007
• 5: Minqing Hu , Bing Liu, Mining and summarizing customer reviews, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1014073]
• 6: Minqing Hu and Bing Liu, "Mining Opinion Features in Customer Reviews", American Association for Artificial Intelligence, 2004
• 7: Andrea Esuli and Fabrizio Sebastiani, "SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining", LREC2006 Conference on Language Resources and Evaluation, Genova, 2006
• 8: Ana-Maria Popescu , Oren Etzioni, Extracting product features and opinions from reviews, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, p.339-346, October 06-08, 2005, Vancouver, British Columbia, Canada [doi>10.3115/1220575.1220618]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Information filtering

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Text analysis
  I.5 PATTERN RECOGNITION
      I.5.4 Applications
          Subjects: Text processing

General Terms:
Algorithms, Experimentation

Keywords:
customer feedbacks, noisy text, opinion mining, wordnet