Mining the peanut gallery: opinion extraction and semantic classification of product reviews

ABSTRACT

The web contains a wealth of product reviews, but sifting through them is a daunting task. Ideally, an opinion mining tool would process a set of search results for a given item, generating a list of product attributes (quality, features, etc.) and aggregating opinions about each of them (poor, mixed, good). We begin by identifying the unique properties of this problem and develop a method for automatically distinguishing between positive and negative reviews. Our classifier draws on information retrieval techniques for feature extraction and scoring, and the results for various metrics and heuristics vary depending on the testing situation. The best methods work as well as or better than traditional machine learning. When operating on individual sentences collected from web searches, performance is limited due to noise and ambiguity. But in the context of a complete web-based tool and aided by a simple method for grouping sentences into attributes, the results are qualitatively quite useful.

REFERENCES

• 1: Sanjiv Ranjan Das and Mike Y. Chen. Yahoo! for Amazon: Sentiment parsing from small talk on the web. Proceedings of the 8th Asia Pacific Finance Association Annual Conference, 2001.
• 2: Aidan Finn , Nicholas Kushmerick , Barry Smyth, Genre Classification and Domain Transfer for Information Filtering, Proceedings of the 24th BCS-IRSG European Colloquium on IR Research: Advances in Information Retrieval, p.353-362, March 25-27, 2002
• 3: W. Gale. Good-Turing smoothing without tears. Journal of Quantitative Linguistics, 2:217--37, 1995.
• 4: Vasileios Hatzivassiloglou , Kathleen R. McKeown, Predicting the semantic orientation of adjectives, Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics, p.174-181, July 07-12, 1997, Madrid, Spain
• 5: Vasileios Hatzivassiloglou , Janyce M. Wiebe, Effects of adjective orientation and gradability on sentence subjectivity, Proceedings of the 18th conference on Computational linguistics, p.299-305, July 31-August 04, 2000, Saarbrücken, Germany [doi>10.3115/990820.990864]
• 6: M. Hearst. Direction-Based Text Interpretation as an Information Access Refinement. 1992.
• 7: David Holtzmann. Detecting and tracking opinions in on-line discussions. UCB/SIMS Web Mining Workshop, 2001.
• 8: Dekang Lin, Automatic retrieval and clustering of similar words, Proceedings of the 17th international conference on Computational linguistics, p.768-774, August 10-14, 1998, Montreal, Quebec, Canada
• 9: Andrew Kachites McCallum. Bow: A toolkit for statistical language modeling, text retrieval, classification and clustering. http://www.cs.cmu.edu/~mccallum/bow, 1996.
• 10: Hugo Liu , Henry Lieberman , Ted Selker, A model of textual affect sensing using real-world knowledge, Proceedings of the 8th international conference on Intelligent user interfaces, January 12-15, 2003, Miami, Florida, USA [doi>10.1145/604045.604067]
• 11: Dunja Mladenic, Feature Subset Selection in Text-Learning, Proceedings of the 10th European Conference on Machine Learning, p.95-100, April 21-23, 1998
• 12: R. Mooney, P. Bennett, and L. Roy. Book recommending using text categorization with extracted information. Proceedings of the AAAI Workshop on Recommender Systems, 1998.
• 13: Satoshi Morinaga , Kenji Yamanishi , Kenji Tateishi , Toshikazu Fukushima, Mining product reputations on the Web, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, July 23-26, 2002, Edmonton, Alberta, Canada [doi>10.1145/775047.775098]
• 14: Bo Pang , Lillian Lee , Shivakumar Vaithyanathan, Thumbs up?: sentiment classification using machine learning techniques, Proceedings of the ACL-02 conference on Empirical methods in natural language processing, p.79-86, July 06, 2002 [doi>10.3115/1118693.1118704]
• 15: Fernando Pereira, Yoram Singer, and Naftali Tishby. Beyond word N-grams. In David Yarovsky and Kenneth Church, editors, Proceedings of the Third Workshop on Very Large Corpora, pages 95--106, Somerset, New Jersey, 1995. Association for Computational Linguistics.
• 16: Fernando Pereira , Naftali Tishby , Lillian Lee, Distributional clustering of English words, Proceedings of the 31st annual meeting on Association for Computational Linguistics, p.183-190, June 22-26, 1993, Columbus, Ohio [doi>10.3115/981574.981598]
• 17: M.F. Porter. An algorithm for suffix stripping. Program, 14(3):130--137, 1980. http://www.tartarus.org/~martin/PorterStemmer/.
• 18: Deepak Ravichandran , Eduard Hovy, Learning surface text patterns for a Question Answering system, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073092]
• 19: Ellen Riloff. Automatically generating extraction patterns from untagged text. Proceedings of AAAI/IAAI, Vol. 2, pages 1044--1049, 1996.
• 20: P. Subasic and A. Huettner. Affect analysis of text using fuzzy semantic typing. IEEE-FS, 9:483--496, Aug. 2001.
• 21: Loren Terveen , Will Hill , Brian Amento , David McDonald , Josh Creter, PHOAKS: a system for sharing recommendations, Communications of the ACM, v.40 n.3, p.59-62, March 1997 [doi>10.1145/245108.245122]
• 22: Richard M. Tong. An operational system for detecting and tracking opinions in on-line discussion. SIGIR Workshop on Operational Text Classifiation, 2001.
• 23: P.D. Turney and M.L. Littman. Unsupervised learning of semantic orientation from a hundred-billion-word corpus. Technical Report ERB-1094, National Research Council Canada, Institute for Information Technology, 2002.
• 24: Janyce Wiebe, Learning Subjective Adjectives from Corpora, Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence, p.735-740, July 30-August 03, 2000
• 25: Janyce Wiebe , Rebecca Bruce , Matthew Bell , Melanie Martin , Theresa Wilson, A corpus study of evaluative and speculative language, Proceedings of the Second SIGdial Workshop on Discourse and Dialogue, p.1-10, September 01-02, 2001, Aalborg, Denmark [doi>10.3115/1118078.1118104]
• 26: Janyce Wiebe, Theresa Wilson, and Matthew Bell. Identifying collocations for recognizing opinions. Proceedings of ACL/EACL 2001 Workshop on Collocation.
• 27: Mikio Yamamoto and Kenneth Church. Using suffix arrays to compute term frequency and document frequency for all substrings in a corpus. Proceedings of the 6th Workshop on Very Large Corpora.

INDEX TERMS

General Terms:
Algorithms, Measurement

Keywords:
document classification, opinion mining