Deriving quantitative overviews of free text assessments on the web

ABSTRACT

Many research efforts are addressing the problem of enabling automatic summarization of opinions and assessments stated on the web in product reviews, discussion forums, and blogs. One key difficulty is that relevant assessments scattered throughout web pages are obscured by variations in natural language. In this paper, we focus on a novel aspect of enabling aggregations of assessments of degree to which a given property holds for a given entity (for instance, how touristy is Boston). We present GrainPile, a user interface for extracting from the web, aggregating and quantifying degree assessments of unconstrained topics. The interface provides a variety of functions: a) identification of dimensions of comparison (properties) relevant to a particular entity or set of entities, b) comparisons of like entities on user-specified properties (for example, which university is more prestigious, Yale or Cornell), c) tracing the derived opinions back to their sources (so that the reasons for the opinions can be found). A central contribution in GrainPile is the evaluated demonstration of feasibility of mapping the recognized expressions (such as fairly, very, extremely, and so on) to a common scale of numerical values and aggregating across all the extracted assessments to derive an overall assessment of degree. GrainPile's novel assessment and aggregation of degree expressions is shown to strongly outperform an interpretation-free, co-occurrence based method.

REFERENCES

• 1: Bierwisch, M.. The semantics of gradation. In Dimensional Adjectives, ed by Bierwisch, M. and Lang, E. Springer-Verlag, 1989.
• 2: Chklovski, T. and Pantel, P. 2004. VerbOcean: Mining the Web for Fine-Grained Semantic Verb Relations. In Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP-04). Barcelona, Spain.
• 3: Timothy Chklovski , Varun Ratnakar , Yolanda Gil, User interfaces with semi-formal representations: a study of designing argumentation structures, Proceedings of the 10th international conference on Intelligent user interfaces, January 10-13, 2005, San Diego, California, USA [doi>10.1145/1040830.1040864]
• 4: Cresswell, M. J. The semantics of degree. In Montague grammar, ed. by B. Partee. 261--292. Academic Press, New York, 1977.
• 5: Kushal Dave , Steve Lawrence , David M. Pennock, Mining the peanut gallery: opinion extraction and semantic classification of product reviews, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775226]
• 6: Oren Etzioni , Michael Cafarella , Doug Downey , Stanley Kok , Ana-Maria Popescu , Tal Shaked , Stephen Soderland , Daniel S. Weld , Alexander Yates, Web-scale information extraction in knowitall: (preliminary results), Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988687]
• 7: D. Gruhl , David Liben-Nowell , R. Guha , A. Tomkins, Information diffusion through blogspace, ACM SIGKDD Explorations Newsletter, v.6 n.2, p.43-52, December 2004 [doi>10.1145/1046456.1046462]
• 8: R. Guha , Ravi Kumar , Prabhakar Raghavan , Andrew Tomkins, Propagation of trust and distrust, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988727]
• 9: Minqing Hu , Bing Liu, Mining and summarizing customer reviews, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1014073]
• 10: Frank Keller , Mirella Lapata, Using the web to obtain frequencies for unseen bigrams, Computational Linguistics, v.29 n.3, p.459-484, September 2003 [doi>10.1162/089120103322711604]
• 11: Kennedy, C. and McNally, L. 2005. Scale Structure, Degree Modification, and the Semantics of Gradable Predicates. Language 81.2, 345--381.
• 12: Cody C. T. Kwok , Oren Etzioni , Daniel S. Weld, Scaling question answering to the Web, Proceedings of the 10th international conference on World Wide Web, p.150-161, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.371973]
• 13: Lehmann, E. L. and D'Abrera, H. J. M. Nonparametrics: Statistical Methods Based on Ranks, rev. ed. Englewood Cliffs, NJ: Prentice-Hall, 1998.
• 14: Bing Liu , Minqing Hu , Junsheng Cheng, Opinion observer: analyzing and comparing opinions on the Web, Proceedings of the 14th international conference on World Wide Web, May 10-14, 2005, Chiba, Japan [doi>10.1145/1060745.1060797]
• 15: Christopher D. Manning , Hinrich Schütze, Foundations of statistical natural language processing, MIT Press, Cambridge, MA, 1999
• 16: Miller, G. WordNet: An online lexical database. International Journal of Lexicography, 3(4), 1990.
• 17: Satoshi Morinaga , Kenji Yamanishi , Kenji Tateishi , Toshikazu Fukushima, Mining product reputations on the Web, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, July 23-26, 2002, Edmonton, Alberta, Canada [doi>10.1145/775047.775098]
• 18: Tetsuya Nasukawa , Jeonghee Yi, Sentiment analysis: capturing favorability using natural language processing, Proceedings of the 2nd international conference on Knowledge capture, October 23-25, 2003, Sanibel Island, FL, USA [doi>10.1145/945645.945658]
• 19: Nigam, K. and Hurst, M. Towards a Robust Metric of Polarity. In Shanahan, J., Qu, J. & Wiebe, J. (Eds.). Computing Attitude and Affect in Text. Springer, Dordrecht, The Netherlands. 2005.
• 20: Bo Pang , Lillian Lee , Shivakumar Vaithyanathan, Thumbs up?: sentiment classification using machine learning techniques, Proceedings of the ACL-02 conference on Empirical methods in natural language processing, p.79-86, July 06, 2002 [doi>10.3115/1118693.1118704]
• 21: Dragomir Radev , Weiguo Fan , Hong Qi , Harris Wu , Amardeep Grewal, Probabilistic question answering on the web, Proceedings of the 11th international conference on World Wide Web, May 07-11, 2002, Honolulu, Hawaii, USA [doi>10.1145/511446.511500]
• 22: Richardson, M., Agrawal, R. and Domingos, P. Trust Management for the Semantic Web, with. In Proceedings of the Second International Semantic Web Conference (pp. 351--368), 2003. Sanibel Island, FL: Springer.
• 23: Skowron, M., and Araki, K.. Voluntary Contributions of Unaware Internet Users? On Automatic Knowledge Retrieval from the WWW. In 2005 AAAI Spring Symposium on Knowledge Collection from Volunteer Cotnributors (KCVC'05).
• 24: Peter D. Turney, Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073153]
• 25: Vanderwende, L. Volunteers Created the Web. 2005 AAAI Spring Symposium on Knowledge Collection from Volunteer Cotnributors (KCVC'05).
• 26: Janyce M. Wiebe , Rebecca F. Bruce , Thomas P. O'Hara, Development and use of a gold-standard data set for subjectivity classifications, Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, p.246-253, June 20-26, 1999, College Park, Maryland [doi>10.3115/1034678.1034721]
• 27: Wilson, T., Wiebe, J., and Hwa, R. Just how mad are you? Finding strong and weak opinion clauses. In Proceedings of the Nineteenth National Conference on Artificial Intelligence (AAAI'04). 2004.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Text analysis

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

Keywords:
question answering, sentiment analysis, text extraction