Opinion retrieval from blogs

ABSTRACT

Opinion retrieval is a document retrieval process, which requires documents to be retrieved and ranked according to their opinions about a query topic. A relevant document must satisfy two criteria: relevant to the query topic, and contains opinions about the query, no matter if they are positive or negative. In this paper, we describe an opinion retrieval algorithm. It has a traditional information retrieval (IR) component to find topic relevant documents from a document set, an opinion classification component to find documents having opinions from the results of the IR step, and a component to rank the documents based on their relevance to the query, and their degrees of having opinions about the query. We implemented the algorithm as a working system and tested it using TREC 2006 Blog Track data in automatic title-only runs. Our result showed 28% to 32% improvements in MAP score over the best automatic runs in this 2006 track. Our result is also 13% higher than a state-of-art opinion retrieval system, which is tested on the same data set.

REFERENCES

• 1: H. Chernoff and E. Lehmann. The use of maximum likelihood estimates in _χ² tests for goodness-of-fit. The Annals of Mathematical Statistics. 1954.
• 2: S. Das, and M. Chen. Yahoo! for Amazon: Extracting market sentiment from stock message boards. APFA, 2001.
• 3: Kushal Dave , Steve Lawrence , David M. Pennock, Mining the peanut gallery: opinion extraction and semantic classification of product reviews, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775226]
• 4: Andrea Esuli , Fabrizio Sebastiani, Determining the semantic orientation of terms through gloss classification, Proceedings of the 14th ACM international conference on Information and knowledge management, October 31-November 05, 2005, Bremen, Germany [doi>10.1145/1099554.1099713]
• 5: Vasileios Hatzivassiloglou , Kathleen R. McKeown, Predicting the semantic orientation of adjectives, Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics, p.174-181, July 07-12, 1997, Madrid, Spain
• 6: Vasileios Hatzivassiloglou , Janyce M. Wiebe, Effects of adjective orientation and gradability on sentence subjectivity, Proceedings of the 18th conference on Computational linguistics, p.299-305, July 31-August 04, 2000, Saarbrücken, Germany [doi>10.3115/990820.990864]
• 7: http://blogsearch.google.com
• 8: http://en.wikipedia.org
• 9: Minqing Hu , Bing Liu, Mining and summarizing customer reviews, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1014073]
• 10: Nitin Jindal , Bing Liu, Identifying comparative sentences in text documents, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148215]
• 11: Thorsten Joachims, Making large-scale support vector machine learning practical, Advances in kernel methods: support vector learning, MIT Press, Cambridge, MA, 1999
• 12: Soo-Min Kim , Eduard Hovy, Determining the sentiment of opinions, Proceedings of the 20th international conference on Computational Linguistics, p.1367-es, August 23-27, 2004, Geneva, Switzerland [doi>10.3115/1220355.1220555]
• 13: N. Kobayashi, R. Iida, K. Inui, and Y. Matsumoto. Opinion mining on the Web by extracting subject-attribute-value relations. AAAI-CAAW'06, 2006.
• 14: Lun-Wei Ku , Li-Ying Lee , Tung-Ho Wu , Hsin-Hsi Chen, Major topic detection and its application to opinion summarization, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076161]
• 15: Bing Liu , Minqing Hu , Junsheng Cheng, Opinion observer: analyzing and comparing opinions on the Web, Proceedings of the 14th international conference on World Wide Web, May 10-14, 2005, Chiba, Japan [doi>10.1145/1060745.1060797]
• 16: Shuang Liu , Fang Liu , Clement Yu , Weiyi Meng, An effective approach to document retrieval via utilizing WordNet and recognizing phrases, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009039]
• 17: I. Ounis, M. de Rijke, C. Macdonald, G. Mishne, and I. Soboroff. Overview of the TREC-2006 Blog Track. In Proceedings of TREC. 2006.
• 18: Bo Pang , Lillian Lee , Shivakumar Vaithyanathan, Thumbs up?: sentiment classification using machine learning techniques, Proceedings of the ACL-02 conference on Empirical methods in natural language processing, p.79-86, July 06, 2002 [doi>10.3115/1118693.1118704]
• 19: Bo Pang , Lillian Lee, Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales, Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, p.115-124, June 25-30, 2005, Ann Arbor, Michigan [doi>10.3115/1219840.1219855]
• 20: Ana-Maria Popescu , Oren Etzioni, Extracting product features and opinions from reviews, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, p.339-346, October 06-08, 2005, Vancouver, British Columbia, Canada [doi>10.3115/1220575.1220618]
• 21: S. Robertson, S. Walker Okapi/Keenbow at TREC-8, 1999.
• 22: R. Tong. An operational system for detecting and tracking opinions in on-line discussions. SIGIR 2001 Workshop on Operational Text Classification, 2001.
• 23: Peter D. Turney, Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073153]
• 24: Ellen M. Voorhees. Overview of the TREC 2005 Robust Retrieval Track. In Proceedings of TREC-2005.
• 25: Casey Whitelaw , Navendu Garg , Shlomo Argamon, Using appraisal groups for sentiment analysis, Proceedings of the 14th ACM international conference on Information and knowledge management, October 31-November 05, 2005, Bremen, Germany [doi>10.1145/1099554.1099714]
• 26: Janyce Wiebe , Theresa Wilson , Rebecca Bruce , Matthew Bell , Melanie Martin, Learning Subjective Language, Computational Linguistics, v.30 n.3, p.277-308, September 2004 [doi>10.1162/0891201041850885]
• 27: Yiming Yang , Jan O. Pedersen, A Comparative Study on Feature Selection in Text Categorization, Proceedings of the Fourteenth International Conference on Machine Learning, p.412-420, July 08-12, 1997
• 28: Hong Yu , Vasileios Hatzivassiloglou, Towards answering opinion questions: separating facts from opinions and identifying the polarity of opinion sentences, Proceedings of the 2003 conference on Empirical methods in natural language processing, p.129-136, July 11, 2003 [doi>10.3115/1119355.1119372]
• 29: G. Mishne. Multiple ranking strategies for opinion retrieval in blogs. In TREC 2006, 2006.
• 30: Gilad Mishne. Using Blog Properties to Improve Retrieval. In proceedings of ICWSM. 2007.
• 31: Wei Zhang and Clement Yu. UIC at TREC 2006 Blog Track. In proceedings of 15th TREC. 2006.
• 32: K. Yang, N. Yu, A. Valerio and H. Zhang. WIDIT in TREC 2006 Blog Track. In proceedings of 15th TREC. 2006.
• 33: Douglas Oard, Tamer Elsayed, Jianqiang Wang, Yejun Wu, Pengyi Zhang, Eileen Abels, Jimmy Lin and Dagbert Soergel. TREC-2006 at Maryland: Blog, Enterprise, Legal and QA Tracks. In proceedings of 15th TREC. 2006.
• 34: Wei Zhang , Shuang Liu , Clement Yu , Chaojing Sun , Fang Liu , Weiyi Meng, Recognition and classification of noun phrases in queries for effective retrieval, Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, November 06-10, 2007, Lisbon, Portugal [doi>10.1145/1321440.1321540]
• 35: Jinxi Xu , W. Bruce Croft, Query expansion using local and global document analysis, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.4-11, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243202]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Selection process

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Text analysis

General Terms:
Algorithms, Experimentation, Performance

Keywords:
blog retrieval, information retrieval, opinion retrieval, trec