Exploring the characteristics of opinion expressions for political opinion classification

ABSTRACT

Recently there has been increasing interest in constructing general-purpose political opinion classifiers for applications in e-Rulemaking. This problem is generally modeled as a sentiment classification task in a new domain. However, the classification accuracy is not as good as that in other domains such as customer reviews. In this paper, we report the results of a series of experiments designed to explore the characteristics of political opinion expression which might affect the sentiment classification performance. We found that the average sentiment level of Congressional debate is higher than that of neutral news articles, but lower than that of movie reviews. Also unlike the adjective-centered sentiment expression in movie reviews, the choice of topics, as reflected in nouns, serves as an important mode of political opinion expression. Manual annotation results demonstrate that a significant number of political opinions are expressed in neutral tones. These characteristics suggest that recognizing the sentiment is not enough for political opinion classification. Instead, what seems to be needed is a more fine-grained model of individuals' ideological positions and the different ways in which those positions manifest themselves in political discourse.

REFERENCES

• 1: Rakesh Agrawal , Sridhar Rajagopalan , Ramakrishnan Srikant , Yirong Xu, Mining newsgroups using networks arising from social behavior, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775227]
• 2: Biber, D. (1988). Variation across speech and writing. Cambridge University Press.
• 3: Eric Brill, Transformation-based error-driven learning and natural language processing: a case study in part-of-speech tagging, Computational Linguistics, v.21 n.4, p.543-565, December 1995
• 4: Converse, P. E. (1964). The nature of belief systems in mass publics." In Ideology and Discontent, edited by D. E. Apter. New York: Free Press.
• 5: Diermeier, D., Godbout, J-F, Kaufmann, S., and Yu, B. (2007). Language and ideology in Congress. MPSA 2007, Chicago
• 6: Kushal Dave , Steve Lawrence , David M. Pennock, Mining the peanut gallery: opinion extraction and semantic classification of product reviews, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775226]
• 7: Susan Dumais , John Platt , David Heckerman , Mehran Sahami, Inductive learning algorithms and representations for text categorization, Proceedings of the seventh international conference on Information and knowledge management, p.148-155, November 02-07, 1998, Bethesda, Maryland, United States [doi>10.1145/288627.288651]
• 8: Durant, K. T. & Smith M. D. (2006). Mining sentiment classification from political web logs. Proceedings of workshop on Web Mining and Web Usage Analysis of the 12th ACM SIGKDD international conference on Knowledge Discovery and Data Mining (WebKDD2006)
• 9: Esuli, A. (2006). A bibliography on sentiment classification. http://liinwww.ira.uka.de/bibliography/Misc/Sentiment.html (last visited: 10/31/2007)
• 10: George Forman, An extensive empirical study of feature selection metrics for text classification, The Journal of Machine Learning Research, 3, 3/1/2003
• 11: Isabelle Guyon , Jason Weston , Stephen Barnhill , Vladimir Vapnik, Gene Selection for Cancer Classification using Support Vector Machines, Machine Learning, v.46 n.1-3, p.389-422, 2002 [doi>10.1023/A:1012487302797]
• 12: Vasileios Hatzivassiloglou , Kathleen R. McKeown, Predicting the semantic orientation of adjectives, Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics, p.174-181, July 07-12, 1997, Madrid, Spain
• 13: Vasileios Hatzivassiloglou , Janyce M. Wiebe, Effects of adjective orientation and gradability on sentence subjectivity, Proceedings of the 18th conference on Computational linguistics, p.299-305, July 31-August 04, 2000, Saarbrücken, Germany [doi>10.3115/990820.990864]
• 14: Minqing Hu , Bing Liu, Mining and summarizing customer reviews, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1014073]
• 15: Thorsten Joachims, Text Categorization with Suport Vector Machines: Learning with Many Relevant Features, Proceedings of the 10th European Conference on Machine Learning, p.137-142, April 21-23, 1998
• 16: Koppel, M. and Schler, J. (2006). The importance of neutral examples for learning sentiment. Computational Intelligence 22(2), 100--109
• 17: Namhee Kwon , Liang Zhou , Eduard Hovy , Stuart W. Shulman, Identifying and classifying subjective claims, Proceedings of the 8th annual international conference on Digital government research: bridging disciplines & domains, May 20-23, 2007, Philadelphia, Pennsylvania
• 18: Laver, M., Benoit, K., & Garry, J. (2003). Extracting policy positions from political texts using words as data." American Political Science Review 97(2), 311--337
• 19: Mullen, T. and Malouf R. 2006. A preliminary investigation into sentiment analysis of informal political discourse. In Proceedings of the AAAI Symposium on Computational Approaches to Analyzing Weblogs (). 159--162. DOI=
• 20: Ounis, I., de Rijke, M., Macdonald, C., Mishne, G. and Soboroff, I. (2007). Overview of the TREC-2006 blog track. Proceedings of the 15th Text REtrieval Conference. NIST
• 21: Bo Pang , Lillian Lee , Shivakumar Vaithyanathan, Thumbs up?: sentiment classification using machine learning techniques, Proceedings of the ACL-02 conference on Empirical methods in natural language processing, p.79-86, July 06, 2002 [doi>10.3115/1118693.1118704]
• 22: Bo Pang , Lillian Lee, A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts, Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p.271-es, July 21-26, 2004, Barcelona, Spain [doi>10.3115/1218955.1218990]
• 23: Bo Pang , Lillian Lee, Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales, Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, p.115-124, June 25-30, 2005, Ann Arbor, Michigan [doi>10.3115/1219840.1219855]
• 24: Pennebaker, J. W. & Francis, M. E. (1999). Linguistic Inquiry and Word Count (LIWC). Mahwah, NJ: LEA Software and Alternative Media/Erlbaum
• 25: Pennebaker, J. W., Mehl, M. R., Niederhoffer, K. G. (2003). Psychological aspects of natural language use: our words, our selves. Annu. Rev. Psychol. 54, 547--77.
• 26: Ellen Riloff , Janyce Wiebe , Theresa Wilson, Learning subjective nouns using extraction pattern bootstrapping, Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003, p.25-32, May 31, 2003, Edmonton, Canada [doi>10.3115/1119176.1119180]
• 27: Schonhardt-Bailey, C. (2008). The Congressional debate on partial-birth abortion: constitutional gravitas and moral passion. British Journal of Political Science. Forthcoming
• 28: Stone, P. J. (1966). The General Inquirer: A computer approach to content analysis. MIT Press
• 29: Thomas, M., Pang, B., & Lee, L. (2006). Get out the vote: Determining support or opposition from Congressional floor-debate transcripts. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP2006), 327--335
• 30: Peter D. Turney, Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073153]
• 31: Peter D. Turney , Michael L. Littman, Measuring praise and criticism: Inference of semantic orientation from association, ACM Transactions on Information Systems (TOIS), v.21 n.4, p.315-346, October 2003 [doi>10.1145/944012.944013]
• 32: Whissell, C. M. (1989). The dictionary of affect in language. Journal of Emotion: Theory, Research and Experience, vol 4, 113--131
• 33: Janyce Wiebe, Learning Subjective Adjectives from Corpora, Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence, p.735-740, July 30-August 03, 2000
• 34: Wilson, T., Wiebe, J. and Hwa, R. (2006). Recognizing strong and weak opinion clauses. Computational Intelligence, 22(2), 73--99
• 35: Janyce Wiebe , Theresa Wilson , Rebecca Bruce , Matthew Bell , Melanie Martin, Learning Subjective Language, Computational Linguistics, v.30 n.3, p.277-308, September 2004 [doi>10.1162/0891201041850885]
• 36: Wiebe, J. and Riloff, E. (2005). Creating subjective and objective sentence classifiers from un-annotated corpus. Proceedings of the 6th International Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2005)
• 37: Yiming Yang , Xin Liu, A re-examination of text categorization methods, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.42-49, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312647]
• 38: Yu, B. (forthcoming). An evaluation of text classification methods for literary study. Journal of Literary and Linguistic Computing

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Linguistic processing

General Terms:
Algorithms, Design, Experimentation, Measurement

Keywords:
e-rulemaking, feature selection, machine learning, sentiment classification, text categorization