Automatic scoring of online discussion posts

ABSTRACT

Online discussions forums, known as forums for short, are conversational social cyberspaces constituting rich repositories of content and an important source of collaborative knowledge. However, most of this knowledge is buried inside the forum infrastructure and its extraction is both complex and difficult. The ability to automatically rate postings in online discussion forums, based on the value of their contribution, enhances the ability of users to find knowledge within this content. Several key online discussion forums have utilized collaborative intelligence to rate the value of postings made by users. However, a large percentage of posts go unattended and hence lack appropriate rating.

In this paper, we focus on automatic rating of postings in online discussion forums. A set of features derived from the posting content and the threaded discussion structure are generated for each posting. These features are grouped into five categories, namely (i) relevance, (ii) originality, (iii) forum-specific features, (iv) surface features, and (v) posting-component features. Using a non-linear SVM classifier, the value of each posting is categorized into one of three levels High, Medium, or Low. This rating represents a seed value for each posting that is leveraged in filtering forum content. Experimental results have shown promising performance on forum data.

REFERENCES

• 1: Christian Borgs , Jennifer Chayes , Mohammad Mahdian , Amin Saberi, Exploring the community structure of newsgroups, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1016914]
• 2: Chang, C., and Lin, C. 2001. LibSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
• 3: Dikli, S., 2006. An Overview of Automatic Scoring of Essays. The Journal of Technology, Learning, and Assessment, Vol 5(1) August 2006, 3--35.
• 4: Andrew T. Fiore , Scott Lee Tiernan , Marc A. Smith, Observed behavior and perceived value of authors in usenet newsgroups: bridging the gap, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503434]
• 5: Danyel Fisher , Marc Smith , Howard T. Welser, You Are Who You Talk To: Detecting Roles in Usenet Newsgroups, Proceedings of the 39th Annual Hawaii International Conference on System Sciences, p.59.2, January 04-07, 2006 [doi>10.1109/HICSS.2006.536]
• 6: Blaz Fortuna , Eduarda Mendes Rodrigues , Natasa Milic-Frayling, Improving the classification of newsgroup messages through social network analysis, Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, November 06-10, 2007, Lisbon, Portugal [doi>10.1145/1321440.1321565]
• 7: Natalie Glance , Matthew Hurst , Kamal Nigam , Matthew Siegler , Robert Stockton , Takashi Tomokiyo, Deriving marketing intelligence from online discussion, Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, August 21-24, 2005, Chicago, Illinois, USA [doi>10.1145/1081870.1081919]
• 8: Vicenç Gómez , Andreas Kaltenbrunner , Vicente López, Statistical analysis of the social network and discussion threads in slashdot, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367585]
• 9: Cliff Lampe , Paul Resnick, Slash(dot) and burn: distributed moderation in a large online conversation space, Proceedings of the SIGCHI conference on Human factors in computing systems, p.543-550, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985761]
• 10: Lui, A., Li, S., and Choy, S. 2007. An Evaluation of Automatic Text Categorization in Online Discussion Analysis. In Proceedings of the Seventh IEEE International Conference on Advanced Learning Technologies (Niigata, Japan, July 18-20, 2007) ICALT 2007, IEEE Computer Society Press, New Jersey, NJ, 205--209
• 11: Weimer, M., Gurevych, I., and Mühlhäuser, M. 2007. Automatically Assessing the Post Quality in Online Discussions on Software. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (Prague, Czech Republic, June 23-30, 2007). ACL2007 Volume P07-2, 125--128.
• 12: Wu, Q., Burges, C. Svore, K, and Gao, J, 2008, Ranking, Boosting, and Model Adaptation, Technical Report, MSR-TR-2008-109, Microsoft Corporation, Redmond, WA, August 2008.
• 13: Zhi-Li Wu , Chun-hung Li, Topic Detection in Online Discussion Using Non-negative Matrix Factorization, Proceedings of the 2007 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Workshops, p.272-275, November 02-05, 2007

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology

General Terms:
Algorithms, Human Factors, Theory

Keywords:
content filtering, forums, online communities