Finding high-quality content in social media

ABSTRACT

The quality of user-generated content varies drastically from excellent to abuse and spam. As the availability of such content increases, the task of identifying high-quality content sites based on user contributions --social media sites -- becomes increasingly important. Social media in general exhibit a rich variety of information sources: in addition to the content itself, there is a wide array of non-content information available, such as links between items and explicit quality ratings from members of the community. In this paper we investigate methods for exploiting such community feedback to automatically identify high quality content. As a test case, we focus on Yahoo! Answers, a large community question/answering portal that is particularly rich in the amount and types of content and social interactions available in it. We introduce a general classification framework for combining the evidence from different sources of information, that can be tuned automatically for a given social media type and quality definition. In particular, for the community question/answering domain, we show that our system is able to separate high-quality items from the rest with an accuracy close to that of humans

REFERENCES

• 1: Eugene Agichtein , Eric Brill , Susan Dumais , Robert Ragno, Learning user interaction models for predicting web search result preferences, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148175]
• 2: Kamal Ali , Mark Scarr, Robust methodologies for modeling web click distributions, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242642]
• 3: Chris Anderson, The Long Tail: Why the Future of Business Is Selling Less of More, Hyperion, 2006
• 4: Y. Attali and J. Burstein. Automated essay scoring with e-rater v. 2. Journal of Technology, Learning, and Assessment, 4(3), February 2006.
• 5: Adam L. Berger , Vincent J. Della Pietra , Stephen A. Della Pietra, A maximum entropy approach to natural language processing, Computational Linguistics, v.22 n.1, p.39-71, March 1996
• 6: Jill Burstein , Karen Kukich , Susanne Wolff , Chi Lu , Martin Chodorow , Lisa Braden-Harder , Mary Dee Harris, Automated scoring using a hybrid feature identification technique, Proceedings of the 17th international conference on Computational linguistics, August 10-14, 1998, Montreal, Quebec, Canada
• 7: Jill Burstein , Magdalena Wolska, Toward evaluation of writing style: finding overly repetitive word use in student essays, Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics, April 12-17, 2003, Budapest, Hungary [doi>10.3115/1067807.1067814]
• 8: Christopher S. Campbell , Paul P. Maglio , Alex Cozzi , Byron Dom, Expertise identification using email communications, Proceedings of the twelfth international conference on Information and knowledge management, November 03-08, 2003, New Orleans, LA, USA [doi>10.1145/956863.956965]
• 9: Martin Chodorow , Claudia Leacock, An unsupervised method for detecting grammatical errors, Proceedings of the first conference on North American chapter of the Association for Computational Linguistics, p.140-147, April 29-May 04, 2000, Seattle, Washington
• 10: M. Collins and Y. Singer. Unsupervised models for named entity classification. In Natural Language Processing and Very Large Corpora, 1999.
• 11: Byron Dom , Iris Eiron , Alex Cozzi , Yi Zhang, Graph-based ranking algorithms for e-mail expertise analysis, Proceedings of the 8th ACM SIGMOD workshop on Research issues in data mining and knowledge discovery, June 13-13, 2003, San Diego, California [doi>10.1145/882082.882093]
• 12: Danyel Fisher , Marc Smith , Howard T. Welser, You Are Who You Talk To: Detecting Roles in Usenet Newsgroups, Proceedings of the 39th Annual Hawaii International Conference on System Sciences, p.59.2, January 04-07, 2006 [doi>10.1109/HICSS.2006.536]
• 13: Jerome H. Friedman, Stochastic gradient boosting, Computational Statistics & Data Analysis, v.38 n.4, p.367-378, 28 February 2002 [doi>10.1016/S0167-9473(01)00065-2]
• 14: R. Guha , Ravi Kumar , Prabhakar Raghavan , Andrew Tomkins, Propagation of trust and distrust, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988727]
• 15: R. Gunning. The technique of clear writing. McGraw-Hill, 1952.
• 16: F. Heylighen and J.-M. Dewaele. Variation in the contextuality of language: An empirical measure. Context in Context. Special issue Foundations of Science,7(3):293--340, 2002.
• 17: Jiwoon Jeon , W. Bruce Croft , Joon Ho Lee , Soyeon Park, A framework to predict the quality of answers with non-textual features, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148212]
• 18: Thorsten Joachims , Laura Granka , Bing Pan , Helene Hembrooke , Geri Gay, Accurately interpreting clickthrough data as implicit feedback, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076063]
• 19: Pawel Jurczyk , Eugene Agichtein, Discovering authorities in question answer communities by using link analysis, Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, November 06-10, 2007, Lisbon, Portugal [doi>10.1145/1321440.1321575]
• 20: Pawel Jurczyk , Eugene Agichtein, Hits on question answer portals: exploration of link analysis for author ranking, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277938]
• 21: J. P. Kincaid, R. P. Fishburn, R. L. Rogers, and B. S. Chissom. Derivation of new readability formulas for navy enlisted personnel. Technical Report Research Branch Report 8-75, Millington, Tenn, Naval Air Station, 1975.
• 22: Jon M. Kleinberg, Authoritative sources in a hyperlinked environment, Journal of the ACM (JACM), v.46 n.5, p.604-632, Sept. 1999 [doi>10.1145/324133.324140]
• 23: G. H. McLaughlin. SMOG grading: A new readability formula. Journal of Reading, 12(8):639--646, 1969.
• 24: E. B. Page. Computer grading of student prose, using modern concepts and software. Journal of Experimental Education, 62(2), 1994.
• 25: L. Page, S. Brin, R. Motwani, and T. Winograd. The PageRank citation ranking: bringing order to the Web. Technical report, Stanford Digital Library Technologies Project, 1998.
• 26: Bo Pang , Lillian Lee , Shivakumar Vaithyanathan, Thumbs up?: sentiment classification using machine learning techniques, Proceedings of the ACL-02 conference on Empirical methods in natural language processing, p.79-86, July 06, 2002 [doi>10.3115/1118693.1118704]
• 27: L. Prescott. Yahoo! Answers captures 96% of Q and A market share, 2006.
• 28: L. M. Rudner and T. Liang. Automated essay scoring using bayes. Journal of Technology, Learning, and Assessment, 1(2), June 2002.
• 29: C. Sang-Hun. To outdo Google, Naver taps into Korea's collective wisdom. International Herald Tribune, July 4 2007.
• 30: J. P. Scott. Social Network Analysis: A Handbook. SAGE Publications, January 2000.
• 31: Qi Su , Dmitry Pavlov , Jyh-Herng Chow , Wendell C. Baker, Internet-scale collection of human-reviewed data, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242604]
• 32: Jun Zhang , Mark S. Ackerman , Lada Adamic, Expertise networks in online communities: structure and algorithms, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242603]
• 33: Cai-Nicolas Ziegler , Georg Lausen, Propagation Models for Trust and Distrust in Social Networks, Information Systems Frontiers, v.7 n.4-5, p.337-358, December 2005 [doi>10.1007/s10796-005-4807-3]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Linguistic processing

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process; Information filtering

Keywords:
community question answering, media, user interactions