Assessment of conversation co-mentions as a resource for software module recommendation

ABSTRACT

Conversation double pivots recommend target items related to a source item, based on co-mentions of source and target items in online forums. We deployed several variants on the drupal.org site that supports the Drupal open source community, and assessed them through clickthrough rates. A similarity metric based on correlation of mentions rather than mere co-occurrence reduced the problem of over-recommending the most popular modules, but additional corrections for recency and uniqueness of mentions were not helpful. Detection of more module mentions in conversations dramatically improved the quality of recommendations, even though the detection algorithm then had more false positives. Recommendations based on conversation co-mention were more effective than those based on co-installation, because co-installation data only led to recommendations of complementary modules and not substitutes. Recommendations based on co-mention were more effective than those based on text similarity matching for navigating from the most popular modules, but less effective than text matching for less popular modules.

REFERENCES

• 1: Bennett, J., and Lanning, J. 2007. The Netflix Prize. In Proceedings of KDD Cup and Workshop, 2007.
• 2: Breese, J., Heckerman, D., and Kadie, C. 1998. Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of 14th Conference on the Uncertainty in Artificial Intelligence
• 3: Brin, S., and Page, L. 1998. The anatomy of a large-scale hypertextual Web search engine. Computer networks and ISDN systems, 30(1-7), 107--117
• 4: Drenner, S., Harper, M., Frankowski, D., Riedl, J., and Terveen, L. 2006. Insert movie reference here: a system to bridge conversation and item-oriented web sites. In Proceedings of the SIGCHI Conference.
• 5: Herlocker, J. L., Konstan, J. A., Terveen, L. G., and Riedl, J. T. 2004. Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. 22, 1 (Jan. 2004), 5--53.
• 6: Joachims, T., Granka, L., Pan, B., Hembrooke, H., and Gay, G. 2005. Accurately interpreting clickthrough data as implicit feedback. In Proc. of the 28th ACM SIGIR Conf.
• 7: Linden, G., Smith, B., and York, J. 2003. Amazon.com Recommendations: Item-to-Item Collaborative Filtering. IEEE Internet Computing 7, 1 (Jan. 2003), 76--80
• 8: Morita, M. and Shinoda, Y. 1994. Information filtering based on user behavior analysis and best match text retrieval. In Proceedings of the 17th ACM SIGIR Conference on Research and Development in information Retrieval.
• 9: Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., and Riedl, J. 1994. GroupLens: an open architecture for collaborative filtering of netnews. In Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work.
• 10: Salton, G., Wong, A., and Yang, C. S. 1975. A vector space model for automatic indexing. Commun. ACM 18, 11 (Nov. 1975), 613--620.
• 11: Sarwar, B., Karypis, G., Konstan, J., and Reidl, J. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th international Conference on World Wide Web.
• 12: Small, H. 1973. Co-citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for Information Science 24, 4.
• 13: Terveen, L., Hill, W., Amento, B., McDonald, D., and Creter, J. 1997. PHOAKS: a system for sharing recommendations. Commun. ACM 40, 3 (Mar. 1997), 59--62
• 14: Zhou, D., Oostendorp, N., Hess, M., and Resnick, P. 2008. Conversation pivots and double pivots. In Proc. of the 26th SIGCHI Conf. on Human Factors in Computing Systems.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Design

Keywords:
Drupal, conversation double pivot, evaluation, item-item recommender