Improving recommender systems with adaptive conversational strategies

ABSTRACT

Conversational recommender systems (CRSs) assist online users in their information-seeking and decision making tasks by supporting an interactive process. Although these processes could be rather diverse, CRSs typically follow a fixed strategy, e.g., based on critiquing or on iterative query reformulation. In a previous paper, we proposed a novel recommendation model that allows conversational systems to autonomously improve a fixed strategy and eventually learn a better one using reinforcement learning techniques. This strategy is optimal for the given model of the interaction and it is adapted to the users' behaviors. In this paper we validate our approach in an online CRS by means of a user study involving several hundreds of testers. We show that the optimal strategy is different from the fixed one, and supports more effective and efficient interaction sessions.

REFERENCES

• 1: Gediminas Adomavicius , Alexander Tuzhilin, Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions, IEEE Transactions on Knowledge and Data Engineering, v.17 n.6, p.734-749, June 2005 [doi>10.1109/TKDE.2005.99]
• 2: Eugene Agichtein , Eric Brill , Susan Dumais , Robert Ragno, Learning user interaction models for predicting web search result preferences, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148175]
• 3: David W. Aha , Len Breslow, Refining Conversational Case Libraries, Proceedings of the Second International Conference on Case-Based Reasoning Research and Development, p.267-278, July 25-27, 1997
• 4: L. Ardissono, G. Petrone, and M. Segnan. A conversational approach to the interaction with web services. Computational Intelligence, 20(4):693--709, 2004.
• 5: D. Billsus and M. Pazzani. Learning probabilistic user models. In UM97 Workshop on Machine Learning for User Modeling, 1997.
• 6: R. Burke. Hybrid web recommender systems. In The Adaptive Web, pages 377--408. Springer Berlin / Heidelberg, 2007.
• 7: Nick Golovin , Erhard Rahm, Reinforcement Learning Architecture for Web Recommendations, Proceedings of the International Conference on Information Technology: Coding and Computing (ITCC'04) Volume 2, p.398, April 05-07, 2004
• 8: A. Goy, L. Ardissono, and G. Petrone. Personalization in e-commerce applications. In The Adaptive Web, pages 485--520. Springer Berlin / Heidelberg, 2007.
• 9: James Henderson , Oliver Lemon , Kallirroi Georgila, Hybrid reinforcement/supervised learning of dialogue policies from fixed data sets, Computational Linguistics, v.34 n.4, p.487-511, December 2008 [doi>10.1162/coli.2008.07-028-R2-05-82]
• 10: Tariq Mahmood , Francesco Ricci, Learning and adaptivity in interactive recommender systems, Proceedings of the ninth international conference on Electronic commerce, August 19-22, 2007, Minneapolis, MN, USA [doi>10.1145/1282100.1282114]
• 11: Tariq Mahmood , Francesco Ricci, Adapting the interaction state model in conversational recommender systems, Proceedings of the 10th international conference on Electronic commerce, August 19-22, 2008, Innsbruck, Austria [doi>10.1145/1409540.1409584]
• 12: T. Mahmood, F. Ricci, A. Venturini, and W. Höpken. Adaptive recommender systems for travel planning. In W. H. Peter OConnor and U. Gretzel, editors, Proc. ENTER 2008, pages 1--11. Springer, 2008.
• 13: Lorraine Mcginty , Barry Smyth, Adaptive Selection: An Analysis of Critiquing and Preference-Based Feedback in Conversational Recommender Systems, International Journal of Electronic Commerce, v.11 n.2, p.35-57, Number 2 / Winter 2006-7 [doi>10.2753/JEC1086-4415110202]
• 14: Nader Mirzadeh , Francesco Ricci, COOPERATIVE QUERY REWRITING FOR DECISION MAKING SUPPORT AND RECOMMENDER SYSTEMS, Applied Artificial Intelligence, v.21 n.10, p.895-932, November 2007 [doi>10.1080/08839510701527515]
• 15: Jakob Nielsen, Usability Engineering, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1995
• 16: J. Reilly, K. McCarthy, L. McGinty, and B. Smyth. Incremental critiquing. Knowledge-Based Systems, 18(4--5):143--151, 2005.
• 17: Paul Resnick , Hal R. Varian, Recommender systems, Communications of the ACM, v.40 n.3, p.56-58, March 1997 [doi>10.1145/245108.245121]
• 18: Guy Shani , David Heckerman , Ronen I. Brafman, An MDP-Based Recommender System, The Journal of Machine Learning Research, 6, p.1265-1295, 12/1/2005
• 19: S. P. Singh, D. J. Litman, M. J. Kearns, and M. A. Walker. Optimizing dialogue management with reinforcement learning: Experiments with the NJFun system. J. Artif. Intell. Res. (JAIR), 16:105--133, 2002.
• 20: Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
• 21: Nima Taghipour , Ahmad Kardan, A hybrid web recommender system based on Q-learning, Proceedings of the 2008 ACM symposium on Applied computing, March 16-20, 2008, Fortaleza, Ceara, Brazil [doi>10.1145/1363686.1363954]
• 22: Nima Taghipour , Ahmad Kardan , Saeed Shiry Ghidary, Usage-based web recommendations: a reinforcement learning approach, Proceedings of the 2007 ACM conference on Recommender systems, October 19-20, 2007, Minneapolis, MN, USA [doi>10.1145/1297231.1297250]
• 23: S. ten Hagen, M. van Someren, and V. Hollink. Exploration/exploitation in adaptive recommender systems. In Proceedings of the European Symposium on Intelligent Technologies, Hybrid Systems and their implementation on Smart Adaptive Systems, 2003.
• 24: C. A. Thompson, M. H. Goker, and P. Langley. A personalized system for conversational recommendations. Artificial Intelligence Research, 21:393--428, 2004.
• 25: Y. Z. Wei, L. Moreau, and N. R. Jennings. Learning users' interests in a market-based recommender system. In Proc. IDEAL, pages 833--840, 2004.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.2 Types of Systems
          Subjects: Decision support (e.g., MIS)

General Terms:
Algorithms, Design, Experimentation

Keywords:
adaptivity, conversational recommender systems, markov decision process, reinforcement learning, user study