Cross channel optimized marketing by reinforcement learning

ABSTRACT

The issues of cross channel integration and customer life time value modeling are two of the most important topics surrounding customer relationship management (CRM) today. In the present paper, we describe and evaluate a novel solution that treats these two important issues in a unified framework of Markov Decision Processes (MDP). In particular, we report on the results of a joint project between IBM Research and Saks Fifth Avenue to investigate the applicability of this technology to real world problems. The business problem we use as a testbed for our evaluation is that of optimizing direct mail campaign mailings for maximization of profits in the store channel. We identify a problem common to cross-channel CRM, which we call the Cross-Channel Challenge, due to the lack of explicit linking between the marketing actions taken in one channel and the customer responses obtained in another. We provide a solution for this problem based on old and new techniques in reinforcement learning. Our in-laboratory experimental evaluation using actual customer interaction data show that as much as 7 to 8 per cent increase in the store profits can be expected, by employing a mailing policy automatically generated by our methodology. These results confirm that our approach is valid in dealing with the cross channel CRM scenarios in the real world.

REFERENCES

• 1: N. Abe, N. Verma, C. Apte, and R. Schroko. Cross channel optimized marketing by reinforcement learning. Technical Report RC23132(W0403-021), IBM Research, March 2004.
• 2: C. Apte , E. Bibelnieks , R. Natarajan , E. Pednault , F. Tipu , D. Campbell , B. Nelson, Segmentation-based modeling for advanced targeted marketing, Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, p.408-413, August 26-29, 2001, San Francisco, California [doi>10.1145/502512.502573]
• 3: L. C. Baird. Reinforcement learning in continuous time: Advantage updating. In Proceedings of the International Conference on Neural Networks, June 1994.
• 4: S. Bradtke and M. Duff. Reinforcement learing methods for continuous-time Markov decision problems. In Advances in Neural Information Processing Systems, volume 7, pages 393--400. The MIT Press, nov 1995.
• 5: L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 1996.
• 6: Sham Kakade , John Langford, Approximately Optimal Approximate Reinforcement Learning, Proceedings of the Nineteenth International Conference on Machine Learning, p.267-274, July 08-12, 2002
• 7: R. Natarajan and E. Pednault. Segmented regression estimators for massive data sets. In Second SIAM International Conference on Data Mining, Arlington, Virginia, 2002. to appear.
• 8: Edwin Pednault , Naoki Abe , Bianca Zadrozny, Sequential cost-sensitive decision making with reinforcement learning, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, July 23-26, 2002, Edmonton, Alberta, Canada [doi>10.1145/775047.775086]
• 9: Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
• 10: Bianca Zadrozny , Charles P. Elkan, Policy mining: learning decision policies from fixed sets of data, 2003

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

General Terms:
Experimentation

Keywords:
CRM, cost sensitive learning, customer life time value, reinforcement learning, targeted marketing