Mining a stream of transactions for customer patterns

ABSTRACT

Transaction data can arrive at a ferocious rate in the order that transactions are completed. The data contain an enormous amount of information about customers, not just transactions, but extracting up-to-date customer information from an ever changing stream of data and mining it in real-time is a challenge. This paper describes a statistically principled approach to designing short, accurate summaries or signatures of high dimensional customer behavior that can be kept current with a stream of transactions. A signature database can then be used for data mining and to provide approximate answers to many kinds of queries about current customers quickly and accurately, as an empirical study of the calling patterns of 96,000 wireless customers who made about 18 million wireless calls over a three month period shows.

REFERENCES

• 1: A. Agresti. Cateorical data analysis. John Wiley & Soils, New York, NY, 1990.
• 2: D. BarbarA, W. DuMouchel, C. Faloutsos, P. J. Haas, J. M. Hellerstein, Y. Ioannidis, H. V. Jagadish, T. Johnson, R. Ng, V. Poosala, K. A. Ross, and K. C. Sevcik. The New Jersey data reduction report. Bulletin of the Technical Committee on Data Engineering, 20(4):3-42, 1997.
• 3: Fei Chen , Diane Lambert , José C. Pinheiro, Incremental quantile estimation for massive tracking, Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, p.516-522, August 20-23, 2000, Boston, Massachusetts, United States [doi>10.1145/347090.347195]
• 4: Phillip B. Gibbons , Yossi Matias , Viswanath Poosala, Fast Incremental Maintenance of Approximate Histograms, Proceedings of the 23rd International Conference on Very Large Data Bases, p.466-475, August 25-29, 1997
• 5: I. Grabec. Modelling of chaos by a self-organizing neural network. In K. Makisara, O. Simula, and J. Kangas, editors, Artificial Neural Networks, Proceedings of ICANN, volume 1, pages 151-156. Elsevier Science Publishers, 1991.
• 6: D. Lambert, J. C. Pinheiro, and D. X. Sun. Updating timing profiles for millions of customers in real-time. Journal of the American Statistical Association, 96(453):316-330, 2001.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS

Keywords:
Approximate queries, customer profiles, dynamic database, histograms, incremental updates, massive data, signatures