A bayesian mixture model with linear regression mixing proportions

ABSTRACT

Classic mixture models assume that the prevalence of the various mixture components is fixed and does not vary over time. This presents problems for applications where the goal is to learn how complex data distributions evolve. We develop models and Bayesian learning algorithms for inferring the temporal trends of the components in a mixture model as a function of time. We show the utility of our models by applying them to the real-life problem of tracking changes in the rates of antibiotic resistance in Escherichia coli and Staphylococcus aureus. The results show that our methods can derive meaningful temporal antibiotic resistance patterns.

REFERENCES

• 1: http://www.armprogram.com/.
• 2: Charu C. Aggarwal , Jiawei Han , Jianyong Wang , Philip S. Yu, A framework for clustering evolving data streams, Proceedings of the 29th international conference on Very large data bases, p.81-92, September 09-12, 2003, Berlin, Germany
• 3: A. J. Bagnall , G. J. Janacek, Clustering time series from ARMA models with clipped data, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1014061]
• 4: David M. Blei , John D. Lafferty, Dynamic topic models, Proceedings of the 23rd international conference on Machine learning, p.113-120, June 25-29, 2006, Pittsburgh, Pennsylvania [doi>10.1145/1143844.1143859]
• 5: David M. Blei , Andrew Y. Ng , Michael I. Jordan, Latent dirichlet allocation, The Journal of Machine Learning Research, 3, p.993-1022, 3/1/2003
• 6: Deepayan Chakrabarti , Ravi Kumar , Andrew Tomkins, Evolutionary clustering, Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, August 20-23, 2006, Philadelphia, PA, USA [doi>10.1145/1150402.1150467]
• 7: Yun Chi , Xiaodan Song , Dengyong Zhou , Koji Hino , Belle L. Tseng, Evolutionary spectral clustering by incorporating temporal smoothness, Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, August 12-15, 2007, San Jose, California, USA [doi>10.1145/1281192.1281212]
• 8: Bill Chiu , Eamonn Keogh , Stefano Lonardi, Probabilistic discovery of time series motifs, Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, August 24-27, 2003, Washington, D.C. [doi>10.1145/956750.956808]
• 9: A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal Royal Stat. Soc., Series B, 39(1):1--38, 1977.
• 10: T. S. Ferguson. A bayesian analysis of some nonparametric problems. Annals of Statistics, vol. 1, no. 2:209--230, 1973.
• 11: Mario A. T. Figueiredo , Anil K. Jain, Unsupervised Learning of Finite Mixture Models, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.3, p.381-396, March 2002 [doi>10.1109/34.990138]
• 12: K. G. and B. M. Antibiotic susceptibility pattern of escherichia coli strains with verocytotoxic e. coli-associated virulence factors from food and animal faeces. Food Microbiology, Volume 20, Number 1:27--33(7), February 2003.
• 13: S. Geman and D. Geman. Stochastic relaxation, gibbs distribution and bayesian restoration of images. IEEE PAMI, vol. 6:721--741, 1984.
• 14: Geoff Hulten , Laurie Spencer , Pedro Domingos, Mining time-changing data streams, Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, p.97-106, August 26-29, 2001, San Francisco, California [doi>10.1145/502512.502529]
• 15: Alexander Ihler , Jon Hutchins , Padhraic Smyth, Adaptive event detection with time-varying poisson processes, Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, August 20-23, 2006, Philadelphia, PA, USA [doi>10.1145/1150402.1150428]
• 16: Alexander Ihler , Jon Hutchins , Padhraic Smyth, Adaptive event detection with time-varying poisson processes, Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, August 20-23, 2006, Philadelphia, PA, USA [doi>10.1145/1150402.1150428]
• 17: Jon Kleinberg, Bursty and Hierarchical Structure in Streams, Data Mining and Knowledge Discovery, v.7 n.4, p.373-397, October 2003 [doi>10.1023/A:1024940629314]
• 18: Daniel B. Neill , Andrew W. Moore , Maheshkumar Sabhnani , Kenny Daniel, Detection of emerging space-time clusters, Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, August 21-24, 2005, Chicago, Illinois, USA [doi>10.1145/1081870.1081897]
• 19: C. E. Rasmussen. The infinite gaussian mixture model. In NIPS, pages 554--560, 1999.
• 20: Peter Schlattmann, Estimating the number of components in a finite mixture model: the special case of homogeneity, Computational Statistics & Data Analysis, v.41 n.3-4, p.441-451, 28 January 2003 [doi>10.1016/S0167-9473(02)00173-1]
• 21: Xuerui Wang , Andrew McCallum, Topics over time: a non-Markov continuous-time model of topical trends, Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, August 20-23, 2006, Philadelphia, PA, USA [doi>10.1145/1150402.1150450]

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications

General Terms:
Algorithms, Reliability

Keywords:
bayesian mixture model, gibbs sampler, linear regression, mixing proportion