Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

Learning temporal graph structures from time series data reveals important dependency relationships between current observations and histories. Most previous work focuses on learning and predicting with "static" temporal graphs only. However, in many applications such as mechanical systems and biology systems, the temporal dependencies might change over time. In this paper, we develop a dynamic temporal graphical models based on hidden Markov model regression and lasso-type algorithms. Our method is able to integrate two usually separate tasks, i.e. inferring underlying states and learning temporal graphs, in one unified model. The output temporal graphs provide better understanding about complex systems, i.e. how their dependency graphs evolve over time, and achieve more accurate predictions. We examine our model on two synthetic datasets as well as a real application dataset for monitoring oil-production equipment to capture different stages of the system, and achieve promising results.

REFERENCES

• 1: D. W. Aha and R. L. Bankert. A comparative evaluation of sequential feature selection algorithms. In Proceedings of the Fifth International Workshop on Artificial Intelligence and Statistics, pages 1--7. Springer-Verlag, 1995.
• 2: Andrew Arnold , Yan Liu , Naoki Abe, Temporal causal modeling with graphical granger methods, Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, August 12-15, 2007, San Jose, California, USA [doi>10.1145/1281192.1281203]
• 3: D. Brillinger. Remarks concerning graphical models for time series and point processes. Revista de Econometria, 16:1--23, 1996.
• 4: Haibin Cheng , Pang-Ning Tan, Semi-supervised learning with data calibration for long-term time series forecasting, Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, August 24-27, 2008, Las Vegas, Nevada, USA [doi>10.1145/1401890.1401911]
• 5: D. Chickering. Learning bayesian networks is npcomplete. Learning from data: AI and statistics V, 1996.
• 6: D. Eaton and K. Murphy. Bayesian structure learning using dynamic programming and mcmc. In UAI, 2007.
• 7: J. Friedman, T. Hastie, and R. Tibshirani. Regularized paths for generalized linear models via coordinate descent. 2008.
• 8: N. Friedman. Inferring cellular networks using probabilistic graphical models science. Science, 303:799--805, 2004.
• 9: K. Fujinaga, M. Nakai, H. Shimodaira, and S. Sagayama. Multiple-regression hidden markov model. In In Proceedings of International Conference on Acoustics, Speech, and Signal Processing (ICASSP-01), pages 513--516, 2001.
• 10: Anna Goldenberg , Andrew Moore, Tractable learning of large Bayes net structures from sparse data, Proceedings of the twenty-first international conference on Machine learning, p.44, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015406]
• 11: Fan Guo , Steve Hanneke , Wenjie Fu , Eric P. Xing, Recovering temporally rewiring networks: a model-based approach, Proceedings of the 24th international conference on Machine learning, p.321-328, June 20-24, 2007, Corvalis, Oregon [doi>10.1145/1273496.1273537]
• 12: Michael I. Jordan, Learning in graphical models, MIT Press, Cambridge, MA, 1999
• 13: S.-I. Lee, V. Ganapathi, and D. Koller. Efficient structure learning of markov networks using l1-regularization. In B. Scholkopf, J. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems 19, pages 817--824. MIT Press,Cambridge, MA, 2007.
• 14: Sun-In Lee , Honglak Lee , Pieter Abbeel , Andrew Y. Ng, EfficientL1regularized logistic regression, Proceedings of the 21st national conference on Artificial intelligence, p.401-408, July 16-20, 2006, Boston, Massachusetts
• 15: A. Lozano, N. Abe, Y. Liu, and S. Rosset. Grouped graphical granger modeling for gene expression regulatory networks discovery. In Proceedings of International Conference on Intelligent Systems for Molecular Biology (ISMB-09), 2009.
• 16: Aurelie C. Lozano , Naoki Abe , Yan Liu , Saharon Rosset, Grouped graphical Granger modeling methods for temporal causal modeling, Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, June 28-July 01, 2009, Paris, France [doi>10.1145/1557019.1557085]
• 17: N. Meinshausen and P. Buhlmann. High dimensional graphs and variable selection with the lasso. Annals of Statistics, 34(6):1436--1462, 2006.
• 18: K. Noto and M. Craven. Learning hidden markov models for regression using path aggregation. In Proceedings of the 24th Annual Conference on Uncertainty in Artificial Intelligence (UAI-08), 2008.
• 19: L. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77:257--286, 1989.
• 20: Pradeep Ravikumar , John Lafferty, Approximate inference, structure learning and feature estimation in markov random fields, Carnegie Mellon University, Pittsburgh, PA, 2007
• 21: R. Tibshirani. Regression shrinkage and selection via the lasso. J. Royal. Statist., 58(1):267--288, 1996.
• 22: M. J. Wainwright, P. Ravikumar, and J. D. Lafferty. High-dimensional graphical model selection using l1-regularized logistic regression. In Advances in Neural Information Processing Systems (NIPS-07), pages 1465--1472. 2007.
• 23: M. Yuan and Y. Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society, Series B, 68:49--67, 2006.
• 24: H. Zou and T. Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society, Series B, 67:301--320, 2005.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

General Terms:
Algorithms, Design, Performance

Keywords:
graphical models, structure learning, time series data