Semi-supervised learning with data calibration for long-term time series forecasting

ABSTRACT

Many time series prediction methods have focused on single step or short term prediction problems due to the inherent difficulty in controlling the propagation of errors from one prediction step to the next step. Yet, there is a broad range of applications such as climate impact assessments and urban growth planning that require long term forecasting capabilities for strategic decision making. Training an accurate model that produces reliable long term predictions would require an extensive amount of historical data, which are either unavailable or expensive to acquire. For some of these domains, there are alternative ways to generate potential scenarios for the future using computer-driven simulation models, such as global climate and traffic demand models. However, the data generated by these models are currently utilized in a supervised learning setting, where a predictive model trained on past observations is used to estimate the future values. In this paper, we present a semi-supervised learning framework for long-term time series forecasting based on Hidden Markov Model Regression. A covariance alignment method is also developed to deal with the issue of inconsistencies between historical and model simulation data. We evaluated our approach on data sets from a variety of domains, including climate modeling. Our experimental results demonstrate the efficacy of the approach compared to other supervised learning methods for long-term time series forecasting.

REFERENCES

• 1: http://www.ccsn.ca/, canadian climate change scenarios network, environment canada.
• 2: P. M. Baggenstos. A modified Baum-Welch algorithm for hidden markov models with multiple observation spaces. IEEE Trans. on Speech Audio Processing, pages 411--416, 2001.
• 3: Avrim Blum , Shuchi Chawla, Learning from Labeled and Unlabeled Data using Graph Mincuts, Proceedings of the Eighteenth International Conference on Machine Learning, p.19-26, June 28-July 01, 2001
• 4: Avrim Blum , Tom Mitchell, Combining labeled and unlabeled data with co-training, Proceedings of the eleventh annual conference on Computational learning theory, p.92-100, July 24-26, 1998, Madison, Wisconsin, United States [doi>10.1145/279943.279962]
• 5: Yann-Aël Le Borgne , Silvia Santini , Gianluca Bontempi, Adaptive model selection for time series prediction in wireless sensor networks, Signal Processing, v.87 n.12, p.3010-3020, December, 2007 [doi>10.1016/j.sigpro.2007.05.015]
• 6: Ulf Brefeld , Thomas Gärtner , Tobias Scheffer , Stefan Wrobel, Efficient co-regularised least squares regression, Proceedings of the 23rd international conference on Machine learning, p.137-144, June 25-29, 2006, Pittsburgh, Pennsylvania [doi>10.1145/1143844.1143862]
• 7: G. Celeux and J. Durand. Selecting hidden markov model state number with cross-validated likelihood. In Computational Statistics, 2007.
• 8: S. Charles, B. Bates, I. Smith, and J. Hughes. Statistical downscaling of daily precipitation from observed and modelled atmospheric fields. In Hydrological Processes, pages 1373--1394, 2004.
• 9: H. Cheng, P.-N. Tan, J. Gao, and J. Scripps. Multistep-ahead time series prediction. In Proc. of the Pacific-Asia Conf on Knowledge Discovery and Data Mining, pages 765--774, 2006.
• 10: Ira Cohen , Fabio G. Cozman , Nicu Sebe , Marcelo C. Cirelo , Thomas S. Huang, Semisupervised Learning of Classifiers: Theory, Algorithms, and Their Application to Human-Computer Interaction, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.26 n.12, p.1553-1567, December 2004 [doi>10.1109/TPAMI.2004.127]
• 11: C. Cortes and M. Mohri. On transductive regression. In Advances in Neural Information Processing Systems, 2006.
• 12: Fabio G. Cozman , Ira Cohen, Unlabeled Data Can Degrade Classification Performance of Generative Classifiers, Proceedings of the Fifteenth International Florida Artificial Intelligence Research Society Conference, p.327-331, May 14-16, 2002
• 13: F. Cozman, I. Cohen, and M. Cirelo. Semi-supervised learning of mixture models. In Proc of the 20th Int'l Conf. on Machine Learning, 2003.
• 14: W. Enke and A. Spekat. Downscaling climate model outputs into local and regional weather elements by classification and regression. In Climate Research 8, pages 195--207, 1997.
• 15: K. Fujinaga, M. Nakai, H. Shimodaira, and S. Sagayama. Multiple-regression hidden markov model. In Proc. of IEEE Int'l Conf. on Acoustics, Speech, and Signal Processing, 2001.
• 16: C. Lee Giles , Steve Lawrence , Ah Chung Tsoi, Noisy Time Series Prediction using Recurrent Neural Networks and Grammatical Inference, Machine Learning, v.44 n.1-2, p.161-183, July-August 2001 [doi>10.1023/A:1010884214864]
• 17: T. Hastie and C. Loader. Local regression: Automatic kernel carpentry. In Statistical Science, pages 120--143, 1993.
• 18: W. Hong, P. Pai, S. Yang, and R. Theng. Highway traffic forecasting by support vector regression model with tabu search algorithms. In Proc. of Int'l Joint Conf. on Neural Networks, pages 1617--1621, 2006.
• 19: Thorsten Joachims, Transductive Inference for Text Classification using Support Vector Machines, Proceedings of the Sixteenth International Conference on Machine Learning, p.200-209, June 27-30, 1999
• 20: B. Kedem and K. Fokianos. Regression models for time series analysis. Wiley-Interscience ISBN: 0-471-36355, 2002.
• 21: E. Keogh and T. Folias. Uc riverside time series data mining archive. http://www.cs.ucr.edu/ eamonn/TSDMA/index.html.
• 22: C. Leggetter and P. Woodland. Maximum likelihood linear regression for speaker adaptation of continuous density hidden markov models. In Computer Speech and Language, pages 171--185(15). Academic Press, 1995.
• 23: A. Ober-Sundermeier and H. Zackor. Prediction of congestion due to road works on freeways. In Proc. of IEEE Intelligent Transportation Systems, pages 240--244, 2001.
• 24: Alex J. Smola , Bernhard Schölkopf, A tutorial on support vector regression, Statistics and Computing, v.14 n.3, p.199-222, August 2004 [doi>10.1023/B:STCO.0000035301.49549.88]
• 25: Q. Tian, J. Yu, Q. Xue, and N. Sebe. A new analysis of the value of unlabeled data in semi-supervised learning for image retrieval. In Proc. of IEEE Int'l Conf. on Multimedia and Expo., pages 1019--1022, 2004.
• 26: Meng Wang , Xian-Sheng Hua , Yan Song , Li-Rong Dai , Hong-Jiang Zhang, Semi-Supervised Kernel Regression, Proceedings of the Sixth International Conference on Data Mining, p.1130-1135, December 18-22, 2006 [doi>10.1109/ICDM.2006.143]
• 27: R. Wilby, S. Charles, E. Zorita, B. Timbal, P. Whetton, and L. Mearns. Guidelines for use of climate scenarios developed from statistical downscaling methods. Available from the DDC of IPCC TGCIA, 2004.
• 28: C.-C. C. Wong, M.-C. Chan, and C.-C. Lam. Financial time series forecasting by neural network using conjugate gradient learning algorithm and multiple linear regression weight initialization. Technical Report 61, Society for Computational Economics, Jul 2000.
• 29: D. Zhou, O. Bousquet, T. Lal, J. Weston, and B. Schölkopf. Learning with local and global consistency. In Advances in Neural Information Processing Systems 16, 2003.
• 30: Z. Zhou and M. Li. Semi-supervised regression with co-training. In Proc. of Int'l Joint Conf. on Artificial Intelligence, 2005.
• 31: X. Zhu. Semi-supervised learning literature survey. In Technical Report,Computer Sciences, University of Wisconsin-Madison, 2005.
• 32: X. Zhu, Z. Ghahramani, and J. Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Proc. of the 20th Int'l Conf. on Machine Learning, volume 20, 2003.
• 33: X. Zhu and A. Goldberg. Kernel regression with order preferences. In Association for the Advancement of Artificial Intelligence, page 681, 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.m MISCELLANEOUS

General Terms:
Algorithms, Design, Experimentation

Keywords:
semi-supervised learning, time series prediction