Heteroscedastic Gaussian process regression

ABSTRACT

This paper presents an algorithm to estimate simultaneously both mean and variance of a non parametric regression problem. The key point is that we are able to estimate variance locally unlike standard Gaussian Process regression or SVMs. This means that our estimator adapts to the local noise. The problem is cast in the setting of maximum a posteriori estimation in exponential families. Unlike previous work, we obtain a convex optimization problem which can be solved via Newton's method.

REFERENCES

• 1: Yasemin Altun , Alex J. Smola , Thomas Hofmann, Exponential families for conditional random fields, Proceedings of the 20th conference on Uncertainty in artificial intelligence, p.2-9, July 07-11, 2004, Banff, Canada
• 2: Cawley, G., Talbot, N., Foxall, R., Dorling, S., & Mandic, D. (2003). Approximately unbiased estimation of conditional variance in heteroscedastic kernel ridge regression. European Symposium on Artificial Neural Networks (pp. 209--214). d-side.
• 3: Shai Fine , Katya Scheinberg, Efficient svm training using low-rank kernel representations, The Journal of Machine Learning Research, 2, 3/1/2002
• 4: Paul W. Goldberg , Christopher K. I. Williams , Christopher M. Bishop, Regression with input-dependent noise: a Gaussian process treatment, Proceedings of the 1997 conference on Advances in neural information processing systems 10, p.493-499, July 1998, Denver, Colorado, United States
• 5: Gert R. G. Lanckriet , Nello Cristianini , Peter Bartlett , Laurent El Ghaoui , Michael I. Jordan, Learning the Kernel Matrix with Semidefinite Programming, The Journal of Machine Learning Research, 5, p.27-72, 12/1/2004
• 6: Schölkopf, B., Smola, A. J., Williamson, R. C., & Bartlett, P. L. (2000). New support vector algorithms. Neural Computation, 12, 1207 -- 1245.
• 7: Wahba, G. (1990). Spline models for observational data, vol. 59 of CBMS-NSF Regional Conference Series in Applied Mathematics. Philadelphia: SIAM.
• 8: C. K. I. Williams, Prediction with Gaussian processes: from linear regression to linear prediction and beyond, Learning in graphical models, MIT Press, Cambridge, MA, 1999
• 9: Yuan, M., & Wahba, G. (2004). Doubly penalized likelihood estimator in heteroscedastic regression (Technical Report 1084rr). University of Winconsin.