Gaussian Processes for Machine Learning

@inproceedings{Rasmussen2009GaussianPF,
  title={Gaussian Processes for Machine Learning},
  author={Carl Edward Rasmussen and Christopher K. I. Williams},
  booktitle={Adaptive computation and machine learning},
  year={2009}
}
Gaussian processes (GPs) provide a principled, practical, probabilistic approach to learning in kernel machines. [] Key Method A wide variety of covariance (kernel) functions are presented and their properties discussed. Model selection is discussed both from a Bayesian and a classical perspective.
Flexible and efficient Gaussian process models for machine learning
TLDR
Several new techniques to reduce the complexity of Gaussian process models to 0(N3) complexity and relax the Gaussianity assumption of the process by learning a nonlinear transformation of the output space are developed.
Sparse gaussian processes for large-scale machine learning
TLDR
This thesis presents several novel sparse GP models that compare favorably with SPGP, both in terms of predictive performance and error bar quality, and provides two broad classes of models: Marginalized Networks (MNs) and Inter- Domain GPs (IDGPs).
Variational Mixtures of Gaussian Processes for Classification
TLDR
A new Mixture of Gaussian Processes for Classification (MGPC) is proposed, which employs the logistic function as likelihood to obtain the class probabilities, which is suitable for classification problems.
SPARSE GAUSSIAN PROCESSES FOR LARGE-SCALE MACHINE LEARNING
TLDR
This thesis presents several novel sparse GP models that compare favorably with SPGP, both in terms of predictive performance and error bar quality, and provides two broad classes of models: Marginalized Networks (MNs) and Inter-Domain GPs (IDGPs).
Variable sigma Gaussian processes: An expectation propagation perspective
TLDR
This work describes how VSGP can be applied to classification and other problems, by deriving it as an expectation propagation algorithm, and shows that endowing each basis point with its own full covariance matrix provides a significant increase in approximation power.
Sparse-posterior Gaussian Processes for general likelihoods
TLDR
A new sparse GP framework that uses expectation propagation to directly approximate general GP likelihoods using a sparse and smooth basis is proposed that outperforms previous GP classification methods on benchmark datasets in terms of minimizing divergence to the non-sparse GP solution as well as lower misclassification rate.
Nonparametric Mixtures of Gaussian Processes With Power-Law Behavior
TLDR
This paper considers a fully generative nonparametric Bayesian model with power-law behavior, generating GPs over the whole input space of the learned task, and provides an efficient algorithm for model inference.
Semi-Supervised Learning with Gaussian Processes
TLDR
In the presence of few labeled examples, the proposed algorithm outperforms cross-validation methods, and the experimental results demonstrating the effectiveness of this algorithm in comparison with other related works in the literature are presented.
Sparse nonlinear methods for predicting structured data
TLDR
The goals of this work are to develop nonlinear, nonparametric modelling techniques for structure learning and prediction problems in which there are structured dependencies among the observed data, and to equip the authors' models with sparse representations which serve both to handle prior sparse connectivity assumptions and to reduce computational complexity.
Consistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimation
TLDR
The purpose of this paper is to compare the empirical Bayesian and approximation theoretic approaches to hierarchical learning, in terms of large data consistency, variance of estimators, robustness of the estimators to model misspecification, and computational cost.
...
...

References

SHOWING 1-10 OF 231 REFERENCES
Gaussian Processes For Machine Learning
  • M. Seeger
  • Computer Science
    Int. J. Neural Syst.
  • 2004
TLDR
This paper gives an introduction to Gaussian processes on a fairly elementary level with special emphasis on characteristics relevant in machine learning and shows up precise connections to other "kernel machines" popular in the community.
Introduction to Gaussian processes
TLDR
This chapter will assess whether the feedforward network has been superceded, for supervised regression and classification tasks, and will review work on this idea by Williams and Rasmussen (1996), Neal (1997), Barber and Williams (1997) and Gibbs and MacKay (1997).
PAC-Bayesian Generalisation Error Bounds for Gaussian Process Classification
  • M. Seeger
  • Computer Science
    J. Mach. Learn. Res.
  • 2002
TLDR
By applying the PAC-Bayesian theorem of McAllester (1999a), this paper proves distribution-free generalisation error bounds for a wide range of approximate Bayesian GP classification techniques, giving a strong learning-theoretical justification for the use of these techniques.
Regression and Classification Using Gaussian Process Priors
TLDR
Gaussian processes are in my view the simplest and most obvious way of defining flexible Bayesian regression and classification models, but despite some past usage, they appear to have been rather neglected as a general-purpose technique.
Regression and Classification Using Gaussian Process Priors
TLDR
Gaussian processes are in my view the simplest and most obvious way of defining flexible Bayesian regression and classification models, but despite some past usage, they appear to have been rather neglected as a general-purpose technique.
An introduction to Support Vector Machines
This book is the first comprehensive introduction to Support Vector Machines (SVMs), a new generation learning system based on recent advances in statistical learning theory. The book also introduces
Evaluation of gaussian processes and other methods for non-linear regression
TLDR
It is shown that a Bayesian approach to learning in multi-layer perceptron neural networks achieves better performance than the commonly used early stopping procedure, even for reasonably short amounts of computation time.
Learning with Uncertainty: Gaussian Processes and Relevance Vector Machines
TLDR
This paper modify this method for training generalized linear models by adapting automatically the width of the basis functions to the optimal for the data at hand, and tries the Adaptive RVM for prediction on the chaotic Mackey-Glass time series.
On the Noise Model of Support Vector Machines Regression
TLDR
This work can be extended to justify nonquadratic loss functions in any Maximum Likelihood or Maximum AP osteriori approach, and applies not only to the ILF, but to a much broader class of loss functions.
Gaussian Processes for Classification: Mean-Field Algorithms
TLDR
A mean-field algorithm for binary classification with gaussian processes that is based on the TAP approach originally proposed in statistical physics of disordered systems is derived and an approximate leave-one-out estimator for the generalization error is computed.
...
...