# Bayesian Classification With Gaussian Processes

@article{Williams1998BayesianCW, title={Bayesian Classification With Gaussian Processes}, author={Christopher K. I. Williams and David Barber}, journal={IEEE Trans. Pattern Anal. Mach. Intell.}, year={1998}, volume={20}, pages={1342-1351} }

We consider the problem of assigning an input vector to one of m classes by predicting P(c|x) for c=1,...,m. For a two-class problem, the probability of class one given x is estimated by /spl sigma/(y(x)), where /spl sigma/(y)=1/(1+e/sup -y/). A Gaussian process prior is placed on y(x), and is combined with the training data to obtain predictions for new x points. We provide a Bayesian treatment, integrating over uncertainty in y and in the parameters that control the Gaussian process prior the…

## 789 Citations

### Sparse Gaussian Process Classification With Multiple Classes

- Computer Science
- 2004

This work shows how to generalize the binary classification informative vector machine (IVM) to multiple classes and is a principled approximation to Bayesian inference which yields valid uncertainty estimates and allows for hyperparameter adaption via marginal likelihood maximization.

### Gaussian process models for robust regression, classification, and reinforcement learning

- Computer Science
- 2006

Gaussian process models constitute a class of probabilistic statistical models in which a Gaussian process is used to describe the Bayesian a priori uncertainty about a latent function, and it will be shown how this can be used to estimate value functions.

### Multi-class Gaussian Process Classification with Noisy Inputs

- Computer ScienceJ. Mach. Learn. Res.
- 2021

The results obtained show that, although the classification error is similar across methods, the predicted distribution of the proposed methods is better, in terms of the test log-likelihood, than the predictive distribution of a classifier based on GPs that ignores input noise.

### Twin Gaussian Processes for Binary Classification

- Computer Science2011 IEEE 11th International Conference on Data Mining
- 2011

A new Gaussian process model termed as twin Gaussian processes for binary classification, which allows for an explicit inference based on analytical methods, thereby avoiding the high computational cost caused by approximating the posterior with Gaussian distribution.

### Approximations for Binary Gaussian Process Classification

- Computer Science
- 2008

We provide a comprehensive overview of many recent algorithms for approximate inference in Gaussian process models for probabilistic binary classification. The relationships between several…

### Exact Bayesian bin classification: a fast alternative to Bayesian classification and its application to neural response analysis

- Computer Science, BiologyJournal of Computational Neuroscience
- 2007

This work introduces a fast, exact alternative to Bayesian classification, which allows for the computation of feedback signals, which can be used as input to subsequent stages of inference, e.g. neural network training.

### Bayesian Gaussian Process Classification with the EM-EP Algorithm

- Computer ScienceIEEE Transactions on Pattern Analysis and Machine Intelligence
- 2006

An approximate EM algorithm, the EM-EP algorithm, is presented, which is found to converge in practice and provides an efficient Bayesian framework for learning hyperparameters of the kernel.

### Gaussian Processes for Object Categorization

- Computer ScienceInternational Journal of Computer Vision
- 2009

This work shows that with an appropriate combination of kernels a significant boost in classification performance is possible, and indicates the utility of active learning with probabilistic predictive models, especially when the amount of training data labels that may be sought for a category is ultimately very small.

### Variational Multinomial Logit Gaussian Process

- Computer ScienceJ. Mach. Learn. Res.
- 2012

A variational approximation to the Gaussian process multi-class model is proposed, and criteria which can be used to select the inducing set are derived and the effectiveness of these criteria over random selection in an experiment is shown.

## References

SHOWING 1-10 OF 33 REFERENCES

### Soft Classiication, A. K. A. Risk Estimation, via Penalized Log Likelihood and Smoothing Spline Analysis of Variance

- Computer Science
- 1993

A family of methods which are well suited for the estimation of the probability that the next example will be in class 1 are described, which combines results from Penalized log likelihood estimation, Smoothing splines, and Analysis of variance to get the PSA class of methods.

### Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification

- Computer Science, Mathematics
- 1997

Software is now available that implements Gaussian process methods using covariance functions with hierarchical parameterizations, which can discover high-level properties of the data, such as which inputs are relevant to predicting the response.

### Evaluation of gaussian processes and other methods for non-linear regression

- Computer Science
- 1997

It is shown that a Bayesian approach to learning in multi-layer perceptron neural networks achieves better performance than the commonly used early stopping procedure, even for reasonably short amounts of computation time.

### Gaussian Processes for Bayesian Classification via Hybrid Monte Carlo

- Computer ScienceNIPS
- 1996

Gaussian processes are used to approximate the weight space integral analytically, so that only a small number of hyperparameters need be integrated over by MCMC methods.

### Bayesian Methods for Backpropagation Networks

- Computer Science
- 1996

This chapter describes numerical techniques based on Gaussian approximations for implementation of powerful and practical methods for controlling, comparing, and using adaptive network models.

### Gaussian Processes for Regression

- Computer Science, MathematicsNIPS
- 1995

This paper investigates the use of Gaussian process priors over functions, which permit the predictive Bayesian analysis for fixed values of hyperparameters to be carried out exactly using matrix operations.

### Bayesian Learning for Neural Networks

- Computer Science
- 1995

This book demonstrates how Bayesian methods allow complex neural network models to be used without fear of the "overfitting" that can occur with traditional training methods.

### Computing with Infinite Networks

- Computer ScienceNIPS
- 1996

For neural networks with a wide class of weight-priors, it can be shown that in the limit of an infinite number of hidden units the prior over functions tends to a Gaussian process. In this paper…

### Maximum likelihood estimation of models for residual covariance in spatial regression

- Mathematics
- 1984

We describe the maximum likelihood method for fitting the linear model when residuals are correlated and when the covariance among the residuals is determined by a parametric model containing unknown…

### Flexible Non-linear Approaches to Classification

- Mathematics
- 1994

Questions common to all members of the class, including how best to use the outputs to classify, how to fit the class of functions and also how to choose between classes are explored via some theory and three real examples.