• Corpus ID: 162184285

Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentation

@article{GalyFajou2019MultiClassGP,
  title={Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentation},
  author={Th{\'e}o Galy-Fajou and F. Wenzel and Christian Donner and Manfred Opper},
  journal={ArXiv},
  year={2019},
  volume={abs/1905.09670}
}
We propose a new scalable multi-class Gaussian process classification approach building on a novel modified softmax likelihood function. The new likelihood has two benefits: it leads to well-calibrated uncertainty estimates and allows for an efficient latent variable augmentation. The augmented model has the advantage that it is conditionally conjugate leading to a fast variational inference method via block coordinate ascent updates. Previous approaches suffered from a trade-off between… 

Figures from this paper

Automated Augmented Conjugate Inference for Non-conjugate Gaussian Process Models
TLDR
The proposed automated augmented conjugate inference, a new inference method for non-conjugate Gaussian processes (GP) models, develops two inference methods, a fast and scalable stochastic variational inference method that uses efficient block coordinate ascent updates, which are computed in closed form.
Scalable Cross Validation Losses for Gaussian Process Models
TLDR
It is argued that the good predictive performance can be traced to the non-parametric nature of the resulting predictive distributions as well as to the cross-validation loss, which provides robustness against model mis-specification.
Deep Structured Mixtures of Gaussian Processes
TLDR
Deep structured mixtures of GP experts are introduced, a stochastic process model which allows exact posterior inference, has attractive computational and memory costs, and when used as GP approximation, captures predictive uncertainties consistently better than previous expert-based approximations.
Adaptive Inducing Points Selection For Gaussian Processes
TLDR
This work developed an efficient variational approach to train online GPs by using sparsity techniques and proposed to adaptively add new points, based on the properties of the GP and the structure of the data, to improve the performance of the algorithm.
Scalable Gaussian Process Classification with Additive Noise for Various Likelihoods
Gaussian process classification (GPC) provides a flexible and powerful statistical framework describing joint distributions over function space. Conventional GPCs however suffer from (i) poor
A Gaussian Process-Bayesian Bernoulli Mixture Model for Multi-Label Active Learning
TLDR
This work proposes to conduct multi-label active learning (ML-AL) through a novel integrated Gaussian Process-Bayesian Bernoulli Mixture model (GP-BM) to accurately quantify a data sample’s overall contribution to a correlated label space and choose the most informative samples for cost-effective annotation.
Extreme Classification via Adversarial Softmax Approximation
TLDR
This paper proposes a simple training method for drastically enhancing the gradient signal by drawing negative samples from an adversarial model that mimics the data distribution, and presents results on large scale data sets that show a reduction of the training time by an order of magnitude relative to several competitive baselines.
ADVERSARIAL SOFTMAX APPROXIMATION
TLDR
This paper proposes a simple training method for drastically enhancing the gradient signal by drawing negative samples from an adversarial model that mimicks the data distribution, and presents results on large scale data sets that show a reduction of the training time by an order of magnitude relative to several competitive baselines.
Dense Incremental Metric-Semantic Mapping for Multi-Agent Systems via Sparse Gaussian Process Regression
TLDR
An online probabilistic metric-semantic mapping approach for mobile robot teams relying on streaming RGB-D observations based on online Gaussian Process training and inference, and avoids the complexity of GP classification by regressing a truncated signed distance function of the regions occupied by different semantic classes.
The Natural Neural Tangent Kernel: Neural Network Training Dynamics under Natural Gradient Descent
TLDR
The theoretical results are consistent with the empirical discrepancy between the functions obtained from linearized and non-linearized natural gradient descent and that the discrepancy is small on a set of regression benchmark problems.
...
1
2
...

References

SHOWING 1-10 OF 39 REFERENCES
Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentation
We propose a scalable stochastic variational approach to GP classification building on Pólya-Gamma data augmentation and inducing points. Unlike former approaches, we obtain closed-form updates based
Scalable Multi-Class Gaussian Process Classification using Expectation Propagation
TLDR
The results show that the proposed expectation propagation method for multi-class classification with Gaussian processes performs similar or even better than these techniques, which sometimes give significantly worse predictive distributions in terms of the test log-likelihood.
Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition
TLDR
The key idea of theTT-GP is to use Tensor Train decomposition for variational parameters, which allows to train GPs with billions of inducing inputs and achieve state-of-the-art results on several benchmarks.
MCMC for Variationally Sparse Gaussian Processes
TLDR
A Hybrid Monte-Carlo sampling scheme which allows for a non-Gaussian approximation over the function values and covariance parameters simultaneously, with efficient computations based on inducing-point sparse GPs.
Variational Multinomial Logit Gaussian Process
TLDR
A variational approximation to the Gaussian process multi-class model is proposed, and criteria which can be used to select the inducing set are derived and the effectiveness of these criteria over random selection in an experiment is shown.
Variational Learning of Inducing Variables in Sparse Gaussian Processes
TLDR
A variational formulation for sparse approximations that jointly infers the inducing inputs and the kernel hyperparameters by maximizing a lower bound of the true log marginal likelihood.
Robust Multi-Class Gaussian Process Classification
TLDR
This work investigates a robust MGPC (RMGPC) which considers labeling errors independently of their distance to the decision boundaries, and shows how this method can be used for successfully identifying data instances which are difficult to classify correctly in practice.
Variational Bayesian Multinomial Probit Regression with Gaussian Process Priors
TLDR
This is the first time that a fully variational Bayesian treatment for multiclass GP classification has been developed without having to resort to additional explicit approximations to the nongaussian likelihood term.
Gaussian Processes for Big Data
TLDR
Stochastic variational inference for Gaussian process models is introduced and it is shown how GPs can be variationally decomposed to depend on a set of globally relevant inducing variables which factorize the model in the necessary manner to perform Variational inference.
Scalable Variational Gaussian Process Classification
TLDR
This work shows how to scale the model within a variational inducing point framework, outperforming the state of the art on benchmark datasets, and can be exploited to allow classification in problems with millions of data points.
...
1
2
3
4
...