• Corpus ID: 88511900

Multiple Gaussian Process Models

@article{Archambeau2011MultipleGP,
  title={Multiple Gaussian Process Models},
  author={C. Archambeau and Francis R. Bach},
  journal={arXiv: Machine Learning},
  year={2011}
}
We consider a Gaussian process formulation of the multiple kernel learning problem. The goal is to select the convex combination of kernel matrices that best explains the data and by doing so improve the generalisation on unseen data. Sparsity in the kernel weights is obtained by adopting a hierarchical Bayesian approach: Gaussian process priors are imposed over the latent functions and generalised inverse Gaussians on their associated weights. This construction is equivalent to imposing a… 

Figures from this paper

Regularization Strategies and Empirical Bayesian Learning for MKL
TLDR
This paper shows how different MKL algorithms can be understood as applications of either regularization on the kernel weights or block-norm-based regularization, which is more common in structured sparsity and multi-task learning.
Upgrading from Gaussian Processes to Student's-T Processes
TLDR
The Student's-T distribution has higher Kurtosis than a Gaussian distribution and so outliers are much more likely, and the posterior variance increases or decreases depending on the variance of observed data sample values.
Extended T-process Regression Models
Multiple Kernel Learning and Automatic Subspace Relevance Determination for High-dimensional Neuroimaging Data
TLDR
The research results demonstrate that the Gaussian Process models are competitive with or better than the well-known Support Vector Machine in terms of classification performance even in the cases of single kernel learning.
Efficient global optimization with ensemble and selection of kernel functions for engineering design
TLDR
It is revealed that the ensemble techniques improve the robustness and performance of EGO and that the use of Matérn-kernels yields better results than those of the Gaussian kernel when EGO with a single kernel is considered.

References

SHOWING 1-10 OF 27 REFERENCES
Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond
The main aim of this paper is to provide a tutorial on regression with Gaussian processes. We start from Bayesian linear regression, and show how by a change of viewpoint one can see this method as a
Gaussian Processes for Object Categorization
TLDR
This work shows that with an appropriate combination of kernels a significant boost in classification performance is possible, and indicates the utility of active learning with probabilistic predictive models, especially when the amount of training data labels that may be sought for a category is ultimately very small.
Gaussian Processes for Machine Learning
TLDR
The treatment is comprehensive and self-contained, targeted at researchers and students in machine learning and applied statistics, and deals with the supervised learning problem for both regression and classification.
Gaussian Process Dynamical Models for Human Motion
TLDR
This work marginalize out the model parameters in closed form by using Gaussian process priors for both the dynamical and the observation mappings, which results in a nonparametric model for dynamical systems that accounts for uncertainty in the model.
Propagation of uncertainty in Bayesian kernel models - application to multiple-step ahead forecasting
TLDR
This work derives novel analytic expressions for the predictive mean and variance for Gaussian kernel shapes under the assumption of a Gaussian input distribution in the static case, and of a recursive Gaussian predictive density in iterative forecasting.
Variational algorithms for approximate Bayesian inference
TLDR
A unified variational Bayesian (VB) framework which approximates computations in models with latent variables using a lower bound on the marginal likelihood and is compared to other methods including sampling, Cheeseman-Stutz, and asymptotic approximations such as BIC.
Learning the Kernel Matrix with Semidefinite Programming
TLDR
This paper shows how the kernel matrix can be learned from data via semidefinite programming (SDP) techniques and leads directly to a convex method for learning the 2-norm soft margin parameter in support vector machines, solving an important open problem.
Scale Mixtures of Normal Distributions
SUMMARY This paper presents necessary and sufficient conditions under which a random variable X may be generated as the ratio ZI V where Z and V are independent and Z has a standard normal
Calibration of multivariate generalized hyperbolic distributions using the em algorithm, with applications in risk management, portfolio optimization and portfolio credit risk
TLDR
This dissertation describes a way to stably calibrate GH distributions for a wider range of parameters than has previously been reported and develops a version of the EM algorithm for calibrating GH distributions, which enables for the first time certain GH distributions to be used in modeling contexts when previously they have been numerically intractable.
Gaussian Processes for Classification: Mean-Field Algorithms
TLDR
A mean-field algorithm for binary classification with gaussian processes that is based on the TAP approach originally proposed in statistical physics of disordered systems is derived and an approximate leave-one-out estimator for the generalization error is computed.
...
...