• Corpus ID: 88512016

Efficient Marginal Likelihood Computation for Gaussian Process Regression

@inproceedings{Schirru2011EfficientML,
  title={Efficient Marginal Likelihood Computation for Gaussian Process Regression},
  author={Andrea Schirru and Simone Pampuri and Giuseppe De Nicolao and Se{\'a}n F. McLoone},
  year={2011}
}
In a Bayesian learning setting, the posterior distribution of a predictive model arises from a trade-off between its prior distribution and the conditional likelihood of observed data. Such distribution functions usually rely on additional hyperparameters which need to be tuned in order to achieve optimum predictive performance; this operation can be efficiently performed in an Empirical Bayes fashion by maximizing the posterior marginal likelihood of the observed data. Since the score function… 

Figures from this paper

Computational Aspects of Gaussian Process Regression
Gaussian Process Regression (GPR) has proven to be an extremely useful methodology in classification, machine learning, geostatistics, genetic analysis, and so on. Estimation of GPR hyperparameters
Evaluating the potential of Gaussian process regression for data-driven renewable energy management
TLDR
It was found that meter failure can be bridged and no significant improvement in the quality of interval deficient wind speed data was observed after applying Gaussian process regression using the Matérn kernel.
Metis: Robustly Tuning Tail Latencies of Cloud Systems
TLDR
This paper uses Bing Ads key-value store clusters as the running example and shows that Metis reduces the overall tuning time by 98.41%, while reducing the 99-percentile latency by another 3.43%.
Data-Driven Approaches for Sensing and Control of Robot Manipulators
TLDR
A series of data-driven approaches to improve the sensing and control of robot manipulators from several aspects, including sensor fusion for motion sensing, statistical learning for feedback compensation, nonparametric learning control, and intelligent modeling and identification are presented.
Shamo: A Tool for Electromagnetic Modeling, Simulation and Sensitivity Analysis of the Head.
TLDR
A tool to assess the uncertainty in the model parameters, the tissue conductivity, as well as compute a parametric forward models for electroencephalography (EEG) and transcranial direct current stimulation (tDCS) current distribution is proposed.
GPfit: An R Package for Fitting a Gaussian Process Model to Deterministic Simulator Outputs
TLDR
This paper implements a slightly modified version of the model proposed by Ranjan et al. (2011) in the R package GPfit, with a novel parameterization of the spatial correlation function and a clustering based multi-start gradient based optimization algorithm that yield robust optimization that is typically faster than the genetic algorithm based approach.
A Short Note on Gaussian Process Modeling for Large Datasets using Graphics Processing Units
TLDR
This small study suggests that GP models are fertile ground for further implementation on CPU+GPU systems, and demonstrates that the computational cost of implementing GP models can be significantly reduced by using a CPU+ GPU heterogeneous computing system over an analogous implementation on a traditional computing system with no GPU acceleration.

References

SHOWING 1-10 OF 15 REFERENCES
Optimization of Gaussian Process Models with Evolutionary Algorithms
TLDR
Stochastic optimization methods with proper tuning differential evolution and particle swarm optimization significantly outperform the conjugate gradient method in Gaussian process (GP) model optimization.
Gaussian Processes for Machine Learning
TLDR
The treatment is comprehensive and self-contained, targeted at researchers and students in machine learning and applied statistics, and deals with the supervised learning problem for both regression and classification.
Sparse Spectrum Gaussian Process Regression
TLDR
The achievable trade-offs between predictive accuracy and computational requirements are compared, and it is shown that these are typically superior to existing state-of-the-art sparse approximations.
Sparse Greedy Gaussian Process Regression
TLDR
A simple sparse greedy technique to approximate the maximum a posteriori estimate of Gaussian Processes with much improved scaling behaviour in the sample size m, and shows applications to large scale problems.
Gaussian Processes in Reinforcement Learning
TLDR
It is speculated that the intrinsic ability of GP models to characterise distributions of functions would allow the method to capture entire distributions over future values instead of merely their expectation, which has traditionally been the focus of much of reinforcement learning.
The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition
TLDR
This major new edition features many topics not covered in the original, including graphical models, random forests, ensemble methods, least angle regression and path algorithms for the lasso, non-negative matrix factorization, and spectral clustering.
Bayesian Theory
TLDR
The book Bayesian Theory, first published in hardback in 1994, provides in about 600 pages a very clear, careful and well structured description of the foundations and key theoretical concepts of Bayesian Statistics.
A Generalized Approach to Portfolio Optimization: Improving Performance by Constraining Portfolio Norms
TLDR
A general nonlinear programming framework for identifying portfolios that have superior out-of-sample performance in the presence of estimation error and it is found that the norm-constrained portfolios the authors propose outperform shortsale- Constrained portfolio approaches, shrinkage approaches, the 1/N portfolio, factor portfolios, and also other strategies considered in the literature.
Gaussian elimination is not optimal
t. Below we will give an algorithm which computes the coefficients of the product of two square matrices A and B of order n from the coefficients of A and B with tess than 4 . 7 n l°g7 arithmetical
Statistical Learning for Humanoid Robots
TLDR
It is concluded that real-time learning for complex motor system like humanoid robots is possible with appropriately tailored algorithms, such that increasingly autonomous robots with massive learning abilities should be achievable in the near future.
...
...