Skip to search formSkip to main contentSkip to account menu
Batch Bayesian Optimization via Local Penalization
TLDR
A simple heuristic based on an estimate of the Lipschitz constant is investigated that captures the most important aspect of this interaction at negligible computational overhead and compares well, in running time, with much more elaborate alternatives.
View PDF on arXiv
Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets
TLDR
A generative model for the validation error as a function of training set size is proposed, which learns during the optimization process and allows exploration of preliminary configurations on small subsets, by extrapolating to the full dataset.
View PDF on arXiv
Entropy Search for Information-Efficient Global Optimization
TLDR
This paper develops desiderata for probabilistic optimization algorithms, then presents a concrete algorithm which addresses each of the computational intractabilities with a sequence of approximations and explicitly addresses the decision problem of maximizing information gain from each evaluation.
View PDF on arXiv
The Randomized Dependence Coefficient
TLDR
The Randomized Dependence Coefficient is introduced, a measure of nonlinear dependence between random variables of arbitrary dimension based on the Hirschfeld-Gebelein-Renyi Maximum Correlation Coefficient, which has low computational cost and is easy to implement.
View PDF on arXiv
Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences
This paper is an attempt to bridge the conceptual gaps between researchers working on the two widely used approaches based on positive definite kernels: Bayesian learning or inference using Gaussian…
View PDF on arXiv
Probabilistic numerics and uncertainty in computations
TLDR
It is shown that the probabilistic view suggests new algorithms that can flexibly be adapted to suit application specifics, while delivering improved empirical performance.
View on Royal Society
Probabilistic Line Searches for Stochastic Optimization
TLDR
A probabilistic line search is constructed by combining the structure of existing deterministic methods with notions from Bayesian optimization, which retains a Gaussian process surrogate of the univariate optimization objective, and uses a probabilism belief over the Wolfe conditions to monitor the descent.
View PDF on arXiv
Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients
TLDR
This analysis extends recent results on adverse effects of ADAM on generalization, isolating the sign aspect as the problematic one and transfers the variance adaptation to SGD gives rise to a novel method, completing the practitioner's toolbox for problems where ADAM fails.
View PDF on arXiv
Limitations of the empirical Fisher approximation for natural gradient descent
TLDR
It is argued that the conditions under which the empirical Fisher approaches the Fisher (and the Hessian) are unlikely to be met in practice, and that, even on simple optimization problems, the pathologies of the empirical fisher can have undesirable effects.
View Paper
Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature
TLDR
A warped model for probabilistic integrands (likelihoods) that are known to be non-negative are introduced, permitting a cheap active learning scheme to optimally select sample locations.
View PDF on arXiv
...
...
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy, Terms of Service, and Dataset License