Regret Bounds for Expected Improvement Algorithms in Gaussian Process Bandit Optimization

  title={Regret Bounds for Expected Improvement Algorithms in Gaussian Process Bandit Optimization},
  author={Hung Tran-The and S. Gupta and Santu Rana and Svetha Venkatesh},
The expected improvement (EI) algorithm is one of the most popular strategies for optimization under uncertainty due to its simplicity and efficiency. Despite its popularity, the theoretical aspects of this algorithm have not been properly analyzed. In particular, whether in the noisy setting, the EI strategy with a standard incumbent converges is still an open question of the Gaussian process bandit optimization problem. We aim to answer this question by proposing a variant of EI with a standard… 

Figures and Tables from this paper

Adjusted Expected Improvement for Cumulative Regret Minimization in Noisy Bayesian Optimization

A near-optimal regret upper bound of EI for the squared-exponential covariance kernel under mild regularity conditions is established, and experiments are performed to illustrate the improvement of EIC over several popular BO algorithms.

Regret Bounds for Noise-Free Cascaded Kernelized Bandits

This work proposes a sequential upper confidence bound based algorithm GPN-UCB along with a general theoretical upper bound on the cumulative regret and provides algorithm-independent lower bounds on the simple regret and cumulative regret, showing that GPN -UCB is near-optimal for chains and multi-output chains in broad cases of interest.

Regret Bounds for Noise-Free Kernel-Based Bandits

Several upper bounds on regret are discussed; none of which seem order optimal, and a conjecture on the order optimal regret bound is provided.



Convergence Rates of Efficient Global Optimization Algorithms

  • Adam D. Bull
  • Computer Science, Mathematics
    J. Mach. Learn. Res.
  • 2011
This work provides convergence rates for expected improvement, and proposes alternative estimators, chosen to minimize the constants in the rate of convergence, and shows these estimators retain the convergence rates of a fixed prior.

On the Convergence Rates of Expected Improvement Methods

  • I. Ryzhov
  • Computer Science, Mathematics
    Oper. Res.
  • 2016
This work presents new results indicating that, for known sampling noise, variants of EI produce asymptotic simulation allocations that are essentially identical to those chosen by the optimal computing budget allocation (OCBA) methodology, which is known to yield near-optimal asymPTotic performance in ranking and selection.

Improved Algorithms for Linear Stochastic Bandits

A simple modification of Auer's UCB algorithm achieves with high probability constant regret and improves the regret bound by a logarithmic factor, though experiments show a vast improvement.

Adversarially Robust Optimization with Gaussian Processes

It is shown that standard GP optimization algorithms do not exhibit the desired robustness properties, and a novel confidence-bound based algorithm StableOpt is provided for this purpose, which consistently succeeds in finding a stable maximizer where several baseline methods fail.

Using trajectory data to improve bayesian optimization for reinforcement learning

This work shows how to more effectively apply Bayesian Optimization to RL by exploiting the sequential trajectory information generated by RL agents, and shows that the model-based approach developed can recover from model inaccuracies when good transition and reward models cannot be learned.

Theoretical Analysis of Bayesian Optimisation with Unknown Gaussian Process Hyper-Parameters

A cumulative regret bound for Bayesian optimisation with Gaussian processes and unknown kernel hyper-parameters in the stochastic setting is derived, which applies to the expected improvement acquisition function and sub-Gaussian observation noise and provides guidelines on how to design hyper- parameters estimation methods.

Studies on the generalisation of Gaussian processes and Bayesian neural networks

A general distance matrix is able to estimate the effective dimensionality of the regression problem as well as to discover the linear transformation from the manifest variables to the hidden-feature space, with a significant reduction of the input dimension.

Expected improvement for expensive optimization: a review

A comprehensive review of the EI extensions designed for parallel optimization, multiobjective optimization, constrained optimization, noisy optimization,Multi-fidelity optimization and high-dimensional optimization, and the monotonicity properties of the multiobjectives EI criteria and constrained EI criterion are analyzed in detail.

Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization

An IRL framework called Bayesian optimization-IRL is presented which identifies multiple solutions that are consistent with the expert demonstrations by efficiently exploring the reward function space by utilizing Bayesian Optimization and a newly proposed kernel that projects the parameters of policy invariant reward functions to a single point in a latent space.

Interpolation of Spatial Data

  • Rae. Z. H. Aliyev
  • Environmental Science
    Biomedical Journal of Scientific & Technical Research
  • 2018
In the course of the study, methods for correcting and analyzing spatial data recorded in a vector format are disclosed, which is best suited for spatial analysis of discrete objects.