Cross-Validation-based Adaptive Sampling for Gaussian Process Models

@article{Mohammadi2022CrossValidationbasedAS,
  title={Cross-Validation-based Adaptive Sampling for Gaussian Process Models},
  author={Hossein Mohammadi and Peter Challenor and Daniel Williamson and Marc Goodfellow},
  journal={SIAM/ASA J. Uncertain. Quantification},
  year={2022},
  volume={10},
  pages={294-316}
}
In many real-world applications, we are interested in approximating black-box, costly functions as accurately as possible with the smallest number of function evaluations. A complex computer code is an example of such a function. In this work, a Gaussian process (GP) emulator is used to approximate the output of complex computer code. We consider the problem of extending an initial experiment sequentially to improve the emulator. A sequential sampling approach based on leave-one-out (LOO) cross… 
Sequential adaptive design for emulating costly computer codes
TLDR
The results suggest that the proposed adaptive sampling criterion, VIGF, has a superior performance in predicting the benchmarking functions in most cases.
Calibrated and recalibrated expected improvements for Bayesian optimization
TLDR
It is argued that, in addition to the “over-exploitation” of EI, EI can also get trapped in querying samples with maximum Sigma but poor objective function value, which leads to the sub-optimal convergence rate of BO.

References

SHOWING 1-10 OF 83 REFERENCES
Kriging-based sequential design strategies using fast cross-validation techniques with extensions to multi-fidelity computer codes
TLDR
A method of kriging-based sequential design which combines both the error evaluation providing by the kriged model and the observed errors of a Leave-One-Out cross-validation procedure is proposed.
A Novel Hybrid Sequential Design Strategy for Global Surrogate Modeling of Computer Experiments
TLDR
A novel hybrid sequential design strategy is proposed which uses a Monte Carlo-based approximation of a Voronoi tessellation for exploration and local linear approximations of the simulator for exploitation, and can be used in heterogeneous modeling environments, where multiple model types are used at the same time.
Universal Prediction Distribution for Surrogate Models
TLDR
A universal method to define a measure of uncertainty suitable for any surrogate model either deterministic or probabilistic, based on Cross-Validation sub-models predictions is proposed, called the Universal Prediction distribution (UP distribution), which allows the definition of many sampling criteria.
Computer experiments and global optimization
TLDR
This thesis considers the problem of finding the global optimum of the response with few function evaluations and presents strategies to conduct the optimization in stages and for optimization subject to constraints on additional response variables.
Regression and Classification Using Gaussian Process Priors
TLDR
Gaussian processes are in my view the simplest and most obvious way of defining flexible Bayesian regression and classification models, but despite some past usage, they appear to have been rather neglected as a general-purpose technique.
Adaptive Design and Analysis of Supercomputer Experiments
TLDR
This work proposes an approach that automatically explores the space while simultaneously fitting the response surface, using predictive uncertainty to guide subsequent experimental runs, and develops an adaptive sequential design framework to cope with an asynchronous, random, agent–based supercomputing environment.
Cross validation of kriging in a unique neighborhood
TLDR
This work presents a much faster method for cross validation in a unique neighborhood that only requires the inversion of one n × nmatrix, and generalized this method to leaving out several points instead of one.
Efficient Global Optimization of Expensive Black-Box Functions
TLDR
This paper introduces the reader to a response surface methodology that is especially good at modeling the nonlinear, multimodal functions that often occur in engineering and shows how these approximating functions can be used to construct an efficient global optimization algorithm with a credible stopping rule.
An Adaptive Bayesian Sequential Sampling Approach for Global Metamodeling
TLDR
This article investigates the performance of current Bayesian sampling approaches and proposes an adaptive maximum entropy (AME) approach, and demonstrates that the AME approach is very promising for global metamodeling.
Sequential adaptive designs in computer experiments for response surface model fit
TLDR
This thesis considers sequential adaptive designs as an “efficient” alternative to fixed-point designs and proposes new adaptive design criteria based on a cross validation approach and on an expected improvement criterion, the latter inspired by a criterion originally proposed for global optimization.
...
...