• Corpus ID: 242757787

Consensus-based Optimization and Ensemble Kalman Inversion for Global Optimization Problems with Constraints

@inproceedings{Carrillo2021ConsensusbasedOA,
  title={Consensus-based Optimization and Ensemble Kalman Inversion for Global Optimization Problems with Constraints},
  author={Jos{\'e} Antonio Carrillo and Claudia Totzeck and Urbain Vaes},
  year={2021}
}
We introduce a practical method for incorporating equality and inequality constraints in global optimization methods based on stochastic interacting particle systems, specifically consensusbased optimization (CBO) and ensemble Kalman inversion (EKI). Unlike other approaches in the literature, the method we propose does not constrain the dynamics to the feasible region of the state space at all times; the particles evolve in the full space, but are attracted towards the feasible set by means of… 
View PDF on arXiv
9 Citations
Highly Influential Citations
1
Background Citations
2
Methods Citations
1

9 Citations

Constrained consensus-based optimization

A consensus based multi-objective optimization method on the search space combined with an additional heuristic strategy to adapt parameters during the computations is proposed, which aims to distribute the particles uniformly over the image space by using energy-based measures to quantify the diversity of the system.

Consensus-Based Optimization for Saddle Point Problems

In this paper, we propose consensus-based optimization for saddle point problems (CBO-SP), a novel multi-particle metaheuristic derivative-free optimization method capable of provably finding global…

Consensus based optimization via jump-diffusion stochastic differential equations

Weintroduce a new consensus basedoptimization (CBO)method whereinteracting particle system is driven by jump-diffusion stochastic differential equations. We study well-posedness of the particle system…

A Consensus-Based Algorithm for Multi-Objective Optimization and Its Mean-Field Description

A multi-agent algorithm for multi-objective optimization problems, which extends the class of consensus-based optimization methods and relies on a scalarization strategy, and is described by a mean-field model, which is suitable for a theoretical analysis of the algorithm convergence.

Leveraging Memory Effects and Gradient Information in Consensus-Based Optimization: On Global Convergence in Mean-Field Law

This paper rigorously proves that the underlying dynamics of consensus-based optimization converges to a global minimizer of the objective function in mean-field law for a vast class of functions under minimal assumptions on the initialization of the method.

Efficient derivative-free Bayesian inference for large-scale inverse problems

We consider Bayesian inference for large-scale inverse problems, where computational challenges arise from the need for repeated evaluations of an expensive forward model. This renders most Markov…

Polarized consensus-based dynamics for optimization and sampling

Polarizing the dynamics with a localizing kernel can be viewed as a bounded confidence model for opinion formation in the presence of common objective and it is proved that the polarized CBS dynamics is unbiased in case of a Gaussian target.

Swarm-Based Gradient Descent Method for Non-Convex Optimization

Convergence analysis and numerical simulations in one-, two-, and 20-dimensional benchmarks demonstrate the effectiveness of SBGD as a global optimizer.

Non-intrusive surrogate modelling using sparse random features with applications in crashworthiness analysis

,

References

SHOWING 1-10 OF 58 REFERENCES

On the Incorporation of Box-Constraints for Ensemble Kalman Inversion

This work proposes a new variant of the ensemble Kalman inversion to include box constraints on the unknown parameters motivated by the theory of projected preconditioned gradient flows, and discusses a complete convergence analysis for linear forward problems.

Consensus‐based sampling

The analysis and numerical simulation establish that the method has potential for general purpose optimization tasks over Euclidean space; contraction properties of the algorithm are established under suitable conditions, and computational experiments demonstrate wide basins of attraction for various specific problems.

Consensus-based Optimization on the Sphere II: Convergence to Global Minimizers and Machine Learning

The proof of convergence of the numerical scheme to global minimizers provided conditions of well-preparation of the initial datum is presented, which combines previous results of mean-field limit with a novel asymptotic analysis, and classical convergence results of numerical methods for SDE.

Ensemble Kalman methods for inverse problems

It is demonstrated that the ensemble Kalman method for inverse problems provides a derivative-free optimization method with comparable accuracy to that achieved by traditional least-squares approaches, and that the accuracy is of the same order of magnitude as that achieve by the best approximation.

Consensus-based optimization on hypersurfaces: Well-posedness and mean-field limit

The well-posedness of the model is studied and its mean-field approximation for large particle limit is derived rigorously, which shows that as soon as the consensus is reached, then the stochastic component vanishes.

Analysis of the Ensemble Kalman Filter for Inverse Problems

The goal of this paper is to analyze the EnKF when applied to inverse problems with fixed ensemble size, and to demonstrate that the conclusions of the analysis extend beyond the linear inverse problem setting.

Ensemble Kalman inversion: a derivative-free technique for machine learning tasks

An efficient, gradient-free algorithm for finding a solution to classical inverse or filtering problems using ensemble Kalman inversion (EKI), which is inherently parallelizable and is applicable to problems with non-differentiable loss functions, for which back-propagation is not possible.

Investigation of the sampling performance of ensemble-based methods with a simple reservoir model

This paper uses a small but highly nonlinear reservoir model so that it can generate the reference posterior distribution of reservoir properties using a very long chain generated by a Markov chain Monte Carlo sampling algorithm.

Ensemble Randomized Maximum Likelihood Method as an Iterative Ensemble Smoother

The ensemble Kalman filter (EnKF) is a sequential data assimilation method that has been demonstrated to be effective for history matching reservoir production data and seismic data. To avoid,…

Interacting Langevin Diffusions: Gradient Structure and Ensemble Kalman Sampler

A new version of such a methodology for solving inverse problems without the use of derivatives or adjoints of the forward model is proposed, and numerical evidence of the practicality of the method is presented.
...