# Consensus-based Optimization and Ensemble Kalman Inversion for Global Optimization Problems with Constraints

@inproceedings{Carrillo2021ConsensusbasedOA, title={Consensus-based Optimization and Ensemble Kalman Inversion for Global Optimization Problems with Constraints}, author={Jos{\'e} Antonio Carrillo and Claudia Totzeck and Urbain Vaes}, year={2021} }

We introduce a practical method for incorporating equality and inequality constraints in global optimization methods based on stochastic interacting particle systems, specifically consensusbased optimization (CBO) and ensemble Kalman inversion (EKI). Unlike other approaches in the literature, the method we propose does not constrain the dynamics to the feasible region of the state space at all times; the particles evolve in the full space, but are attracted towards the feasible set by means of…

## 9 Citations

### Constrained consensus-based optimization

- Computer Science
- 2021

A consensus based multi-objective optimization method on the search space combined with an additional heuristic strategy to adapt parameters during the computations is proposed, which aims to distribute the particles uniformly over the image space by using energy-based measures to quantify the diversity of the system.

### Consensus-Based Optimization for Saddle Point Problems

- Computer ScienceArXiv
- 2022

In this paper, we propose consensus-based optimization for saddle point problems (CBO-SP), a novel multi-particle metaheuristic derivative-free optimization method capable of provably ﬁnding global…

### Consensus based optimization via jump-diffusion stochastic differential equations

- MathematicsMathematical Models and Methods in Applied Sciences
- 2023

Weintroduce a new consensus basedoptimization (CBO)method whereinteracting particle system is driven by jump-diﬀusion stochastic diﬀerential equations. We study well-posedness of the particle system…

### A Consensus-Based Algorithm for Multi-Objective Optimization and Its Mean-Field Description

- Computer Science2022 IEEE 61st Conference on Decision and Control (CDC)
- 2022

A multi-agent algorithm for multi-objective optimization problems, which extends the class of consensus-based optimization methods and relies on a scalarization strategy, and is described by a mean-field model, which is suitable for a theoretical analysis of the algorithm convergence.

### Leveraging Memory Effects and Gradient Information in Consensus-Based Optimization: On Global Convergence in Mean-Field Law

- Computer ScienceArXiv
- 2022

This paper rigorously proves that the underlying dynamics of consensus-based optimization converges to a global minimizer of the objective function in mean-ﬁeld law for a vast class of functions under minimal assumptions on the initialization of the method.

### Efficient derivative-free Bayesian inference for large-scale inverse problems

- MathematicsInverse Problems
- 2022

We consider Bayesian inference for large-scale inverse problems, where computational challenges arise from the need for repeated evaluations of an expensive forward model. This renders most Markov…

### Polarized consensus-based dynamics for optimization and sampling

- Computer Science
- 2022

Polarizing the dynamics with a localizing kernel can be viewed as a bounded conﬁdence model for opinion formation in the presence of common objective and it is proved that the polarized CBS dynamics is unbiased in case of a Gaussian target.

### Swarm-Based Gradient Descent Method for Non-Convex Optimization

- Computer ScienceArXiv
- 2022

Convergence analysis and numerical simulations in one-, two-, and 20-dimensional benchmarks demonstrate the eﬀectiveness of SBGD as a global optimizer.

## References

SHOWING 1-10 OF 58 REFERENCES

### On the Incorporation of Box-Constraints for Ensemble Kalman Inversion

- MathematicsFoundations of Data Science
- 2019

This work proposes a new variant of the ensemble Kalman inversion to include box constraints on the unknown parameters motivated by the theory of projected preconditioned gradient flows, and discusses a complete convergence analysis for linear forward problems.

### Consensus‐based sampling

- Mathematics, Computer ScienceStudies in Applied Mathematics
- 2022

The analysis and numerical simulation establish that the method has potential for general purpose optimization tasks over Euclidean space; contraction properties of the algorithm are established under suitable conditions, and computational experiments demonstrate wide basins of attraction for various specific problems.

### Consensus-based Optimization on the Sphere II: Convergence to Global Minimizers and Machine Learning

- Mathematics, Computer ScienceJ. Mach. Learn. Res.
- 2021

The proof of convergence of the numerical scheme to global minimizers provided conditions of well-preparation of the initial datum is presented, which combines previous results of mean-field limit with a novel asymptotic analysis, and classical convergence results of numerical methods for SDE.

### Ensemble Kalman methods for inverse problems

- Mathematics
- 2012

It is demonstrated that the ensemble Kalman method for inverse problems provides a derivative-free optimization method with comparable accuracy to that achieved by traditional least-squares approaches, and that the accuracy is of the same order of magnitude as that achieve by the best approximation.

### Consensus-based optimization on hypersurfaces: Well-posedness and mean-field limit

- MathematicsMathematical Models and Methods in Applied Sciences
- 2020

The well-posedness of the model is studied and its mean-field approximation for large particle limit is derived rigorously, which shows that as soon as the consensus is reached, then the stochastic component vanishes.

### Analysis of the Ensemble Kalman Filter for Inverse Problems

- MathematicsSIAM J. Numer. Anal.
- 2017

The goal of this paper is to analyze the EnKF when applied to inverse problems with fixed ensemble size, and to demonstrate that the conclusions of the analysis extend beyond the linear inverse problem setting.

### Ensemble Kalman inversion: a derivative-free technique for machine learning tasks

- Computer ScienceInverse Problems
- 2019

An efficient, gradient-free algorithm for finding a solution to classical inverse or filtering problems using ensemble Kalman inversion (EKI), which is inherently parallelizable and is applicable to problems with non-differentiable loss functions, for which back-propagation is not possible.

### Investigation of the sampling performance of ensemble-based methods with a simple reservoir model

- Environmental ScienceComputational Geosciences
- 2013

This paper uses a small but highly nonlinear reservoir model so that it can generate the reference posterior distribution of reservoir properties using a very long chain generated by a Markov chain Monte Carlo sampling algorithm.

### Ensemble Randomized Maximum Likelihood Method as an Iterative Ensemble Smoother

- Environmental ScienceMathematical Geosciences
- 2011

The ensemble Kalman filter (EnKF) is a sequential data assimilation method that has been demonstrated to be effective for history matching reservoir production data and seismic data. To avoid,…

### Interacting Langevin Diffusions: Gradient Structure and Ensemble Kalman Sampler

- Mathematics, Computer ScienceSIAM J. Appl. Dyn. Syst.
- 2020

A new version of such a methodology for solving inverse problems without the use of derivatives or adjoints of the forward model is proposed, and numerical evidence of the practicality of the method is presented.