COCO: a platform for comparing continuous optimizers in a black-box setting

@article{Hansen2021COCOAP,
  title={COCO: a platform for comparing continuous optimizers in a black-box setting},
  author={Nikolaus Hansen and Anne Auger and Olaf Mersmann and Tea Tusar and Dimo Brockhoff},
  journal={Optimization Methods and Software},
  year={2021},
  volume={36},
  pages={114 - 144}
}
We introduce COCO, an open-source platform for Comparing Continuous Optimizers in a black-box setting. COCO aims at automatizing the tedious and repetitive task of benchmarking numerical optimization algorithms to the greatest possible extent. The platform and the underlying methodology allow to benchmark in the same framework deterministic and stochastic solvers for both single and multiobjective optimization. We present the rationals behind the (decade-long) development of the platform as a… Expand
Mixed-integer benchmark problems for single- and bi-objective optimization
TLDR
Two suites of mixed-integer benchmark problems to be used for analyzing and comparing black-box optimization algorithms and some unexpected findings about their properties are provided. Expand
DMS and MultiGLODS: black-box optimization benchmarking of two direct search methods on the bbob-biobj test suite
TLDR
A small defect in the default initialization of DMS and for both algorithms a decrease in relative performance to other algorithms of the original studies, and an under-performance to previously untested stochastic solvers from the evolutionary computation field, especially when the dimension is higher. Expand
Benchmarking large-scale continuous optimizers: The bbob-largescale testbed, a COCO software guide and beyond
TLDR
A new testbed is presented, called bbob-largescale, that contains functions ranging from dimension 20 to 640, compatible with and extending the well-known single-objective noiseless bbOB test suite to larger dimensions, and uses permuted block diagonal orthogonal matrices to reduce the computational demand of the Orthogonal search space transformations. Expand
Using Well-Understood Single-Objective Functions in Multiobjective Black-Box Optimization Test Suites
TLDR
This paper describes in particular the bbob-biobj test suite with 55 bi-objective functions in continuous domain, and its extended version with 92 bi- objective functions (bbob- biobj-ext), and recommends a general procedure for creating test suites for an arbitrary number of objectives. Expand
Benchmarking for Metaheuristic Black-Box Optimization: Perspectives and Open Challenges
  • R. Sala, Ralf Müller
  • Computer Science, Mathematics
  • 2020 IEEE Congress on Evolutionary Computation (CEC)
  • 2020
TLDR
This communication aims to give a constructive perspective on several open challenges and prospective research directions related to systematic and generalizable benchmarking for black-box optimization. Expand
Benchmarking discrete optimization heuristics with IOHprofiler
TLDR
This work compiles and assesses a selection of discrete optimization problems that subscribe to different types of fitness landscapes, and compares performances of eleven different heuristics for each selected problem. Expand
The SOS Platform: Designing, Tuning and Statistically Benchmarking Optimisation Algorithms
We present Stochastic Optimisation Software (SOS), a Java platform facilitating the algorithmic design process and the evaluation of metaheuristic optimisation algorithms. SOS reduces the burden ofExpand
The SOS Platform: Designing, Tuning and Statistically Benchmarking Optimisation Algorithms
TLDR
It is argued that the vast majority of these algorithms are simply a reformulation of the same methods and that metaheuristics for optimisation should be simply treated as stochastic processes with less emphasis on the inspiring metaphor behind them. Expand
On Using Real-World Problems for Benchmarking Multiobjective Optimization Algorithms
Although the motivation to study multiobjective optimization algorithms comes from practice, there are only a few challenging real-world problems freely available to the research community. BecauseExpand
COCO: The Large Scale Black-Box Optimization Benchmarking (bbob-largescale) Test Suite
TLDR
This documentation presents an approach that replaces a full rotational transformation with a combination of a block-diagonal matrix and two permutation matrices in order to construct test functions whose computational and memory costs scale linearly in the dimension of the problem. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 88 REFERENCES
Mixed-integer benchmark problems for single- and bi-objective optimization
TLDR
Two suites of mixed-integer benchmark problems to be used for analyzing and comparing black-box optimization algorithms and some unexpected findings about their properties are provided. Expand
Using Well-Understood Single-Objective Functions in Multiobjective Black-Box Optimization Test Suites
TLDR
This paper describes in particular the bbob-biobj test suite with 55 bi-objective functions in continuous domain, and its extended version with 92 bi- objective functions (bbob- biobj-ext), and recommends a general procedure for creating test suites for an arbitrary number of objectives. Expand
Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009
TLDR
Results of the BBOB-2009 benchmarking of 31 search algorithms on 24 noiseless functions in a black-box optimization scenario in continuous domain are presented and the choice of the best algorithm depends remarkably on the available budget of function evaluations. Expand
Benchmarking Numerical Multiobjective Optimizers Revisited
TLDR
This work proposes to transfer and adapt standard benchmarking techniques from the single-objective optimization and classical derivative-free optimization community to the field of EMO, and applies this approach to compare three common algorithms on a new test function suite derived from the well-known single- objective BBOB functions. Expand
Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions
TLDR
The testbed of noise-free functions is defined and motivated, and the participants' favorite black-box real-parameter optimizer in a few dimensions a few hundreds of times and execute the provided post-processing script afterwards. Expand
jMetal: A Java framework for multi-objective optimization
TLDR
This paper describes jMetal, an object-oriented Java-based framework aimed at the development, experimentation, and study of metaheuristics for solving multi-objective optimization problems, and includes two case studies to illustrate the use of jMetal in both solving a problem with a metaheuristic and designing and performing an experimental study. Expand
The Impact of Search Volume on the Performance of RANDOMSEARCH on the Bi-objective BBOB-2016 Test Suite
TLDR
It turns out that the suggested region of interest of [-100,100]n (with $n$ being the problem dimension) has a too vast volume for the algorithm to approximate the Pareto set effectively. Expand
Benchmarking RM-MEDA on the Bi-objective BBOB-2016 Test Suite
TLDR
It turns out that, starting from about 200 times dimension many function evaluations, family RM-MEDA shows a linear increase in the solved hypervolume-based target values with time until a stagnation of the performance occurs rather quickly on all problems. Expand
PAVER 2.0: an open source environment for automated performance analysis of benchmarking data
TLDR
PAVER 2.0, an environment for the automated performance analysis of benchmarking data, improves on its predecessor by addressing some of the shortcomings of the original PAVER and making the environment more accessible for the use of and modification by the entire community of potential users. Expand
Single- and multi-objective game-benchmark for evolutionary algorithms
TLDR
This work proposes to use game optimisation problems in order to form a benchmark and implement function suites designed to work with the established COCO benchmarking framework, and creates four function suites based on two optimisationblems previously published in the literature. Expand
...
1
2
3
4
5
...