# Continuous Lunches Are Free Plus the Design of Optimal Optimization Algorithms

@article{Auger2008ContinuousLA, title={Continuous Lunches Are Free Plus the Design of Optimal Optimization Algorithms}, author={Anne Auger and Olivier Teytaud}, journal={Algorithmica}, year={2008}, volume={57}, pages={121-146} }

This paper analyses extensions of No-Free-Lunch (NFL) theorems to countably infinite and uncountable infinite domains and investigates the design of optimal optimization algorithms.The original NFL theorem due to Wolpert and Macready states that, for finite search domains, all search heuristics have the same performance when averaged over the uniform distribution over all possible functions. For infinite domains the extension of the concept of distribution over all possible functions involves… Expand

#### 76 Citations

A Probabilistic Reformulation of No Free Lunch: Continuous Lunches Are Not Free

- Mathematics, Computer Science
- Evolutionary Computation
- 2017

A new formalization of probabilistic NFL is developed that is sufficiently expressive to prove the existence of NFL in large search domains, such as continuous spaces or function spaces, and fills an important gap in the study of performance of stochastic optimizers. Expand

No Free Lunch Theorems: Limitations and Perspectives of Metaheuristics

- Computer Science
- Theory and Principled Methods for the Design of Metaheuristics
- 2014

It is not likely that the preconditions of the NFL theorems are fulfilled for a problem class and thus differences between algorithms exist, therefore, tailored algorithms can exploit structure underlying the optimization problem. Expand

No Free Lunch Theorem: A Review

- Computer Science
- 2019

The objective of this paper is to go through the main research efforts that contributed to this research field, reveal the main issues, and disclose those points that are helpful in understanding the hypotheses, the restrictions, or even the inability of applying No Free Lunch theorems. Expand

When and why metaheuristics researchers can ignore “No Free Lunch” theorems

- Computer Science
- ArXiv
- 2019

An argument against a common paraphrase of NFL findings—that algorithms must be specialised to problem domains to do well—after problematising the usually undefined term “domain” is presented, which offers a novel view of the real meaning of NFL. Expand

When and Why Metaheuristics Researchers can Ignore "No Free Lunch" Theorems

- Computer Science
- SN Comput. Sci.
- 2020

No-Free-Lunch theorems in the continuum

- Mathematics, Computer Science
- Theor. Comput. Sci.
- 2015

This paper provides another approach, which is simpler, requires less assumptions, relates the discrete and continuum cases, and believes that clarifies the role of the cardinality and structure of the domain. Expand

Parameter Tuning by Simple Regret Algorithms and Multiple Simultaneous Hypothesis Testing

- Mathematics, Computer Science
- ICINCO
- 2010

It is seen that for moderate numbers of arms, the possible improvement in terms of computational power required for statistical validation can't be more than linear as a function of the number of arms and provide a simple rule to check if the simple uniform al- gorithm (trivially parallel) is relevant. Expand

Arbitrary function optimisation with metaheuristics

- Computer Science, Mathematics
- Soft Comput.
- 2012

This work proposes an empirical framework, arbitrary function optimisation framework, that allows researchers to formulate conclusions independent of the benchmark problems that were actually addressed, as long as the context of the problem class is mentioned, and presents the first thorough empirical study on the no free lunch theorems. Expand

A framework for co-optimization algorithm performance and its application to worst-case optimization

- Computer Science, Mathematics
- Theor. Comput. Sci.
- 2015

There exist algorithms that are aggregately-optimal for any budget and any starting point and also non-trivially strictly optimal for many budgets and starting points, and this framework can be used to bridge several fields of research, by allowing formalization of their various problems and views on performance. Expand

A Review of No Free Lunch Theorems, and Their Implications for Metaheuristic Optimisation

- Computer Science
- 2018

It is shown that understanding the No Free Lunch theorems brings us to a position where the authors can ask about the specific dynamics of an optimisation algorithm, and how those dynamics relate to the properties of optimisation problems. Expand

#### References

SHOWING 1-10 OF 48 REFERENCES

A No-Free-Lunch theorem for non-uniform distributions of target functions

- Computer Science
- J. Math. Model. Algorithms
- 2004

The sharpened No-Free-Lunch-theorem (NFL-theorem) states that, regardless of the performance measure, the performance of all optimization algorithms averaged uniformly over any finite set F of… Expand

Optimization with randomized search heuristics - the (A)NFL theorem, realistic scenarios, and difficult functions

- Computer Science, Mathematics
- Theor. Comput. Sci.
- 2002

An Almost No Free Lunch (ANFL) theorem shows that for each function which can be optimized efficiently by a search heuristic there can be constructed many related functions where the same heuristic is bad. Expand

On the Futility of Blind Search: An Algorithmic View of No Free Lunch

- Computer Science, Medicine
- Evolutionary Computation
- 1998

It is suggested that the evolution of complex systems exhibiting high degrees of orderliness is not equivalent in difficulty to optimizing hard problems, and that the optimism in genetic algorithms as universal optimizers is not justified by natural evolution. Expand

No free lunch theorems for optimization

- Mathematics, Computer Science
- IEEE Trans. Evol. Comput.
- 1997

A framework is developed to explore the connection between effective optimization algorithms and the problems they are solving. A number of "no free lunch" (NFL) theorems are presented which… Expand

Some multiobjective optimizers are better than others

- Mathematics
- The 2003 Congress on Evolutionary Computation, 2003. CEC '03.
- 2003

The No-Free-Lunch (NFL) theorems hold for general multiobjective fitness spaces, in the sense that, over a space of problems which is closed under permutation, any two algorithms will produce the… Expand

Perhaps Not a Free Lunch But At Least a Free Appetizer

- Mathematics, Computer Science
- GECCO
- 1999

It is argued why the scenario on which the No Free Lunch Theorem is based does not model real life optimization, and why optimization techniques differ in their efficiency. Expand

Completely Derandomized Self-Adaptation in Evolution Strategies

- Computer Science, Medicine
- Evolutionary Computation
- 2001

This paper puts forward two useful methods for self-adaptation of the mutation distribution - the concepts of derandomization and cumulation and reveals local and global search properties of the evolution strategy with and without covariance matrix adaptation. Expand

Fundamental Limitations on Search Algorithms: Evolutionary Computing in Perspective

- Computer Science
- Computer Science Today
- 1995

This paper extends results and draws out some of their implications for the design of search algorithms, and for the construction of useful representations, and focuses attention on tailoring alg- orithms and representations to particular problem classes by exploiting domain knowledge. Expand

The No Free Lunch and problem description length

- Mathematics
- 2001

The No Free Lunch theorem is reviewed and cast within a simple framework for black-box search. A duality result which relates functions being optimized to algorithms optimizing them is obtained and… Expand

Random fields, analysis and synthesis

- Computer Science, Mathematics
- 1983

Random variation over space and time is one of the few attributes that might safely be predicted as characterizing almost any given complex system. Random fields or "distributed disorder systems"… Expand