Continuous Lunches Are Free Plus the Design of Optimal Optimization Algorithms

@article{Auger2008ContinuousLA,
  title={Continuous Lunches Are Free Plus the Design of Optimal Optimization Algorithms},
  author={Anne Auger and Olivier Teytaud},
  journal={Algorithmica},
  year={2008},
  volume={57},
  pages={121-146}
}
This paper analyses extensions of No-Free-Lunch (NFL) theorems to countably infinite and uncountable infinite domains and investigates the design of optimal optimization algorithms.The original NFL theorem due to Wolpert and Macready states that, for finite search domains, all search heuristics have the same performance when averaged over the uniform distribution over all possible functions. For infinite domains the extension of the concept of distribution over all possible functions involves… Expand
A Probabilistic Reformulation of No Free Lunch: Continuous Lunches Are Not Free
TLDR
A new formalization of probabilistic NFL is developed that is sufficiently expressive to prove the existence of NFL in large search domains, such as continuous spaces or function spaces, and fills an important gap in the study of performance of stochastic optimizers. Expand
No Free Lunch Theorems: Limitations and Perspectives of Metaheuristics
  • C. Igel
  • Computer Science
  • Theory and Principled Methods for the Design of Metaheuristics
  • 2014
TLDR
It is not likely that the preconditions of the NFL theorems are fulfilled for a problem class and thus differences between algorithms exist, therefore, tailored algorithms can exploit structure underlying the optimization problem. Expand
No Free Lunch Theorem: A Review
TLDR
The objective of this paper is to go through the main research efforts that contributed to this research field, reveal the main issues, and disclose those points that are helpful in understanding the hypotheses, the restrictions, or even the inability of applying No Free Lunch theorems. Expand
When and why metaheuristics researchers can ignore “No Free Lunch” theorems
TLDR
An argument against a common paraphrase of NFL findings—that algorithms must be specialised to problem domains to do well—after problematising the usually undefined term “domain” is presented, which offers a novel view of the real meaning of NFL. Expand
No-Free-Lunch theorems in the continuum
TLDR
This paper provides another approach, which is simpler, requires less assumptions, relates the discrete and continuum cases, and believes that clarifies the role of the cardinality and structure of the domain. Expand
Parameter Tuning by Simple Regret Algorithms and Multiple Simultaneous Hypothesis Testing
TLDR
It is seen that for moderate numbers of arms, the possible improvement in terms of computational power required for statistical validation can't be more than linear as a function of the number of arms and provide a simple rule to check if the simple uniform al- gorithm (trivially parallel) is relevant. Expand
Arbitrary function optimisation with metaheuristics
TLDR
This work proposes an empirical framework, arbitrary function optimisation framework, that allows researchers to formulate conclusions independent of the benchmark problems that were actually addressed, as long as the context of the problem class is mentioned, and presents the first thorough empirical study on the no free lunch theorems. Expand
A framework for co-optimization algorithm performance and its application to worst-case optimization
TLDR
There exist algorithms that are aggregately-optimal for any budget and any starting point and also non-trivially strictly optimal for many budgets and starting points, and this framework can be used to bridge several fields of research, by allowing formalization of their various problems and views on performance. Expand
A Review of No Free Lunch Theorems, and Their Implications for Metaheuristic Optimisation
TLDR
It is shown that understanding the No Free Lunch theorems brings us to a position where the authors can ask about the specific dynamics of an optimisation algorithm, and how those dynamics relate to the properties of optimisation problems. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 48 REFERENCES
A No-Free-Lunch theorem for non-uniform distributions of target functions
The sharpened No-Free-Lunch-theorem (NFL-theorem) states that, regardless of the performance measure, the performance of all optimization algorithms averaged uniformly over any finite set F ofExpand
Optimization with randomized search heuristics - the (A)NFL theorem, realistic scenarios, and difficult functions
TLDR
An Almost No Free Lunch (ANFL) theorem shows that for each function which can be optimized efficiently by a search heuristic there can be constructed many related functions where the same heuristic is bad. Expand
On the Futility of Blind Search: An Algorithmic View of No Free Lunch
  • J. Culberson
  • Computer Science, Medicine
  • Evolutionary Computation
  • 1998
TLDR
It is suggested that the evolution of complex systems exhibiting high degrees of orderliness is not equivalent in difficulty to optimizing hard problems, and that the optimism in genetic algorithms as universal optimizers is not justified by natural evolution. Expand
No free lunch theorems for optimization
A framework is developed to explore the connection between effective optimization algorithms and the problems they are solving. A number of "no free lunch" (NFL) theorems are presented whichExpand
Some multiobjective optimizers are better than others
The No-Free-Lunch (NFL) theorems hold for general multiobjective fitness spaces, in the sense that, over a space of problems which is closed under permutation, any two algorithms will produce theExpand
Perhaps Not a Free Lunch But At Least a Free Appetizer
TLDR
It is argued why the scenario on which the No Free Lunch Theorem is based does not model real life optimization, and why optimization techniques differ in their efficiency. Expand
Completely Derandomized Self-Adaptation in Evolution Strategies
TLDR
This paper puts forward two useful methods for self-adaptation of the mutation distribution - the concepts of derandomization and cumulation and reveals local and global search properties of the evolution strategy with and without covariance matrix adaptation. Expand
Fundamental Limitations on Search Algorithms: Evolutionary Computing in Perspective
TLDR
This paper extends results and draws out some of their implications for the design of search algorithms, and for the construction of useful representations, and focuses attention on tailoring alg- orithms and representations to particular problem classes by exploiting domain knowledge. Expand
The No Free Lunch and problem description length
The No Free Lunch theorem is reviewed and cast within a simple framework for black-box search. A duality result which relates functions being optimized to algorithms optimizing them is obtained andExpand
Random fields, analysis and synthesis
Random variation over space and time is one of the few attributes that might safely be predicted as characterizing almost any given complex system. Random fields or "distributed disorder systems"Expand
...
1
2
3
4
5
...