Optimization with randomized search heuristics - the (A)NFL theorem, realistic scenarios, and difficult functions

@article{Droste2002OptimizationWR,
  title={Optimization with randomized search heuristics - the (A)NFL theorem, realistic scenarios, and difficult functions},
  author={Stefan Droste and T. Jansen and Ingo Wegener},
  journal={Theor. Comput. Sci.},
  year={2002},
  volume={287},
  pages={131-144}
}
The No Free Lunch (NFL) theorem due to Wolpert and Macready (IEEE Trans. Evol. Comput. 1(1) (1997) 67) has led to controversial discussions on the usefulness of randomized search heuristics, in particular, evolutionary algorithms. Here a short and simple proof of the NFL theorem is given to show its elementary character. Moreover, the proof method leads to a generalization of the NFL theorem. Afterwards, realistic complexity theoretical-based scenarios for black box optimization are presented… Expand
NFL theorem is unusable on structured classes of problems
  • Benjamin Weinberg, E. Talbi
  • Computer Science
  • Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753)
  • 2004
TLDR
It is proved that k-coloring problems respect such a notion of structure, for any k, which leads to the observation that the notion ofructure of optimization problems is missing in NFL use. Expand
Continuous Lunches Are Free Plus the Design of Optimal Optimization Algorithms
TLDR
It is proved that the natural extension of NFL theorems, for the current formalization of probability, does not hold, but that a weaker form of NFL does hold, by stating the existence of non-trivial distributions of fitness leading to equal performances for all search heuristics. Expand
On Classes of Functions for which No Free Lunch Results Hold
TLDR
The main result of this paper is proven that the fraction of subsets that are c.u.p. is negligibly small, which means that classes of objective functions resulting from important classes of real-world problems are likely not to be c. Expand
No Free Lunch Theorems: Limitations and Perspectives of Metaheuristics
  • C. Igel
  • Computer Science
  • Theory and Principled Methods for the Design of Metaheuristics
  • 2014
TLDR
It is not likely that the preconditions of the NFL theorems are fulfilled for a problem class and thus differences between algorithms exist, therefore, tailored algorithms can exploit structure underlying the optimization problem. Expand
When and why metaheuristics researchers can ignore “No Free Lunch” theorems
TLDR
An argument against a common paraphrase of NFL findings—that algorithms must be specialised to problem domains to do well—after problematising the usually undefined term “domain” is presented, which offers a novel view of the real meaning of NFL. Expand
Recent Results on No-Free-Lunch Theorems for Optimization
The sharpened No-Free-Lunch-theorem (NFL-theorem) states that the performance of all optimization algorithms averaged over any finite set F of functions is equal if and only if F is closed underExpand
Problem hardness for randomized search heuristics with comparison-based selection : a focus on evolutionary algorithms
TLDR
It is shown that one can learn (and generalize) some properties of the search algorithm from the analysis of small instances and also, that a “linear perspective” of RSHs implies interesting predictions. Expand
Bounding the number of favorable functions in stochastic search
  • George D. Montañez
  • Mathematics, Computer Science
  • 2013 IEEE Congress on Evolutionary Computation
  • 2013
TLDR
This work defines favorable functions as those that allow an algorithm to locate a search target with higher probability than uniform random sampling with replacement, and bound the proportion of favorable functions for stochastic search methods, including genetic algorithms. Expand
A framework for co-optimization algorithm performance and its application to worst-case optimization
TLDR
There exist algorithms that are aggregately-optimal for any budget and any starting point and also non-trivially strictly optimal for many budgets and starting points, and this framework can be used to bridge several fields of research, by allowing formalization of their various problems and views on performance. Expand
No Free Lunch and Free Leftovers Theorems for Multiobjective Optimisation Problems
TLDR
A 'Free Leftovers' theorem for comparative performance of algorithms over permutation functions is provided, in words: over the space of permutation problems, every algorithm has some companion algorithm which it outperforms, according to a certain well-behaved metric, when comparative performance is summed over all problems in the space. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 36 REFERENCES
On the Futility of Blind Search: An Algorithmic View of No Free Lunch
  • J. Culberson
  • Computer Science, Medicine
  • Evolutionary Computation
  • 1998
TLDR
It is suggested that the evolution of complex systems exhibiting high degrees of orderliness is not equivalent in difficulty to optimizing hard problems, and that the optimism in genetic algorithms as universal optimizers is not justified by natural evolution. Expand
Approximation Algorithms for NP-Hard Problems
TLDR
This book reviews the design techniques for approximation algorithms and the developments in this area since its inception about three decades ago and the "closeness" to optimum that is achievable in polynomial time. Expand
Fundamental Limitations on Search Algorithms: Evolutionary Computing in Perspective
TLDR
This paper extends results and draws out some of their implications for the design of search algorithms, and for the construction of useful representations, and focuses attention on tailoring alg- orithms and representations to particular problem classes by exploiting domain knowledge. Expand
Perhaps Not a Free Lunch But At Least a Free Appetizer
TLDR
It is argued why the scenario on which the No Free Lunch Theorem is based does not model real life optimization, and why optimization techniques differ in their efficiency. Expand
No free lunch theorems for optimization
A framework is developed to explore the connection between effective optimization algorithms and the problems they are solving. A number of "no free lunch" (NFL) theorems are presented whichExpand
Approximation Algorithms for NP-Hard Problems
TLDR
This book introduces unifying techniques in the analysis of approximation algorithms, intended for computer scientists and operations researchers interested in specific algorithm implementations, as well as design tools for algorithms. Expand
An Introduction to Kolmogorov Complexity and Its Applications
  • Ming Li, P. Vitányi
  • Computer Science, Psychology
  • Texts and Monographs in Computer Science
  • 1993
TLDR
The book presents a thorough treatment of the central ideas and their applications of Kolmogorov complexity with a wide range of illustrative applications, and will be ideal for advanced undergraduate students, graduate students, and researchers in computer science, mathematics, cognitive sciences, philosophy, artificial intelligence, statistics, and physics. Expand
The Theory and Practice of Simulated Annealing
TLDR
This chapter presents practical guidelines for the implementation of simulated annealing in terms of cooling schedules, neighborhood functions, and appropriate applications, as well as recent advances in the analysis of finite time performance. Expand
Evolution and Optimum Seeking: The Sixth Generation
TLDR
The most comprehensive work of its kind, Evolution and Optimum Seeking offers a state-of-the-art perspective on the field for researchers in computer-aided design, planning, control, systems analysis, computational intelligence, and artificial life. Expand
Combinatorial Optimization: Algorithms and Complexity
This clearly written , mathematically rigorous text includes a novel algorithmic exposition of the simplex method and also discusses the Soviet ellipsoid algorithm for linear programming; efficientExpand
...
1
2
3
4
...