Corpus ID: 12890367

No Free Lunch Theorems for Search

@inproceedings{Wolpert1995NoFL,
  title={No Free Lunch Theorems for Search},
  author={David H. Wolpert and William G. Macready},
  year={1995}
}
We show that all algorithms that search for an extremum of a cost function perform exactly the same, when averaged over all possible cost functions. In particular, if algorithm A outperforms algorithm B on some cost functions, then loosely speaking there must exist exactly as many other functions where B outperforms A. Starting from this we analyze a number of the other a priori characteristics of the search problem, like its geometry and its information-theoretic aspects. This analysis allows… Expand
axon.cs.byu.edu
1,152 Citations
Highly Influential Citations
49
Background Citations
444
Methods Citations
124
Results Citations
24

1,152 Citations

Citation Type

Citation Type

Searching for a Practical Evidence of the No Free Lunch Theorems
TLDR
Several test functions for which Random Search performs better than all other considered algorithms have been evolved and show the effectiveness of the proposed evolutionary approach. Expand
What can we learn from No Free Lunch? a first attempt to characterize the concept of a searchable function
TLDR
This work operationally defines a technique for approaching the question of what makes a function searchable in practice and demonstrates the effectiveness of this technique by giving such a field and a corresponding algorithm; the algorithm performs better than random search for small values of this field. Expand
No free lunch theorems for optimization
A framework is developed to explore the connection between effective optimization algorithms and the problems they are solving. A number of "no free lunch" (NFL) theorems are presented which… Expand
No-Free-Lunch theorems and the diversity of algorithms
In this paper, the no-free-lunch theorem is extended to subsets of functions. It is shown that for algorithm a performing better on a set of functions than algorithm b, three has to be another subset… Expand
Free lunches on the discrete Lipschitz class
TLDR
It is concluded that there exist algorithms outperforming random search on the discrete Lipschitz class in both theoretical and practical aspects and indicates that the effectiveness of search heuristics may not be universal but still general in some broad sense. Expand
No Free Lunch Theorem: A Review
TLDR
The objective of this paper is to go through the main research efforts that contributed to this research field, reveal the main issues, and disclose those points that are helpful in understanding the hypotheses, the restrictions, or even the inability of applying No Free Lunch theorems. Expand
Algorithms' local potential-breakfast included?
  • N. Weicker, K. Weicker
  • Mathematics, Medicine
  • Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406)
  • 1999
TLDR
Under certain assumptions concerning the locality of the algorithms it is shown that no local (non-adapting) search algorithm is superior to all other algorithms for all possible populations. Expand
Representation, Search and Genetic Algorithms
TLDR
It is proved that for local neighborhood search on problems of bounded complexity, where complexity is measured In terms of number of basins of attraction in the search space a Gray coded representation is better than Binary in the sense that on average it induces fewer minima in a Hamming distance 1 search neighborhood. Expand
Fundamental Limitations on Search Algorithms: Evolutionary Computing in Perspective
TLDR
This paper extends results and draws out some of their implications for the design of search algorithms, and for the construction of useful representations, and focuses attention on tailoring alg- orithms and representations to particular problem classes by exploiting domain knowledge. Expand
No more lunch: analysis of sequential search
  • T. M. English
  • Mathematics, Computer Science
  • Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753)
  • 2004
Sequential search algorithms of the type predicated in conservation theorems are studied in their own right. With representation of functions as strings, the sets of test functions and search results… Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 27 REFERENCES
Dynamic Hill Climbing: Overcoming the limitations of optimization techniques
This paper describes a novel search algorithm, called dynamic hill climbing, that borrows ideas from genetic algorithms and hill climbing techniques. Unlike both genetic and hill climbing algorithms,… Expand
OFF-TRAINING SET ERROR AND A PRIORI DISTINCTIONS BETWEEN LEARNING ALGORITHMS
This paper uses off-training set (OTS) error to investigate the assumption-free relationship between learning algorithms. It is shown, loosely speaking, that for any two algorithms A and B, there are… Expand
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
TLDR
The results indicate that for real-word datasets similar to the authors', the best method to use for model selection is ten fold stratified cross validation even if computation power allows using more folds. Expand
Adaptation in natural and artificial systems
TLDR
Names of founding work in the area of Adaptation and modiication, which aims to mimic biological optimization, and some (Non-GA) branches of AI. Expand
Elements of Information Theory
TLDR
The author examines the role of entropy, inequality, and randomness in the design of codes and the construction of codes in the rapidly changing environment. Expand
Adaptive Simulated Annealing (ASA)
ADAPTIVE SIMULATED ANNEALING (ASA) ©Lester Ingberingber@ingber.comingber@alumni.caltech.eduAdaptive Simulated Annealing (ASA) is a C-language code developed to statistically find the bestglobal fit… Expand
The traveling salesman: computational solutions for TSP applications
Basic Concepts.- Related Problems and Applications.- Geometric Concepts.- Candidate Sets.- Construction Heuristics.- Improving Solutions.- Heuristics for Large Geometric Problems.- Further Heuristic… Expand
Operations Research
Introduction to Operations ResearchBy A. Kaufmann, and R. Faure. Translated by Henry C. Sneyd. (Mathematics in Science and Engineering, Vol. 47.) Pp. xi + 300. (Academic Press: New York and London,… Expand
Heuristics - intelligent search strategies for computer problem solving
  • J. Pearl
  • Computer Science
  • Addison-Wesley series in artificial intelligence
  • 1984
TLDR
This book presents, characterizes and analyzes problem solving strategies that are guided by heuristic information and provides examples of how these strategies have changed over time. Expand
Alpha, Evidence, and the Entropic Prior
First, the correct entropic prior is computed by marginalization of alpha. This is followed by a discussion of improvements to the “evidence” approximation. Surprisingly, it appears that the… Expand
...
1
2
3
...