# No Free Lunch Theorems for Search

@inproceedings{Wolpert1995NoFL, title={No Free Lunch Theorems for Search}, author={David H. Wolpert and William G. Macready}, year={1995} }

We show that all algorithms that search for an extremum of a cost function perform exactly the same, when averaged over all possible cost functions. In particular, if algorithm A outperforms algorithm B on some cost functions, then loosely speaking there must exist exactly as many other functions where B outperforms A. Starting from this we analyze a number of the other a priori characteristics of the search problem, like its geometry and its information-theoretic aspects. This analysis allows… Expand

#### 1,152 Citations

Searching for a Practical Evidence of the No Free Lunch Theorems

- Computer Science
- BioADIT
- 2004

Several test functions for which Random Search performs better than all other considered algorithms have been evolved and show the effectiveness of the proposed evolutionary approach. Expand

What can we learn from No Free Lunch? a first attempt to characterize the concept of a searchable function

- Computer Science
- 2001

This work operationally defines a technique for approaching the question of what makes a function searchable in practice and demonstrates the effectiveness of this technique by giving such a field and a corresponding algorithm; the algorithm performs better than random search for small values of this field. Expand

No free lunch theorems for optimization

- Mathematics, Computer Science
- IEEE Trans. Evol. Comput.
- 1997

A framework is developed to explore the connection between effective optimization algorithms and the problems they are solving. A number of "no free lunch" (NFL) theorems are presented which… Expand

No-Free-Lunch theorems and the diversity of algorithms

- Mathematics, Computer Science
- 2004

In this paper, the no-free-lunch theorem is extended to subsets of functions. It is shown that for algorithm a performing better on a set of functions than algorithm b, three has to be another subset… Expand

Free lunches on the discrete Lipschitz class

- Mathematics, Computer Science
- Theor. Comput. Sci.
- 2011

It is concluded that there exist algorithms outperforming random search on the discrete Lipschitz class in both theoretical and practical aspects and indicates that the effectiveness of search heuristics may not be universal but still general in some broad sense. Expand

No Free Lunch Theorem: A Review

- Computer Science
- Approximation and Optimization
- 2019

The objective of this paper is to go through the main research efforts that contributed to this research field, reveal the main issues, and disclose those points that are helpful in understanding the hypotheses, the restrictions, or even the inability of applying No Free Lunch theorems. Expand

Algorithms' local potential-breakfast included?

- Mathematics, Medicine
- Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406)
- 1999

Under certain assumptions concerning the locality of the algorithms it is shown that no local (non-adapting) search algorithm is superior to all other algorithms for all possible populations. Expand

Representation, Search and Genetic Algorithms

- Computer Science
- AAAI/IAAI
- 1997

It is proved that for local neighborhood search on problems of bounded complexity, where complexity is measured In terms of number of basins of attraction in the search space a Gray coded representation is better than Binary in the sense that on average it induces fewer minima in a Hamming distance 1 search neighborhood. Expand

Fundamental Limitations on Search Algorithms: Evolutionary Computing in Perspective

- Computer Science
- Computer Science Today
- 1995

This paper extends results and draws out some of their implications for the design of search algorithms, and for the construction of useful representations, and focuses attention on tailoring alg- orithms and representations to particular problem classes by exploiting domain knowledge. Expand

No more lunch: analysis of sequential search

- Mathematics, Computer Science
- Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753)
- 2004

Sequential search algorithms of the type predicated in conservation theorems are studied in their own right. With representation of functions as strings, the sets of test functions and search results… Expand

#### References

SHOWING 1-10 OF 27 REFERENCES

Dynamic Hill Climbing: Overcoming the limitations of optimization techniques

- Computer Science
- 1993

This paper describes a novel search algorithm, called dynamic hill climbing, that borrows ideas from genetic algorithms and hill climbing techniques. Unlike both genetic and hill climbing algorithms,… Expand

OFF-TRAINING SET ERROR AND A PRIORI DISTINCTIONS BETWEEN LEARNING ALGORITHMS

- Mathematics
- 1994

This paper uses off-training set (OTS) error to investigate the assumption-free relationship between learning algorithms. It is shown, loosely speaking, that for any two algorithms A and B, there are… Expand

A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection

- Computer Science
- IJCAI
- 1995

The results indicate that for real-word datasets similar to the authors', the best method to use for model selection is ten fold stratified cross validation even if computation power allows using more folds. Expand

Adaptation in natural and artificial systems

- Computer Science
- 1975

Names of founding work in the area of Adaptation and modiication, which aims to mimic biological optimization, and some (Non-GA) branches of AI. Expand

Elements of Information Theory

- Engineering, Computer Science
- 1991

The author examines the role of entropy, inequality, and randomness in the design of codes and the construction of codes in the rapidly changing environment. Expand

Adaptive Simulated Annealing (ASA)

- Materials Science
- 1993

ADAPTIVE SIMULATED ANNEALING (ASA) ©Lester Ingberingber@ingber.comingber@alumni.caltech.eduAdaptive Simulated Annealing (ASA) is a C-language code developed to statistically find the bestglobal fit… Expand

The traveling salesman: computational solutions for TSP applications

- Mathematics
- 1994

Basic Concepts.- Related Problems and Applications.- Geometric Concepts.- Candidate Sets.- Construction Heuristics.- Improving Solutions.- Heuristics for Large Geometric Problems.- Further Heuristic… Expand

Operations Research

- Nature
- 1969

Introduction to Operations ResearchBy A. Kaufmann, and R. Faure. Translated by Henry C. Sneyd. (Mathematics in Science and Engineering, Vol. 47.) Pp. xi + 300. (Academic Press: New York and London,… Expand

Heuristics - intelligent search strategies for computer problem solving

- Computer Science
- Addison-Wesley series in artificial intelligence
- 1984

This book presents, characterizes and analyzes problem solving strategies that are guided by heuristic information and provides examples of how these strategies have changed over time. Expand

Alpha, Evidence, and the Entropic Prior

- Mathematics
- 1993

First, the correct entropic prior is computed by marginalization of alpha. This is followed by a discussion of improvements to the “evidence” approximation. Surprisingly, it appears that the… Expand