No Free Lunch versus Occam's Razor in Supervised Learning

  title={No Free Lunch versus Occam's Razor in Supervised Learning},
  author={Tor Lattimore and Marcus Hutter},
  booktitle={Algorithmic Probability and Friends},
  • Tor Lattimore, Marcus Hutter
  • Published in
    Algorithmic Probability and…
  • Computer Science, Mathematics
  • The No Free Lunch theorems are often used to argue that domain specific knowledge is required to design successful algorithms. [...] Key Method Additionally, we give a new algorithm for off-line classification, inspired by Solomonoff induction, with good performance on all structured problems under reasonable assumptions. This includes a proof of the efficacy of the well-known heuristic of randomly selecting training data in the hope of reducing misclassification rates.Expand Abstract
    25 Citations
    What is important about the No Free Lunch theorems?
    Free Lunch for optimisation under the universal distribution
    • 11
    • PDF
    Universal Induction and Optimisation: No Free Lunch
    • 1
    • PDF
    Learning Accurate and Interpretable Classifiers Using Optimal Multi-Criteria Rules
    • 5
    A Review of No Free Lunch Theorems, and Their Implications for Metaheuristic Optimisation
    • 28
    Deep learning generalizes because the parameter-function map is biased towards simple functions
    • 48
    • PDF
    Zeta Distribution and Transfer Learning Problem
    • 2
    Optimisation and Bayesian optimality
    • 1
    Machine learning \& artificial intelligence in the quantum domain
    • 221
    • PDF


    No free lunch theorems for optimization
    • 7,357
    • PDF
    On semimeasures predicting Martin-Löf random sequences
    • 14
    • PDF
    The Minimum Description Length Principle
    • 784
    • PDF
    The Supervised Learning No-Free-Lunch Theorems
    • 353
    • PDF
    A Philosophical Treatise of Universal Induction
    • 73
    • PDF