# ‘All models are wrong...’: an introduction to model uncertainty

@article{Wit2012AllMA, title={‘All models are wrong...’: an introduction to model uncertainty}, author={Ernst C. Wit and Edwin van den Heuvel and Jan-Willem Romeijn}, journal={Statistica Neerlandica}, year={2012}, volume={66} }

In this article, we introduce the concept of model uncertainty. We review the frequentist and Bayesian ideas underlying model selection, which serve as an introduction to the rest of this special issue on ‘All models are wrong...’, a workshop under the same name was held in March 2011 in Groningen to critically examined the field of statistical model selection methods over the past 40 years. We briefly introduce the philosophical debate that is concerned with model selection. We present the…

## 229 Citations

### On model selection in cosmology

- Computer ScienceSciPost Physics Lecture Notes
- 2019

We review some of the common methods for model selection: the goodness of fit, the likelihood ratio test, Bayesian model
selection using Bayes factors, and the classical as well as the Bayesian…

### A Bayesian information criterion for singular models

- Mathematics
- 2013

We consider approximate Bayesian model choice for model selection problems that involve models whose Fisher information matrices may fail to be invertible along other competing submodels. Such…

### Equivalence of non-linear model structures based on Pareto uncertainty

- Business
- 2015

In view of practical limitations, it is not always feasible to find the best model structure. In such situations, a more realistic problem to address seems to be the choice of a set of model…

### Evaluation of Regression Models: Model Assessment, Model Selection and Generalization Error

- Computer ScienceMach. Learn. Knowl. Extr.
- 2019

This paper discusses criterion-based, step-wise selection procedures and resampling methods for model selection, whereas cross-validation provides the most simple and generic means for computationally estimating all required entities.

### Wars and Whales: Extensions and Applications of Confidence Curves and Focused Model Selection

- Mathematics
- 2018

Suppose observations y1, . . . , yn stem from a parametric model f (y, θ ), with the parameter taking one value θL for y1, . . . , yτ and another value θR for yτ+1, . . . , yn. This article provides…

### Comparative Analysis of Behavioral Models for Adaptive Learning in Changing Environments

- Psychology, BiologyFront. Comput. Neurosci.
- 2016

This analysis focuses on two well-established hierarchical probabilistic models that aim at capturing the evolution of beliefs in changing environments: Hierarchical Gaussian Filters and Change Point Models and demonstrates that one can accurately disambiguate between these two models.

### Political Dimensionality Estimation Using a Probabilistic Graphical Model

- Computer ScienceUAI
- 2016

Although the political spectrum is richer than a simple left-right structure, peoples' opinions on seemingly unrelated political issues are very correlated, so fewer than 10 dimensions are enough to represent peoples' entire political opinion.

### A Primer for Model Selection: The Decisive Role of Model Complexity

- Computer Science
- 2018

A classification scheme for model selection criteria that helps to find the right criterion for a specific goal, i.e., which employs the correct complexity interpretation, is proposed and guidance on choosing the right type of criteria for specific model selection tasks is provided.

### Regression model selection via log-likelihood ratio and constrained minimum criterion

- Mathematics
- 2021

Although the log-likelihood is widely used in model selection, the log-likelihood ratio has had few applications in this area. We develop a log-likelihood ratio based method for selecting regression…

### Interactive comment on “ Modelling complex geological angular data with the Projected Normal distribution and mixtures of von Mises distributions ” by R . M . Lark

- Geology
- 2014

It is not uncommon in the geological literature to find that angular data are presented as counts in bins, and for the analysis to proceed from the binned data. It seems that the reviewer thinks that…

## References

SHOWING 1-10 OF 58 REFERENCES

### A Philosophical Analysis of Bayesian model selection

- Economics
- 2008

1 Bayesian Model Selection This chapter provides an answer to the question what it is, philosophically speaking, to choose a model in a statistical procedure, and what this amounts to in the context…

### Key Concepts in Model Selection: Performance and Generalizability.

- BiologyJournal of mathematical psychology
- 2000

It seems that simplicity and parsimony may be an additional factor in managing these errors, in which case the standard methods of model selection are incomplete implementations of Occam's razor.

### Bayes not Bust! Why Simplicity is no Problem for Bayesians1

- Computer ScienceThe British Journal for the Philosophy of Science
- 2007

This work defends a Bayesian alternative: the simplicity of a theory is to be characterised in terms of Wallace's Minimum Message Length (MML), and shows that MML provides answers to many of Forster's objections to Bayesianism.

### A Philosopher’s View on Bayesian Evaluation of Informative Hypotheses

- Psychology
- 2008

This chapter provides an answer to the question: What it is, philosophically speaking, to choose a model in a statistical procedure, and what does this amounts to in the context of a Bayesian…

### The Theory of Statistical Decision

- Economics
- 1951

The critical and philosophical remarks in this exposition may not accurately represent the views of Professor Wald, for both inwriting and lecturing, he prefers to be rather noncommittal on such points.

### Prediction Versus Accommodation and the Risk of Overfitting

- PhilosophyThe British Journal for the Philosophy of Science
- 2004

A new approach to the vexed problem of understanding the epistemic difference between prediction and accommodation is presented, floating the hypothesis that accommodation is a defective methodology only when the methods used to accommodate the data fail to guard against the risk of overfitting.

### The Structure and Dynamics of Scientific Theories: A Hierarchical Bayesian Perspective*

- PhilosophyPhilosophy of Science
- 2010

Hierarchical Bayesian models provide an account of Bayesian inference in a hierarchically structured hypothesis space and help resolve certain issues for Bayesians, such as scientific preference for simplicity and the problem of new theories.

### How to Tell When Simpler, More Unified, or Less Ad Hoc Theories will Provide More Accurate Predictions

- PhilosophyThe British Journal for the Philosophy of Science
- 1994

It is argued that this approach throws light on the theoretical virtues of parsimoniousness, unification, and non ad hocness, on the dispute about Bayesianism, and on empiricism and scientific realism.