‘All models are wrong...’: an introduction to model uncertainty

@article{Wit2012AllMA,
  title={‘All models are wrong...’: an introduction to model uncertainty},
  author={Ernst C. Wit and Edwin van den Heuvel and Jan-Willem Romeijn},
  journal={Statistica Neerlandica},
  year={2012},
  volume={66}
}
In this article, we introduce the concept of model uncertainty. We review the frequentist and Bayesian ideas underlying model selection, which serve as an introduction to the rest of this special issue on ‘All models are wrong...’, a workshop under the same name was held in March 2011 in Groningen to critically examined the field of statistical model selection methods over the past 40 years. We briefly introduce the philosophical debate that is concerned with model selection. We present the… 

On model selection in cosmology

We review some of the common methods for model selection: the goodness of fit, the likelihood ratio test, Bayesian model selection using Bayes factors, and the classical as well as the Bayesian

A Bayesian information criterion for singular models

We consider approximate Bayesian model choice for model selection problems that involve models whose Fisher information matrices may fail to be invertible along other competing submodels. Such

Equivalence of non-linear model structures based on Pareto uncertainty

In view of practical limitations, it is not always feasible to find the  best model structure. In such situations, a more realistic problem to address seems to be the choice of a set of model

Evaluation of Regression Models: Model Assessment, Model Selection and Generalization Error

TLDR
This paper discusses criterion-based, step-wise selection procedures and resampling methods for model selection, whereas cross-validation provides the most simple and generic means for computationally estimating all required entities.

Wars and Whales: Extensions and Applications of Confidence Curves and Focused Model Selection

Suppose observations y1, . . . , yn stem from a parametric model f (y, θ ), with the parameter taking one value θL for y1, . . . , yτ and another value θR for yτ+1, . . . , yn. This article provides

Comparative Analysis of Behavioral Models for Adaptive Learning in Changing Environments

TLDR
This analysis focuses on two well-established hierarchical probabilistic models that aim at capturing the evolution of beliefs in changing environments: Hierarchical Gaussian Filters and Change Point Models and demonstrates that one can accurately disambiguate between these two models.

Political Dimensionality Estimation Using a Probabilistic Graphical Model

TLDR
Although the political spectrum is richer than a simple left-right structure, peoples' opinions on seemingly unrelated political issues are very correlated, so fewer than 10 dimensions are enough to represent peoples' entire political opinion.

A Primer for Model Selection: The Decisive Role of Model Complexity

TLDR
A classification scheme for model selection criteria that helps to find the right criterion for a specific goal, i.e., which employs the correct complexity interpretation, is proposed and guidance on choosing the right type of criteria for specific model selection tasks is provided.

Regression model selection via log-likelihood ratio and constrained minimum criterion

Although the log-likelihood is widely used in model selection, the log-likelihood ratio has had few applications in this area. We develop a log-likelihood ratio based method for selecting regression

Interactive comment on “ Modelling complex geological angular data with the Projected Normal distribution and mixtures of von Mises distributions ” by R . M . Lark

  • Geology
  • 2014
It is not uncommon in the geological literature to find that angular data are presented as counts in bins, and for the analysis to proceed from the binned data. It seems that the reviewer thinks that
...

References

SHOWING 1-10 OF 58 REFERENCES

A Philosophical Analysis of Bayesian model selection

1 Bayesian Model Selection This chapter provides an answer to the question what it is, philosophically speaking, to choose a model in a statistical procedure, and what this amounts to in the context

Model Selection and Model Averaging

Key Concepts in Model Selection: Performance and Generalizability.

  • E M Forster
  • Biology
    Journal of mathematical psychology
  • 2000
TLDR
It seems that simplicity and parsimony may be an additional factor in managing these errors, in which case the standard methods of model selection are incomplete implementations of Occam's razor.

Bayes not Bust! Why Simplicity is no Problem for Bayesians1

TLDR
This work defends a Bayesian alternative: the simplicity of a theory is to be characterised in terms of Wallace's Minimum Message Length (MML), and shows that MML provides answers to many of Forster's objections to Bayesianism.

A Philosopher’s View on Bayesian Evaluation of Informative Hypotheses

This chapter provides an answer to the question: What it is, philosophically speaking, to choose a model in a statistical procedure, and what does this amounts to in the context of a Bayesian

The Theory of Statistical Decision

TLDR
The critical and philosophical remarks in this exposition may not accurately represent the views of Professor Wald, for both inwriting and lecturing, he prefers to be rather noncommittal on such points.

Prediction Versus Accommodation and the Risk of Overfitting

TLDR
A new approach to the vexed problem of understanding the epistemic difference between prediction and accommodation is presented, floating the hypothesis that accommodation is a defective methodology only when the methods used to accommodate the data fail to guard against the risk of overfitting.

AIC, BIC and Recent Advances in Model Selection

The Structure and Dynamics of Scientific Theories: A Hierarchical Bayesian Perspective*

TLDR
Hierarchical Bayesian models provide an account of Bayesian inference in a hierarchically structured hypothesis space and help resolve certain issues for Bayesians, such as scientific preference for simplicity and the problem of new theories.

How to Tell When Simpler, More Unified, or Less Ad Hoc Theories will Provide More Accurate Predictions

TLDR
It is argued that this approach throws light on the theoretical virtues of parsimoniousness, unification, and non ad hocness, on the dispute about Bayesianism, and on empiricism and scientific realism.
...