Bayesian Model Averaging for Linear Regression Models

@article{Raftery1997BayesianMA,
  title={Bayesian Model Averaging for Linear Regression Models},
  author={Adrian E. Raftery and David Madigan and Jennifer A. Hoeting},
  journal={Journal of the American Statistical Association},
  year={1997},
  volume={92},
  pages={179-191}
}
Abstract We consider the problem of accounting for model uncertainty in linear regression models. Conditioning on a single selected model ignores model uncertainty, and thus leads to the underestimation of uncertainty when making inferences about quantities of interest. A Bayesian solution to this problem involves averaging over all possible models (i.e., combinations of predictors) when making inferences about quantities of interest. This approach is often not practical. In this article we… 

Figures and Tables from this paper

An efficient model averaging procedure for logistic regression models using a bayesian estimator with laplace prior
TLDR
This paper extends an idea of Magnus et al. (2009) to the case of logistic regression and uses a Bayesian esimator using a Laplace prior with a special hyperparameter, which can be applied to any generalized linear model.
Bayes model averaging with selection of regressors
Summary. When a number of distinct models contend for use in prediction, the choice of a single model can offer rather unstable predictions. In regression, stochastic search variable selection with
Considerations for assessing model averaging of regression coefficients.
TLDR
There is a gap between the theoretical development of model averaging and its current use in practice, potentially leaving well-intentioned researchers with unclear inferences or difficulties justifying reasons for using (or not using) model averaging, which is attempted to narrow by revisiting some relevant foundations of regression modeling.
Model uncertainty and variable selection in Bayesian lasso regression
TLDR
This paper describes how the marginal likelihood can be accurately computed when the number of predictors in the model is not too large, allowing for model space enumeration when the total number of possible predictors is modest.
On the Quantification of Model Uncertainty: A Bayesian Perspective
TLDR
A detailed and up-to-date review of BMA with a focus on its foundations in Bayesian decision theory and Bayesian predictive modeling and important assumptions regarding BMA are considered.
Credal Model Averaging: dealing robustly with model uncertainty on small data sets
TLDR
Credal Model Averaging is presented, which addresses this problem by simultaneously considering a set of prior probability distributions over the models and allows to represent very weak prior knowledge about the appropriateness of the different models and also to easily accommodate expert judgments.
A Bayesian Approach to Selecting Covariates for Prediction
TLDR
This work develops Bayesian predictive model selection techniques when proper conjugate priors are used and obtains an easily computed expression for the model selection criterion.
Is model averaging the solution for addressing model uncertainty? : methodological insights, tools for assessment, and considerations for practical use
TLDR
A gap is seen between the theoretical development of model averaging and its current use in practice, potentially leaving well-intentioned researchers with unclear inferences or difficulties justifying reasons for using (or not using) model averaging.
A general framework for frequentist model averaging
TLDR
A general frequentist model averaging framework that does not set any restrictions on the set of candidate models is established and a set of optimal weights to combine the individual estimators so that the expected mean squared error of the average estimator is minimized.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 90 REFERENCES
Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window
Abstract We consider the problem of model selection and accounting for model uncertainty in high-dimensional contingency tables, motivated by expert system applications. The approach most used
Approximate Bayes factors and accounting for model uncertainty in generalised linear models
TLDR
A new approximation is proposed which uses only the output of standard computer programs for estimating generalised linear models, based on the Laplace method for integrals, and can be used to implement the Bayesian approach to accounting for model uncertainty.
Bayesian Variable Selection in Linear Regression
Abstract This article is concerned with the selection of subsets of predictor variables in a linear regression model for the prediction of a dependent variable. It is based on a Bayesian approach,
Predictive Model Selection
We consider the problem of selecting one model from a large class of plausible models. A predictive Bayesian viewpoint is advocated to avoid the specification of prior probabilities for the candidate
Hierarchical Bayesian Analysis using Monte Carlo Integration: Computing Posterior Distributions when
TLDR
The use of Monte Carlo integration in hierarchical Bayesian analysis when there are many possible models of different dimensions is illustrated by an example in multiple logistic regression involving 256 possible models.
Computing Bayes Factors by Combining Simulation and Asymptotic Approximations
Abstract The Bayes factor is a ratio of two posterior normalizing constants, which may be difficult to compute. We compare several methods of estimating Bayes factors when it is possible to simulate
Assessment and Propagation of Model Uncertainty
TLDR
A Bayesian approach to solving this problem that has long been available in principle but is only now becoming routinely feasible, by virtue of recent computational advances, is discussed and its implementation is examined in examples that involve forecasting the price of oil and estimating the chance of catastrophic failure of the U.S. Space Shuttle.
Elicitation of Prior Distributions for Variable-Selection Problems in Regression
This paper addresses the problem of quantifying expert opinion about a normal linear regression model when there is uncertainty as to which independent variables should be included in the model.
The effects of model selection on confidence intervals for the size of a closed population.
TLDR
It is noted here that the resulting confidence intervals may well be misleading in that they can fail to provide an adequate coverage probability and be justified only if external considerations justify the assumption of plausible interactions of sources.
...
1
2
3
4
5
...