# Model Comparisons and Model Selections Based on Generalization Criterion Methodology.

@article{Busemeyer2000ModelCA, title={Model Comparisons and Model Selections Based on Generalization Criterion Methodology.}, author={Busemeyer and Wang}, journal={Journal of mathematical psychology}, year={2000}, volume={44 1}, pages={ 171-189 } }

The purpose of this article is to formalize the generalization criterion method for model comparison. The method has the potential to provide powerful comparisons of complex and nonnested models that may also differ in terms of numbers of parameters. The generalization criterion differs from the better known cross-validation criterion in the following critical procedure. Although both employ a calibration stage to estimate parameters, cross-validation employs a replication sample from the same…

## 239 Citations

### Comparison of Decision Learning Models Using the Generalization Criterion Method

- PsychologyCogn. Sci.
- 2008

The results suggest that the models with the prospect utility function can make generalizable predictions to new conditions, and different learning models are needed for making short-versus long-term predictions on simple gambling tasks.

### Key Concepts in Model Selection: Performance and Generalizability.

- BiologyJournal of mathematical psychology
- 2000

It seems that simplicity and parsimony may be an additional factor in managing these errors, in which case the standard methods of model selection are incomplete implementations of Occam's razor.

### A Survey of Model Evaluation Approaches With a Tutorial on Hierarchical Bayesian Methods

- Computer ScienceCogn. Sci.
- 2008

It is argued that hierarchical methods, generally, and hierarchical Bayesian methods, specifically, can provide a more thorough evaluation of models in the cognitive sciences.

### From Anomalies to Forecasts: A Choice Prediction Competition for Decisions under Risk and Ambiguity

- Economics
- 2015

This paper aims to facilitate the derivation of models that can capture the classical choice anomalies and provide useful forecasts of decisions under risk and ambiguity. Study 1 replicates 14…

### Learning and Decision Model Selection for a Class of Complex Adaptive Systems

- Computer Science
- 2007

This article discusses a pragmatic method for selecting between classes of models that are designed to increase understanding in the most significant single factor behind the global climate change, namely human land-use.

### Modeling decisions from experience: How models with a set of parameters for aggregate choices explain individual choices

- Computer Science
- 2017

Testing the ability of aggregate and hierarchical models to explain choices made by individual participants in the sampling paradigm revealed that the aggregate models accounted for individual choices better than hierarchical models upon generalization to problems that were like those encountered during calibration.

### Visualizing The Implicit Model Selection Tradeoff

- Computer ScienceSSRN Electronic Journal
- 2021

Methods for comparing predictive models in an interpretable manner are proposed that synthesize ideas from supervised learning, unsupervised learning, dimensionality reduction, and visualization and demonstrated how they can be used to inform the model selection process.

### A Hierarchical Bayesian Approach to Assessing Between-Experiment Generalizability of Models of Risky Decision Making

- Computer Science
- 2014

This study introduces a hierarchical Bayesian approach, dubbed hierarchical generalization modeling (HGM), to assessing a model’s generalizability between experimental tasks, and demonstrates the soundness and feasibility of HGM for models of risky decision making with human data from decision-from-description and decision- from-experience experiments, and in simulations with artificial data.

## References

SHOWING 1-10 OF 30 REFERENCES

### Model selection criteria: an investigation of relative accuracy, posterior probabilities, and combinations of criteria

- Computer Science
- 1995

It is suggested that general model comparison, model selection, and model probability estimation be performed using the Schwarz criterion, which can be implemented given the model log likelihoods using only a hand calculator.

### Cross-Validation in Regression and Covariance Structure Analysis

- Psychology
- 1992

This article gives an overview of cross-validation techniques in regression and covariance structure analysis. The method of cross-validation offers a means for checking the accuracy or reliability…

### Cross-Validation Methods.

- Environmental ScienceJournal of mathematical psychology
- 2000

It is seen that the optimal number of parameters suggested by both single-sample and two-sample cross-validation indices will depend on sample size.

### Applying Occam’s razor in modeling cognition: A Bayesian approach

- Computer Science
- 1997

This paper introduces a Bayesian model selection approach that formalizes Occam's razor, choosing the simplest model that describes the data well, and can be applied to the comparison of non-nested models as well as nested ones.

### Akaike's Information Criterion and Recent Developments in Information Complexity.

- Computer ScienceJournal of mathematical psychology
- 2000

This paper presents some recent developments on a new entropic or information complexity (ICOMP) criterion of Bozdogan for model selection and operationalizes the general form of ICOMP based on the quantification of the concept of overall model complexity in terms of the estimated inverse-Fisher information matrix.

### Comparing strong and weak models by fitting them to computer-generated data

- Computer Science
- 1985

This paper illustrates the inferential problem posed by this second means of achieving good fit and demonstrates how appropriate control information can be gathered to put inferences from model-fitting on firmer ground.

### Single Sample Cross-Validation Indices for Covariance Structures.

- MathematicsMultivariate behavioral research
- 1989

This article considers single sample approximations for the cross-validation coefficient in the analysis of covariance structures. An adjustment for predictive validity which may be employed in…

### Cross‐Validatory Choice and Assessment of Statistical Predictions

- Business
- 1974

SUMMARY A generalized form of the cross-validation criterion is applied to the choice and assessment of prediction using the data-analytic concept of a prescription. The examples used to illustrate…

### Selectivity, scope, and simplicity of models: a lesson from fitting judgments of perceived depth.

- PsychologyJournal of experimental psychology. General
- 1992

The FLMP's successes may reflect not its sensitivity in capturing psychological process but its scope in fitting any data and its complexity as measured by equation length, which is based on exploration of integration models reflecting depth judgments.

### I. Problems and Designs of Cross-Validation 1

- Psychology
- 1951

THE term &dquo;cross-validation&dquo; is often loosely applied to any one of several distinct, though closely related, experimental designs. Before we get lost in a swamp of semantic confusion, it…