On the importance of avoiding shortcuts in applying cognitive models to hierarchical data

@article{Boehm2018OnTI,
  title={On the importance of avoiding shortcuts in applying cognitive models to hierarchical data},
  author={Udo Boehm and Maarten Marsman and D{\'o}ra Matzke and Eric-Jan Wagenmakers},
  journal={Behavior Research Methods},
  year={2018},
  volume={50},
  pages={1614 - 1631}
}
Psychological experiments often yield data that are hierarchically structured. A number of popular shortcut strategies in cognitive modeling do not properly accommodate this structure and can result in biased conclusions. To gauge the severity of these biases, we conducted a simulation study for a two-group experiment. We first considered a modeling strategy that ignores the hierarchical data structure. In line with theoretical results, our simulations showed that Bayesian and frequentist… 
A flexible and efficient hierarchical Bayesian approach to the exploration of individual differences in cognitive-model-based neuroscience.
TLDR
This work describes how to apply a approach based on Bayesian hierarchical models that are estimated without reference to neural covariates to efficient exploration of the rich sets of covariates without the computational difficulties associated with refitting the cognitive model.
Assessing the practical differences between model selection methods in inferences about choice response time tasks
  • N. Evans
  • Computer Science
    Psychonomic bulletin & review
  • 2019
TLDR
A systematic comparison between nine different model selection methods using a popular EAM, the linear ballistic accumulator, suggests that researchers should be able to use simpler “parameter counting” methods when applying the LBA and be confident in their inferences, but that researchers need to carefully consider and justify the general class of model selection method that they use.
Using Bayesian regression to test hypotheses about relationships between parameters and covariates in cognitive models
TLDR
This work develops a comprehensive solution to the covariate problem in the form of a Bayesian regression framework that can be easily added to existing cognitive models and allows researchers to quantify the evidential support for relationships between covariates and model parameters using Bayes factors.
Cognitive process modeling addresses context independence violations in the ABCD Study stop-signal task
TLDR
A formal cognitive process model is presented, the BEESTS-ABCD model, that provides a mechanistic explanation for the impact of this design feature, describes key behavioral trends in the ABCD data, and allows biases in SSRT estimates resulting from context independence violations to be quantified.
Evidence accumulation models with R: A practical guide to hierarchical Bayesian methods
TLDR
Efficient, practical software and a step-by-step guide to fit evidence accumulation models with Bayesian methods are provided and an example of fitting complex hierarchical Wiener diffusion models to four shooting-decision data sets is illustrated.
The Reflection Effect in Memory-Based Decisions
TLDR
It is proposed that choosing poorly remembered options is conceptually similar to choosing options with uncertain outcomes, analogous to the reflection effect, according to which uncertainty aversion is stronger in gains than in losses.
Dissociation between asymmetric value updating and perseverance in human reinforcement learning
TLDR
This study addresses the issue of asymmetry in learning and perseverance by using a Hybrid model and shows that empirical data collected from a web-based experiment are governed by perseverance rather than asymmetric learning.
Linking the Ex-Gaussian Parameters to Cognitive Stages: Insights from the Linear Ballistic Accumulator (LBA) Model
The ex-Gaussian distribution has been routinely used by researchers as a descriptive tool to characterize response time data in speeded two-choice tasks. It has also served as a psychological theory
A Simple Method for Comparing Complex Models: Bayesian Model Comparison for Hierarchical Multinomial Processing Tree Models Using Warp-III Bridge Sampling
TLDR
This case study shows how Warp-III bridge sampling can be used to compute the marginal likelihood for hierarchical MPTs, and illustrates the procedure with two published data sets and demonstrates how warp-III facilitates Bayesian model averaging.
...
...

References

SHOWING 1-10 OF 94 REFERENCES
A Bayesian hierarchical diffusion model decomposition of performance in Approach–Avoidance Tasks
TLDR
The advantages of the Bayesian hierarchical diffusion model decomposition are highlighted by applying it to performance on Approach–Avoidance Tasks, widely used in the emotion and psychopathology literature.
Using Bayesian Regression to Incorporate Covariates into Hierarchical Cognitive Models
TLDR
This work develops a comprehensive solution to the covariate problem in the form of a Bayesian regression framework that can be easily added to existing cognitive models and allows researchers to quantify the evidential support for relationships between covariates and model parameters using Bayes factors.
Using Bayesian regression to test hypotheses about relationships between parameters and covariates in cognitive models
TLDR
This work develops a comprehensive solution to the covariate problem in the form of a Bayesian regression framework that can be easily added to existing cognitive models and allows researchers to quantify the evidential support for relationships between covariates and model parameters using Bayes factors.
An introduction to Bayesian hierarchical models with an application in the theory of signal detection
TLDR
This article provides an introduction to Bayesian statistics, hierarchical modeling, and Markov chain Monte Carlo computational techniques and shows that a signal detection analysis of recognition memory data leads to asymptotic underestimation of sensitivity.
A Bayesian model-averaged meta-analysis of the power pose effect with informed and default priors: the case of felt power
ABSTRACTEarlier work found that – compared to participants who adopted constrictive body postures – participants who adopted expansive body postures reported feeling more powerful, showed an increase
A hierarchical bayesian statistical framework for response time distributions
This paper provides a statistical framework for estimating higher-order characteristics of the response time distribution, such as the scale (variability) and shape. Consideration of these higher
A Survey of Model Evaluation Approaches With a Tutorial on Hierarchical Bayesian Methods
TLDR
It is argued that hierarchical methods, generally, and hierarchical Bayesian methods, specifically, can provide a more thorough evaluation of models in the cognitive sciences.
Bayesian t tests for accepting and rejecting the null hypothesis
TLDR
To facilitate use of the Bayes factor, an easy-to-use, Web-based program is provided that performs the necessary calculations and has better properties than other methods of inference that have been advocated in the psychological literature.
...
...