Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning

@article{Yarkoni2017ChoosingPO,
  title={Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning},
  author={Tal Yarkoni and Jacob Westfall},
  journal={Perspectives on Psychological Science},
  year={2017},
  volume={12},
  pages={1100 - 1122}
}
Psychology has historically been concerned, first and foremost, with explaining the causal mechanisms that give rise to behavior. Randomized, tightly controlled experiments are enshrined as the gold standard of psychological research, and there are endless investigations of the various mediating and moderating variables that govern various behaviors. We argue that psychology’s near-total focus on explaining the causes of behavior has led much of the field to be populated by research programs… Expand
Putting Psychology to the Test: Rethinking Model Evaluation Through Benchmarking and Prediction
Consensus on standards for evaluating models and theories is an integral part of every science. Nonetheless, in psychology, relatively little focus has been placed on defining reliable communalExpand
Machine Learning and Psychological Research: The Unexplored Effect of Measurement
  • R. Jacobucci, K. Grimm
  • Computer Science, Medicine
  • Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2020
TLDR
It is argued that this phenomenon results from measurement errors that prevent machine-learning algorithms from accurately modeling nonlinear relationships, if indeed they exist, and that model selection between a machine- learning algorithm and regression depends on the measurement quality, regardless of sample size. Expand
Using Machine Learning to Guide Cognitive Modeling: A Case Study in Moral Reasoning
TLDR
This paper outlines a data-driven, iterative procedure that allows cognitive scientists to use machine learning to generate models that are both interpretable and accurate in the domain of moral decision-making. Expand
Psychological Foundations of Explainability and Interpretability in Artificial Intelligence
TLDR
It is made the case that interpretability and explainability are distinct requirements for machine learning systems and that humans differ from one another in systematic ways, that affect the extent to which they prefer to make decisions based on detailed explanations versus less precise interpretations. Expand
Machine Learning Modeling: A New Way to do Quantitative Research in Social Sciences in the Era of AI
TLDR
The sources and internal mechanisms of overfitting are discussed, and machine learning modeling is introduced by discussing its high-level ideas, goals, and concrete methods, and the shortcomings and limiting factors of machine learning models are discussed. Expand
PredPsych: A toolbox for predictive machine learning-based approach in experimental psychology research
TLDR
The framework of PredPsych is presented via the analysis of a recently published multiple-subject motion capture dataset, and it is anticipated that PredPsych will be of use to researchers with limited programming experience not only in the field of psychology, but also in that of clinical neuroscience. Expand
Scaling up psychology via Scientific Regret Minimization
TLDR
An exploratory approach that combines ideas from machine learning and psychology is offered, and it is found that incorporating a set of deontological principles that capture dimensions along which groups of agents can vary improves a computational model of human moral judgment. Expand
Scaling up Psychology via Scientific Regret Minimization: A Case Study in Moral Decision-Making
TLDR
It is found that incorporating a set of deontological principles that capture dimensions along which groups of agents can vary improves a computational model of human moral judgment, and three interesting moral phenomena are identified and independently validated. Expand
Cognitive Model Priors for Predicting Human Decisions
TLDR
This work constructs "cognitive model priors" by pretraining neural networks with synthetic data generated by cognitive models (i.e., theoretical models developed by cognitive psychologists), and finds that fine-tuning these networks on small datasets of real human decisions results in unprecedented state-of-the-art improvements on two benchmark datasets. Expand
What failure to predict life outcomes can teach us
  • F. Garip
  • Computer Science, Medicine
  • Proceedings of the National Academy of Sciences
  • 2020
Social scientists are increasingly turning to supervised machine learning (SML), a set of methods optimized for using inputs from data to forecast an unobserved outcome, to offer predictions to aidExpand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 129 REFERENCES
Estimating the reproducibility of psychological science
TLDR
A large-scale assessment suggests that experimental reproducibility in psychology leaves a lot to be desired, and correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams. Expand
To Explain or to Predict
TLDR
The purpose of this article is to clarify the distinction between explanatory and predictive modeling, to discuss its sources, and to reveal the practical implications of the distinction to each step in the modeling process. Expand
Statistically Controlling for Confounding Constructs Is Harder than You Think
TLDR
It is found that error rates are highest—in some cases approaching 100%—when sample sizes are large and reliability is moderate, and SEM-based statistical approaches that appropriately control the Type I error rate when attempting to establish incremental validity are reviewed. Expand
Mapping collective behavior in the big-data era
TLDR
The field of discrete-choice theory is drawn on to create a multiscale comparative “map” that captures the essence of decision making along two axes, providing an easily understood empirical framework for evaluating how modern collective behavior may be changing in the digital age. Expand
Psychoinformatics: New Horizons at the Interface of the Psychological and Computing Sciences
Psychologists live in an increasingly data-rich world, and our ability to make continued progress in understanding the mind and brain depends on finding new ways to organize and synthesize anExpand
The Rules of the Game Called Psychological Science
TLDR
This paper considers 13 meta-analyses covering 281 primary studies in various fields of psychology and finds indications of biases and/or an excess of significant results in seven, highlighting the need for sufficiently powerful replications and changes in journal policies. Expand
An Agenda for Purely Confirmatory Research
TLDR
This article proposes that researchers preregister their studies and indicate in advance the analyses they intend to conduct, and proposes that only these analyses deserve the label “confirmatory,” and only for these analyses are the common statistical tests valid. Expand
Using Lasso for Predictor Selection and to Assuage Overfitting: A Method Long Overlooked in Behavioral Sciences
  • D. McNeish
  • Computer Science, Medicine
  • Multivariate behavioral research
  • 2015
TLDR
Potential issues with standard statistical models are discussed, an introduction to regularization is provided with specific details on both Lasso and its related predecessor ridge regression, and an example analysis and code for running a Lasso analysis in R and SAS are provided. Expand
50 Years of Data Science
ABSTRACT More than 50 years ago, John Tukey called for a reformation of academic statistics. In “The Future of Data Analysis,” he pointed to the existence of an as-yet unrecognized science, whoseExpand
Estimating Coefficients in Linear Models: It Don't Make No Nevermind
It is proved that under very general circumstances coefficients in multiple regression models can be replaced with equal weights with almost no loss in accuracy on the original data sample. It isExpand
...
1
2
3
4
5
...