Practical Methodological Reform Needs Good Theory

@article{Gervais2021PracticalMR,
  title={Practical Methodological Reform Needs Good Theory},
  author={Will M. Gervais},
  journal={Perspectives on Psychological Science},
  year={2021},
  volume={16},
  pages={827 - 843}
}
  • Will M. Gervais
  • Published 29 January 2021
  • Medicine
  • Perspectives on Psychological Science
In the face of unreplicable results, statistical anomalies, and outright fraud, introspection and changes in the psychological sciences have taken root. Vibrant reform and metascience movements have emerged. These are exciting developments and may point toward practical improvements in the future. Yet there is nothing so practical as good theory. This article outlines aspects of reform and metascience in psychology that are ripe for an injection of theory, including a lot of excellent and… 

Figures from this paper

Beyond Statistical Ritual: Theory in Psychological Science
  • T. Proulx, R. Morey
  • Medicine
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2021
TLDR
It is suggested that psychologists have too often chosen the latter option, substituting synthetic certainties for theory-guided research, in much the same manner as Scholastic scholars did centuries ago.
A decade of theory as reflected in Psychological Science (2009–2019)
TLDR
Research published in Psychological Science from 2009–2019 is analysed to suggest that the majority of research published in this flagship journal is not driven by theory, nor can it be contributing to cumulative theory building.
The case for formal methodology in scientific reform
TLDR
A formal statistical analysis of three popular claims in the metascientific literature is presented, showing how the use and benefits of such formalism can inform and shape debates about such methodological claims.
The case for formal methodology in scientific reform
TLDR
A formal statistical analysis of three popular claims in the metascientific literature is presented, showing how the use and benefits of such formalism can inform and shape debates about such methodological claims.
A Call for Greater Modesty in Psychology and Cognitive Neuroscience
Exaggerated claims and low levels of reproducibility are commonplace in psychology and cognitive neuroscience, due to an incentive structure that demands “newsworthy” results. My overall argument
How to True Psychology’s Objects
Psychology’s current crisis attends most visibly to perceived problems with statistical models, methods, publication practices, and career incentives. Rarely is close attention given to the objects
How Do We Choose Our Giants? Perceptions of Replicability in Psychological Science
Judgments regarding replicability are vital to scientific progress. The metaphor of “standing on the shoulders of giants” encapsulates the notion that progress is made when new discoveries build on
Replication is important for educational psychology: Recent developments and key issues
Abstract Replication is a key activity in scientific endeavors. Yet explicit replications are rare in many fields, including education and psychology. In this article, we discuss the relevance and
Beyond the “Third Wave of Positive Psychology”: Challenges and Opportunities for Future Research
The positive psychology (PP) landscape is changing, and its initial identity is being challenged. Moving beyond the “third wave of PP,” two roads for future research and practice in well-being
How to Translate a Verbal Theory Into a Formal Model
Abstract. Turning verbal theories into formal models is an essential business of a mature science. Here I elaborate on taxonomies of models, provide ten lessons for translating a verbal theory into...
...
1
2
...

References

SHOWING 1-10 OF 101 REFERENCES
A Short (Personal) Future History of Revolution 2.0
  • B. Spellman
  • Sociology, Medicine
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2015
TLDR
It is argued that the world is going through a revolution analogous to a political revolution, and psychological scientists have not yet made needed advances in the ways in which the authors accumulate, connect, and extract conclusions from their aggregated research.
The natural selection of bad science
TLDR
A 60-year meta-analysis of statistical power in the behavioural sciences is presented and it is shown that power has not improved despite repeated demonstrations of the necessity of increasing power, and that replication slows but does not stop the process of methodological deterioration.
The Natural Selection of Conservative Science
  • Cailin O'Connor
  • Computer Science, Medicine
    Studies in history and philosophy of science
  • 2019
The natural selection of good science.
TLDR
This work model the cultural evolution of research practices when laboratories are allowed to expend effort on theory, enabling them, at a cost, to identify hypotheses that are more likely to be true, before empirical testing.
When and Why to Replicate: As Easy as 1, 2, 3?
The crisis of confidence in psychology has prompted vigorous and persistent debate in the scientific community concerning the veracity of the findings of psychological experiments. This discussion
Scientific polarization
TLDR
A model is presented, based on the network epistemology framework of Bala and Goyal, in which polarization emerges even though agents gather evidence about their beliefs, and true belief yields a pay-off advantage.
How Computational Modeling Can Force Theory Building in Psychological Science
TLDR
This work argues that one of the most divisive factors in psychological science is whether researchers choose to use computational modeling of theories over and above data during the scientific-inference process, and presents scientific inference in psychology as a path function in which each step shapes the next.
The Origins of Unfairness
The central aim of this book is to explore the ways in which social categories—especially gender, but also categories like race and religion—interact with and contribute to social solutions to
Toward a psychology of Homo sapiens: Making psychological science more representative of the human population
TLDR
Almost all research published by one of the leading journals, Psychological Science, relies on Western samples and uses these data in an unreflective way to make inferences about humans in general.
Estimating the reproducibility of psychological science
TLDR
A large-scale assessment suggests that experimental reproducibility in psychology leaves a lot to be desired, and correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.
...
1
2
3
4
5
...