The case for formal methodology in scientific reform

@article{Devezer2020TheCF,
  title={The case for formal methodology in scientific reform},
  author={Berna Devezer and Daniel J. Navarro and Joachim Vandekerckhove and Erkan Ozge Buzbas},
  journal={bioRxiv},
  year={2020}
}
Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific practices in experimental sciences. However, most methodological reform attempts suffer from similar mistakes and over-generalizations to the ones they aim to address. We argue that this can be attributed in part to lack of formalism and first principles. Considering the costs of allowing false claims to become canonized, we argue for formal statistical rigor and… Expand
View via Publisher
royalsocietypublishing.org
26 Citations
Highly Influential Citations
4
Background Citations
7
Methods Citations
2

26 Citations

Citation Type

Citation Type

When numbers fail: do researchers agree on operationalization of published research?
Current discussions on improving the reproducibility of science often revolve around statistical innovations. However, equally important for improving methodological rigour is a valid… Expand
Self-correction in science: The diagnostic and integrative motives for replication
TLDR
Six theses are offered that aim to put science and technology studies and science activism into dialog to show why effective reforms will need to confront issues of disciplinary difference. Expand
Does preregistration improve the credibility of research findings?
Preregistration entails researchers registering their planned research hypotheses, methods, and analyses in a time-stamped document before they undertake their data collection and analyses. This… Expand
Beyond Statistical Ritual: Theory in Psychological Science
  • T. Proulx, R. Morey
  • Medicine
  • Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2021
TLDR
It is suggested that psychologists have too often chosen the latter option, substituting synthetic certainties for theory-guided research, in much the same manner as Scholastic scholars did centuries ago. Expand
How Do We Choose Our Giants? Perceptions of Replicability in Psychological Science
Judgments regarding replicability are vital to scientific progress. The metaphor of “standing on the shoulders of giants” encapsulates the notion that progress is made when new discoveries build on… Expand
Science as collaborative knowledge generation
  • N. Ellemers
  • Psychology
  • The British journal of social psychology
  • 2021
The COVID‐19 pandemic points to the need for scientists to pool their efforts in order to understand this disease and respond to the ensuing crisis. Other global challenges also require such… Expand
Ethical conceptual replication of visualization research considering sources of methodological bias and practical significance
TLDR
It is argued that conceptual replication should emphasize conceptual replication focused on determining practical significance and reducing methodological biases to determine how design principles interact with methodological approaches, laying the groundwork for visualization meta-science. Expand
Making ERP research more transparent: Guidelines for preregistration.
TLDR
An overview of the problems associated with undisclosed analytic flexibility is presented, why and how EEG researchers would benefit from adopting preregistration are discussed, and guidelines and examples on how to preregister data preprocessing and analysis steps in typical ERP studies are provided. Expand
Theory Before the Test: How to Build High-Verisimilitude Explanatory Theories in Psychological Science
TLDR
It is suggested that psychological science, by focusing on effects, may lose sight of its primary explananda: psychological capacities, and how to leverage effects to understand capacities better is explained. Expand
Does Sociology Need Open Science?
Reliability, transparency, and ethical crises pushed many social science disciplines toward dramatic changes, in particular psychology and more recently political science. This paper discusses why… Expand
...
1
2
3
...

References

SHOWING 1-10 OF 183 REFERENCES
Rethinking Reproducibility as a Criterion for Research Quality
  • S. Leonelli
  • Psychology
  • Including a Symposium on Mary Morgan: Curiosity, Imagination, and Surprise
  • 2018
A heated debate surrounds the significance of reproducibility as an indicator for research quality and reliability, with many commentators linking a "crisis of reproducibility" to the rise of… Expand
Exploratory Factor Analysis, Theory Generation, and Scientific Method
  • B. Haig
  • Psychology, Medicine
  • Multivariate behavioral research
  • 2005
TLDR
It is argued that EFA helps researchers generate theories with genuine explanatory merit and can be profitably employed in tandem with confirmatory factor analysis and other methods of theory evaluation. Expand
A problem in theory
TLDR
Muthukrishna & Henrich argue that solving the replication crisis in psychology partly requires well-specified, overarching theoretical frameworks and outline how dual inheritance theory provides one such example that could be adopted by the field. Expand
Understanding Replication in a Way That is True to Science
TLDR
It is maintained that viewing replication as just one element of the wide array of scientific endeavors leads to the conclusion that it is not as prominent in science as is often claimed. Expand
Metascience as a scientific social movement
Emerging out of the “reproducibility crisis” in science, metascientists have become central players in debates about research integrity, scholarly communication, and science policy. The goal of this… Expand
Making replication mainstream
TLDR
There are no theoretical or statistical obstacles to making direct replication a routine aspect of psychological science, and the need for an integrative summary of replication studies is addressed. Expand
A practical guide for transparency in psychological science
The credibility of scientific claims depends upon the transparency of the research products upon which they are based (e.g., study protocols, data, materials, and analysis scripts). As psychology… Expand
Publication bias and the canonization of false facts
TLDR
It is found that unless a sufficient fraction of negative results are published, false claims frequently can become canonized as fact and true and false claims would be more readily distinguished. Expand
Practical Methodological Reform Needs Good Theory
  • Will M. Gervais
  • Medicine
  • Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2021
In the face of unreplicable results, statistical anomalies, and outright fraud, introspection and changes in the psychological sciences have taken root. Vibrant reform and metascience movements have… Expand
An Agenda for Purely Confirmatory Research
TLDR
This article proposes that researchers preregister their studies and indicate in advance the analyses they intend to conduct, and proposes that only these analyses deserve the label “confirmatory,” and only for these analyses are the common statistical tests valid. Expand
...
1
2
3
4
5
...