Editors’ Introduction to the Special Section on Replicability in Psychological Science

@article{Pashler2012EditorsIT,
  title={Editors’ Introduction to the Special Section on Replicability in Psychological Science},
  author={Harold Pashler and Eric-Jan Wagenmakers},
  journal={Perspectives on Psychological Science},
  year={2012},
  volume={7},
  pages={528 - 530}
}
Is there currently a crisis of confidence in psychological science reflecting an unprecedented level of doubt among practitioners about the reliability of research findings in the field? It would certainly appear that there is. These doubts emerged and grew as a series of unhappy events unfolded in 2011: the Diederik Stapel fraud case (see Stroebe, Postmes, & Spears, 2012, this issue), the publication in a major social psychology journal of an article purporting to show evidence of extrasensory… 

Psychology's Renaissance

It is shown that the scientific practices of experimental psychologists have improved dramatically and is argued that meta‐analytical thinking increases the prevalence of false positives.

Less Is More: Psychologists Can Learn More by Studying Fewer People

The reliance on group comparisons is arguably the most fundamental problem at hand because such designs are what often necessitate the kinds of statistical analyses that have led to psychology's professional crisis.

Replicability Crisis in Social Psychology: Looking at the Past to Find New Pathways for the Future

Over the last few years, psychology researchers have become increasingly preoccupied with the question of whether findings from psychological studies are generally replicable. The debates have

Psychological science from 'publish or perish' to 'trust but verify'

At the time when social psychologists believed they could be proud of their discipline, there was the devastating news that Diederik Stapel had committed a major scientific fraud. This event

Using replication projects in teaching research methods

The replication crisis today Rosenthal’s (1979) seminal paper first pointed out the ‘file drawer problem’: journals print papers that are often based on Type 1 errors while the file drawers of

Negativland - a home for all findings in psychology

  • K. Laws
  • Psychology
    BMC psychology
  • 2013
A serious challenge for psychologists to get their house in-order is to make sure that replications and null findings find a home in psychology rather than remain the authors' dirty little secrets that further eat away at the credibility of their science.

The psychology of experimental psychologists: Overcoming cognitive constraints to improve research: The 47th Sir Frederic Bartlett Lecture

  • D. Bishop
  • Psychology
    Quarterly journal of experimental psychology
  • 2019
It is argued that if proposed solutions to the replication crisis are to be effective, they need to take into account human cognitive constraints that can distort all stages of the research process, including design and execution of experiments, analysis of data, and writing up findings for publication.

The Crisis of Confidence in Research Findings in Psychology: Is Lack of Replication the Real Problem? Or Is It Something Else?

There have been frequent expressions of concern over the supposed failure of researchers to conduct replication studies. But the large number of meta-analyses in our literatures shows that

Burning the straw man: What exactly is psychological science?

Problemification: Efendic and Van Zyl (2019) argue for following open access-based principles in IO psychology following the recent crises in psychological research. Among others, these refer to the

Estimating the evidential value of significant results in psychological science

It is concluded that due to the threshold of acceptance having been set too low for psychological findings, a substantial proportion of the published results have weak evidential support.
...

References

SHOWING 1-10 OF 28 REFERENCES

Is the Replicability Crisis Overblown? Three Arguments Examined

  • H. PashlerC. Harris
  • Psychology
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2012
It is argued that there are no plausible concrete scenarios to back up such forecasts and that what is needed is not patience, but rather systematic reforms in scientific practice.

Why Science Is Not Necessarily Self-Correcting

  • J. Ioannidis
  • Psychology
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2012
A number of impediments to self-correction that have been empirically studied in psychological science are cataloged and some proposed solutions to promote sound replication practices enhancing the credibility of scientific results are discussed.

Science or Art? How Aesthetic Standards Grease the Way Through the Publication Bottleneck but Undermine Science

  • R. Giner-Sorolla
  • Psychology
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2012
To open the bottleneck, putting structures in place to reward broader forms of information sharing beyond the exquisite art of present-day journal publication is suggested, suggesting a more palatable solution to the crisis in psychological research.

The poor availability of psychological research data for reanalysis.

A failed attempt to obtain data reported in 141 empirical articles recently published by the APA, to assess the robustness of the research findings to outliers.

Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results

It is suggested that statistical results are particularly hard to verify when reanalysis is more likely to lead to contrasting conclusions, which highlights the importance of establishing mandatory data archiving policies.

Why Most Published Research Findings Are False

Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true.

An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science

  • Brian A. NosekD. Lakens
  • Psychology
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2012
The Reproducibility Project is an open, large-scale, collaborative effort to systematically examine the rate and predictors of reproducibility in psychological science.

Feeling the future: experimental evidence for anomalous retroactive influences on cognition and affect.

  • D. Bem
  • Psychology
    Journal of personality and social psychology
  • 2011
9 experiments are reported, involving more than 1,000 participants, that test for retroactive influence by "time-reversing" well-established psychological effects so that the individual's responses are obtained before the putatively causal stimulus events occur.

A peculiar prevalence of p values just below .05

In null hypothesis significance testing (NHST), p values are judged relative to an arbitrary threshold for significance (.05). The present work examined whether that standard influences the

Data from Paper “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant”

The data includes measures collected for the two experiments reported in “False-Positive Psychology” [1] where listening to a randomly assigned song made people feel younger (Study 1) or actually be