Facts Are More Important Than Novelty

@article{Makel2014FactsAM,
  title={Facts Are More Important Than Novelty},
  author={Matthew C. Makel and Jonathan A. Plucker},
  journal={Educational Researcher},
  year={2014},
  volume={43},
  pages={304 - 316}
}
Despite increased attention to methodological rigor in education research, the field has focused heavily on experimental design and not on the merit of replicating important results. The present study analyzed the complete publication history of the current top 100 education journals ranked by 5-year impact factor and found that only 0.13% of education articles were replications. Contrary to previous findings in medicine, but similar to psychology, the majority of education replications… 

Figures and Tables from this paper

Replication of Special Education Research
Increased calls for rigor in special education have often revolved around the use of experimental research design. However, the replicability of research results is also a central tenet to the
Replication Research and Special Education
Replicating previously reported empirical research is a necessary aspect of an evidence-based field of special education, but little formal investigation into the prevalence of replication research
Promoting conceptual replications in educational research
  • Jinfa Cai
  • Psychology
    Educational Research and Evaluation
  • 2022
Despite the abundant and frequent calls for replication studies from research communities (e.g., Shavelson & Towne, 2002) and funding agencies (e.g., Institute of Education Sciences [IES] & National
Replication in computing education research: researcher attitudes and experiences
TLDR
While respondents agree that published work should be verifiable, they doubt this standard is widely met in the computing education field and are not eager to perform the work of verifying others' work themselves.
Supporting a Culture of Replication: An Examination of Education and Special Education Research Grants Funded by the Institute of Education Sciences
Despite the important role that replication studies play in building scientific evidence, recent reports show that few replications have been conducted in education. The goal of the current study was
Replication is important for educational psychology: Recent developments and key issues
Abstract Replication is a key activity in scientific endeavors. Yet explicit replications are rare in many fields, including education and psychology. In this article, we discuss the relevance and
Are Psychology Journals Anti-replication? A Snapshot of Editorial Practices
TLDR
The instructions to authors and the published aims of 1151 psychology journals were reviewed and whether they indicated that replications were permitted and accepted and whether journal practices differed across branches of the discipline, and whether editorial practices differed between low and high impact journals.
Recommendations for Replication Research in Special Education
Special education researchers conduct studies that can be considered replications. However, they do not often refer to them as replication studies. The purpose of this article is to consider the
An Introduction to Replication Research in Gifted Education
This methodological brief introduces readers to replication methods and their uses. Broadly defined, replication is the duplication of previously conducted research to verify or expand the original
Emergent themes from recent research syntheses in science education and their implications for research design, replication, and reporting practices
This article draws upon the experiences of four recent efforts to synthesize the findings of quantitative studies in science education research. After establishing the need for research syntheses in
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 151 REFERENCES
Teaching Replication
  • Michael C. Frank, R. Saxe
  • Psychology
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2012
TLDR
It is argued that students in laboratory classes should replicate recent findings as part of their training in experimental methods, and it provides object lessons about the scientific process, the importance of reporting standards, and the value of openness.
Welcoming Quality in Non-Significance and Replication Work, but Moving Beyond the p-Value
The self-correcting nature of psychological and educational science has been seriously questioned. Recent special issues of Perspectives on Psychological Science and Psychology of Aesthetics,
Contradicted and initially stronger effects in highly cited clinical research.
TLDR
Contradiction and initially stronger effects are not unusual in highly cited research of clinical interventions and their outcomes, but the extent to which high citations may provoke contradictions and vice versa needs more study.
Peer-review practices of psychological journals: The fate of published articles, submitted again
Abstract A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines.
Why Selective Publication of Statistically Significant Results Can Be Effective
TLDR
It is shown that selectively publishing effects that differ significantly from the cumulative meta-analytic effect evokes the Proteus phenomenon of poorly replicable and alternating findings, and that the selective publication approach yields a scientific record that is content rich as compared to publishing everything.
The ironic effect of significant results on the credibility of multiple-study articles.
TLDR
One major recommendation is to pay more attention to the power of studies to produce positive results without the help of questionable research practices and to request that authors justify sample sizes with a priori predictions of effect sizes.
An Example of Replication of an Experiment for Increased Reliability
In this article findings will be presented from a set of massed ex periments involving seventeen pairs of matched groups, 104 measured group contrasts, all centered on the same hypothesis.1 The
Why Isn't Creativity More Important to Educational Psychologists? Potentials, Pitfalls, and Future Directions in Creativity Research
The construct of creativity has a great deal to offer educational psychology. Creativity appears to be an important component of problem-solving and other cognitive abilities, healthy social and
Harnessing the Undiscovered Resource of Student Research Projects
  • Jon E. Grahe, A. Reifman, R. Wiebe
  • Psychology
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2012
TLDR
It is suggested that undergraduate research can help advance the science of psychology and a hypothetical “question-list paradigm” is introduced as a mechanism to do this.
...
1
2
3
4
5
...