High Appraisal of Methodological Quality of Basic Science Articles Published in Stroke.

Abstract

See related article, p XXX Translational failure from bench to bedside in medical research may be due, at least in part, to the suboptimal quality of experimental studies. For example, the factors such as lack of blinding and randomization in experimental studies may lead to an overestimation of treatment effects, contributes to irreproducibility of experimental data, and subsequently blocks translation into the clinic. To overcome this dilemma, the Animal Research Reporting In vivo Experiments guidelines provided recommendations for design and reporting of animal experimental studies. Stroke was a pioneer in the effort to improve the quality of preclinical research by establishing a requirement for the reporting of methodological quality of animal studies by implementation of the Basic Science Checklist, a prerequisite for experimental research later delineated by the landmark article of Landis et al. In a recent article, Ramirez et al systematically examined the methodology of preclinical studies published in 5 major cardiovascular journals published by the American Heart Association: Circulation, Circulation Research, Hypertension, Stroke, and Arteriosclerosis, Thrombosis, and Vascular Biology, between July 2006 and June 2016. The authors sought to determine how often studies met basic quality indicators, such as randomization, blinding, and a priori sample size estimation/power calculation, and so forth. Studies reporting in vivo experiments in nonhuman mammals describing pathophysiology, genetics, and therapeutic interventions relevant to specific cardiovascular disorders were considered. Twenty-eight thousand six hundred and thirty-six articles were screened; 3396 met inclusion criteria. Overall, the results showed disappointingly poor study quality of articles published in these journals: overall randomization was reported in only 21.8%, blinding procedures in just 32.7%, and sample size estimation/power calculation was de facto absent (reported in only 2.3%). These numbers remained largely unchanged over the whole observation period of 10 years, except for basic science stroke studies. These studies, 90% of which were published in Stroke, show an encouraging and substantial increase in study quality indicators compared with the 4 journals. In the 2015/2016 examination period, randomization in preclinical stroke studies reached 65% versus ≤23% compared with atherosclerosis, hypertension, cardiomyopathy/heart failure; blinding surpassed 80% versus <30%, and sample size estimation/power calculation reached 23% versus <5%, respectively. These positive findings support an earlier analysis made from basic science articles published in Stroke before and after the implementation of the Basic Science Checklist (first introduced in 2011, updated in 2016). The use of the checklist tool led to an increased reporting of inclusion and exclusion criteria definition, allocation concealment, and blinding and showed an increase of studies with the highest quality category of the checklist item sum score compared with the period before where no checklist existed. An interesting question is certainly whether these improvements were because of the journal editorial policy or may even reflect a change in research practice in the basic science stroke community. A post hoc analysis by Ramirez et al came to the conclusion that it is likely a combination of both: Stroke uniquely showed improvements in all study design elements even after adjusting for cardiovascular disorder studied and animal model used, but stroke as the cardiovascular disorder studied was identified as an independent predictor for at least one study design element for every journal examined. Interestingly, but not unexpected, the Ramirez et al report of poor preclinical study quality not to impact on citations at 60 months. Article citation is a most commonly used measure of research impact but is obviously an imperfect indicator of study quality. Despite the superior quality of basic science articles published in Stroke, its impact factor ranges below that of the cardiovascular journals, Circulation or Circulation Research. However, Stroke articles belong to the most accessed ones with >11 million downloads in 2016 and an higher article influence score, measuring the high relative importance of the journal on a per-article basis, compared with other neurological and cardiovascular journals. Beside novelty and innovation, one of the most important issues in real life and in particular in research is quality. Obviously, Stroke basic science authors are on the right path, but further quality improvement of preclinical articles is needed. Well performed experimental studies will impact the field, alleviate the translational roadblock, and finally contribute to a better treatment of stroke patients. We anticipate that the recently updated preclinical checklist will lead to further improvements in the quality of preclinical research published by Stroke over the next few years. High Appraisal of Methodological Quality of Basic Science Articles Published in Stroke

DOI: 10.1161/STROKEAHA.117.017717

Cite this paper

@article{Schbitz2017HighAO, title={High Appraisal of Methodological Quality of Basic Science Articles Published in Stroke.}, author={Wolf - R{\"{u}diger Sch{\"a}bitz and Jens Minnerup and Marc O Fisher and Jaroslaw Aronowski}, journal={Stroke}, year={2017}, volume={48 9}, pages={2337-2338} }