The seven sins of L2 research: A review of 30 journals’ statistical quality and their CiteScore, SJR, SNIP, JCR Impact Factors

@article{AlHoorie2019TheSS,
  title={The seven sins of L2 research: A review of 30 journals’ statistical quality and their CiteScore, SJR, SNIP, JCR Impact Factors},
  author={Ali H. Al-Hoorie and Joseph P. Vitta},
  journal={Language Teaching Research},
  year={2019},
  volume={23},
  pages={727 - 744}
}
This report presents a review of the statistical practices of 30 journals representative of the second language field. A review of 150 articles showed a number of prevalent statistical violations including incomplete reporting of reliability, validity, non-significant results, effect sizes, and assumption checks as well as making inferences from descriptive statistics and failing to correct for multiple comparisons. Scopus citation analysis metrics and whether a journal is SSCI-indexed were… 
View on SAGE
49 Citations
Highly Influential Citations
2
Background Citations
8
Methods Citations
4
Results Citations
1

Tables from this paper

49 Citations

CiteScore: Advances, Evolution, Applications, and Limitations
Elsevier’s (Scopus) CiteScore, a journal-based metric (JBM), has been assigned to more journals than Clarivate Analytics’ Journal Impact Factor (JIF), including journals indexed by Scopus that do not…
CiteScore: Advances, Evolution, Applications, and Limitations
TLDR
While use of the CiteScore can provide interesting information about how a journal is being cited, the use of this JBM as a “quality” metric is cautioned.
Analysis of Percentiles of Computer Science, Theory and Methods Journals CiteScore Versus Impact Factor
TLDR
Caution is recommended in journal evaluation as conflicting different results can be obtained using the same metric and the median test contradicts the ANOVA as the medians of impact factor and CiteScore are different at 0.05 level of significance.
Citing Retracted Papers Affects Education and Librarianship, so Distorted Academic Metrics Need a Correction
TLDR
A prototype concept of the corrected journal impact factor (cJIF) is introduced to correct the most widespread journal-based metric, the Clarivate Analytics journal impact factors (JIF), as a possible solution to correct journal- based metrics that may have become distorted by the citation of retracted papers.
Bibliometric Measures and Journal Characteristics of Leading Accounting and Finance Journals
  • T. Krueger
  • Economics
    Journal of Accounting and Finance
  • 2019
This study analyzes journal quality metrics and characteristics of leading accounting and finance journals in the Academic Journal Guide. The quality metrics analyzed are Journal Citation Reports…
National variation in citation-based business journal quality measures and characteristics
Abstract Research quality is often based on the perceived quality of journals in which the research appears. Journal quality is subjectively based on a wide array of objective journal…
Statistical assumptions in L2 research: A systematic review
TLDR
Overall weak reporting of checks on assumptions in two major journals of second-language (L2) research over a span of six years is examined as well as implications for researcher training.
A BIBLIOMETRIC ANALYSIS OF SECOND LANGUAGE ACQUISITION BETWEEN 1997 AND 2018
  • Xian Zhang
  • Linguistics
    Studies in Second Language Acquisition
  • 2020
TLDR
Three types of bibliometric analyses were performed to identify the prominent scholarly documents, authors, research institutions, geographic regions, and research topics that have been highly influential in the field of SLA over the last two decades.
Study Quality in Quantitative L2 Research: A Path Analysis on the Perceptions of Iranian Published Authors
Inspired by the works of scholars who have done great efforts to improve study quality in second language (L2) papers in the recent decade (e.g., Hu & Plonsky, 2019; Larson-Hall, 2012, 2017; Plonsky,…
A methodological synthesis and meta-analysis of judgment tasks in second language research
Judgment tasks (JTs, often called acceptability or grammaticality judgment tasks) are found extensively throughout the history of second language (L2) research. Data from such instruments have been…
...
...

References

SHOWING 1-10 OF 60 REFERENCES
SJR and SNIP: two new journal metrics in Elsevier's Scopus
TLDR
Two journal metrics recently endorsed by Elsevier's Scopus are introduced: SCImago Journal Rank (SJR) and Source Normalized Impact per Paper (SNIP).
Scopus's Source Normalized Impact per Paper (SNIP) versus a Journal Impact Factor based on Fractional Counting of Citations
TLDR
Fractional counting of citations can be contextualized at the paper level and aggregated impacts of sets can be tested for their significance, and it can be shown that the weighted impact of Annals of Mathematics is not so much lower than that of Molecular Cell despite a five-f old difference.
A further step forward in measuring journals' scientific prestige: The SJR2 indicator
Inferential statistics in Language Teaching Research: A review and ways forward
This article reviews all (quasi)experimental studies appearing in the first 19 volumes (1997–2015) of Language Teaching Research (LTR). Specifically, it provides an overview of how statistical…
Scopus: A system for the evaluation of scientific journals
TLDR
The SJR (SCImago Journal Rank) as the main criterion used by Scopus to evaluate scientific journals is considered and the Scopus and ISI systems are compared using information on the journal Prikladnaya Mekhanika (International Applied Mechanics).
STUDY QUALITY IN SLA
  • Luke Plonsky
  • Linguistics
    Studies in Second Language Acquisition
  • 2013
This study assesses research and reporting practices in quantitative second language (L2) research. A sample of 606 primary studies, published from 1990 to 2010 in Language Learning and Studies in…
Quality Analysis of Journals in TESOL and Applied Linguistics
Journals act as one of the primary storage areas for and distributors of academic knowledge across fields, including TESOL and applied linguistics (AL). The quality of such journals often determines…
The Academic Journal: Has it a Future?
This article examines the current state of the academic journal. It does so for a number of reasons: the increasing expense of paper journals; the advent of electronic publishing; the use of…
Study quality in quantitative l2 research (1990-2010): A methodological synthesis and call for reform
This article builds on the growing line of inquiry into methodological practices in quantitative second language (L2) research. Specifically, the study uses synthetic techniques to examine changes…
Quantitative Research Methods, Study Quality, and Outcomes: The Case of Interaction Research
This article constitutes the first empirical assessment of methodological quality in second language acquisition (SLA). We surveyed a corpus of 174 studies (N = 7,951) within the tradition of…
...
...