When effect sizes disagree: the case of r and d.

@article{McGrath2006WhenES,
  title={When effect sizes disagree: the case of r and d.},
  author={Robert E. McGrath and Gregory J. Meyer},
  journal={Psychological methods},
  year={2006},
  volume={11 4},
  pages={
          386-401
        }
}
The increased use of effect sizes in single studies and meta-analyses raises new questions about statistical inference. Choice of an effect-size index can have a substantial impact on the interpretation of findings. The authors demonstrate the issue by focusing on two popular effect-size measures, the correlation coefficient and the standardized mean difference (e.g., Cohen's d or Hedges's g), both of which can be used when one variable is dichotomous and the other is quantitative. Although the… 

Figures and Tables from this paper

The influence of base rates on correlations: An evaluation of proposed alternative effect sizes with real-world data
TLDR
A large sample of real-world data was used to illustrate the base rate dependence of correlations when applied to dichotomous or ordinal data to recommend AUCs, Pearson/Thorndike adjusted correlations, Cohen’s d, or polychoric correlations should be considered as alternate effect size statistics in many contexts.
Beyond Cohen's d: Alternative Effect Size Measures for Between-Subject Designs
Given the long history of discussion of issues surrounding statistical testing and effect size indices and various attempts by the American Psychological Association and by the American Educational
Cohen's d
Cohen's (1988) d is one of several statistics used to examine differences between means, as in the case of the difference in mean outcome between participants who received treatment X and
On effect size.
TLDR
A definition of effect size is proposed, which is purposely more inclusive than the way many have defined and conceptualized effect size, and it is unique with regard to linking effect size to a question of interest.
A probability-based measure of effect size: robustness to base rates and other factors.
  • J. Ruscio
  • Psychology
    Psychological methods
  • 2008
TLDR
The probability-based measure A, the nonparametric generalization of what K. O. McGraw and S. Wong (1992) called the common language effect size statistic, is insensitive to base rates and more robust to several other factors (e.g., extreme scores, nonlinear transformations).
Guidelines for Using the "Q" Test in Meta-Analysis.
The Q test is regularly used in meta-analysis to examine variation in effect sizes. However, the assumptions of Q are unlikely to be satisfied in practice prompting methodological researchers to
Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs
TLDR
A practical primer on how to calculate and report effect sizes for t-tests and ANOVA's such that effect sizes can be used in a-priori power analyses and meta-analyses and a detailed overview of the similarities and differences between within- and between-subjects designs is provided.
Standardized Effect Sizes for Moderated Conditional Fixed Effects with Continuous Moderator Variables
TLDR
This article presents a descriptive approach to investigate two-way statistical interactions involving continuous moderator variables where the conditional effects underlying these interactions are expressed in standardized effect size metrics (i.e., standardized mean differences and semi-partial correlations).
Standardized or simple effect size: what should be reported?
  • T. Baguley
  • Psychology
    British journal of psychology
  • 2009
TLDR
Factors such as reliability, range restriction and differences in design that distort standardized effect size unless suitable corrections are employed are explored.
Moderator effects differ on alternative effect-size measures
TLDR
It is shown that, under commonly-occurring conditions, popular alternatives for effect-size measures in ANOVA and multiple regression are not moderated identically across independent samples.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 73 REFERENCES
Contrasts and Correlations in Effect-Size Estimation
This article describes procedures for presenting standardized measures of effect size when contrasts are used to ask focused questions of data. The simplest contrasts consist of comparisons of two
Using odds ratios as effect sizes for meta-analysis of dichotomous data: A primer on methods and issues.
Many meta-analysts incorrectly use correlations or standardized mean difference statistics to compute effect sizes on dichotomous data. Odds ratios and their logarithms should almost always be
Effect-size indices for dichotomized outcomes in meta-analysis.
TLDR
The performance in terms of bias and sampling variance of 7 different effect-size indices for estimating the population standardized mean difference from a 2 x 2 table is examined by Monte Carlo simulation, assuming normal and nonnormal distributions.
A Simple, General Purpose Display of Magnitude of Experimental Effect
We introduce the binomial effect size display (BESD), which is useful because it is (a) easily understood by researchers, students, and lay persons; (b) widely applicable; and (c) conveniently
Coefficient Kappa: Some Uses, Misuses, and Alternatives
This paper considers some appropriate and inappropriate uses of coefficient kappa and alternative kappa-like statistics. Discussion is restricted to the descriptive characteristics of these
Biases of success rate differences shown in binomial effect size displays.
  • L. Hsu
  • Psychology
    Psychological methods
  • 2004
TLDR
Differences in the sizes of biases linked to different correlations suggest that BESD SRDs reported for different correlations are not comparable.
Characteristics of Measuresfor 2 × 2 Tables
This article compares 6 measures for the analysis of 2 ×2 tables: Pearson's X2, the correlation ρ, the standard normal z, the log-odds ratio θ, the log-linear interaction λ, and Goodman's weighted
Statistical Methods for Meta-Analysis
Preface. Introduction. Data Sets. Tests of Statistical Significance of Combined Results. Vote-Counting Methods. Estimation of a Single Effect Size: Parametric and Nonparametric Methods. Parametric
On the Validity of Measures of Association: The Nominal-Nominal, Two-by-Two Case
  • A. Hunter
  • Psychology
    American Journal of Sociology
  • 1973
Despite recent in the area, the problem of the selection of appropriate measures of association continues to worsen. One reason for this could be that we have thus far failed to consider some of the
The relationship of validity coefficients to the practical effectiveness of tests in selection: discussion and tables.
IT has often been pointed out that the magnitude of the correlation coefficient as such is not an adequate representation of the magnitude of the relationship between the two variables which are
...
1
2
3
4
5
...