Heterogeneity of Variance in Experimental Studies: A Challenge to Conventional Interpretations

  title={Heterogeneity of Variance in Experimental Studies: A Challenge to Conventional Interpretations},
  author={Anthony Bryk and Stephen Raudenbush},
  journal={Psychological Bulletin},
The presence of heterogeneity of variance across groups indicates that the standard statistical model for treatment effects no longer applies. Specifically, the assumption that treatments add a constant to each subject's development fails. An alternative model is required to represent how treatment effects are distributed across individuals. We develop in this article a simple statistical model to demonstrate the link between heterogeneity of variance and random treatment effects. Next, we… 

Figures and Tables from this paper

Testing Homogeneity of Variance
This paper compares empirical type I error and power of different tests that have been proposed to assess the homogeneity of within-group variances. The main questions answered in the paper are: (1)
Comparing the Performance of Approaches for Testing the Homogeneity of Variance Assumption in One-Factor ANOVA Models
This simulation study evaluated the performance of 14 tests for the homogeneity of variance assumption in one-way ANOVA models in terms of Type I error control and statistical power, finding the Ramsey conditional, O’Brien, Brown–Forsythe, Bootstrap Brown-ForsYthe, and Levene with squared deviations tests maintained adequate Type Ierror control.
Hierarchical Models for Causal Effects 1
Hierarchical models play three important roles in modeling causal effects: (i) accounting for data collection, such as in stratified and split-plot experimental designs; (ii) adjusting for unmeasured
Why Psychologists Should by Default Use Welch's t-test Instead of Student's t-test with Unequal Group Sizes
When comparing two independent groups, psychology researchers commonly use Student’s t -tests. Assumptions of normality and homogeneity of variance underlie this test. More often than not, when these
On some test statistics for testing homogeneity of variances: a comparative study
In this paper, we have reviewed 25 test procedures that are widely reported in the literature for testing the hypothesis of homogeneity of variances under various experimental conditions. Since a
Using multiple group modeling to test moderators in meta‐analysis
This paper details how to use one such technique, multiple group analysis, to test categorical moderators in meta- analysis, and issues related to multiple group meta-analysis and future directions for research are discussed.
How Much Do the Effects of Education and Training Programs Vary Across Sites? Evidence From Past Multisite Randomized Trials
ABSTRACT Multisite trials, in which individuals are randomly assigned to alternative treatment arms within sites, offer an excellent opportunity to estimate the cross-site average effect of treatment
Automatic Bayes Factors for Testing Equality- and Inequality-Constrained Hypotheses on Variances
The adjusted fractional Bayes factor is generally recommendable for testing equality- and inequality-constrained hypotheses on variances and is evaluated by investigating different properties such as information consistency and large sample consistency.
Equivalence of Population Variances: Synchronizing the Objective and Analysis
ABSTRACT Researchers are often interested in establishing equivalence of population variances. Traditional difference-based procedures are appropriate to answer questions about differences in some
On the significance of statistically insignificant results in consumer behavior experiments
Experimentation is the sine qua non of consumer behavior research, and much of what is thought to be known about the behavior of consumers is based on findings from experiments. However, many


Examining Correlates of Diversity
Statistical methods are presented for studying “correlates of diversity”: characteristics of educational organizations which predict dispersion on the dependent variable. The conceptual framework for
Estimating Change in Dispersion
Statistical methods are presented for studying changes in the dispersion of a dependent measure over time, for estimating the effects of treatments on the rate of dispersion change, and for combining
Statistical Methods for Meta-Analysis
Preface. Introduction. Data Sets. Tests of Statistical Significance of Combined Results. Vote-Counting Methods. Estimation of a Single Effect Size: Parametric and Nonparametric Methods. Parametric
Accumulating Evidence: Procedures for Resolving Contradictions among Different Research Studies.
Significant knowledge in the social sciences accrues ever too slowly. A major reason is that various research studies on a particular question tend to be of dissimilar designs, making their results
A General ANOVA Method for Robust Tests of Additive Models for Variances
Abstract Linearly combining Levene's z 2 variable with the jackknife pseudo-values of s 2 produces a family of variables that allows for analysis of variance (ANOVA) tests of additive models for the
Experimental Design: Procedures for the Behavioral Sciences
This chapter discusses research strategies and the Control of Nuisance Variables, as well as randomly Randomized Factorial Design with Three or More Treatments and Randomized Block Factorial design, and Confounded Factorial Designs: Designs with Group-Interaction Confounding.
Bayesian inference in statistical analysis
This chapter discusses Bayesian Assessment of Assumptions, which investigates the effect of non-Normality on Inferences about a Population Mean with Generalizations in the context of a Bayesian inference model.
Communication of Teacher Expectations and Their Effects on Achievement and Attitudes of Secondary School Students
Effects of teacher expectancies upon student behavior were investigated by giving teachers data indicating that certain average ability students were intellectually superior. Dependent measures were
Individual differences.
This book is very referred for you because it gives not only the experience but also lesson that will give wellness for all people from many societies.
The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring
T w o University of Chicago doctoral students in education, Anania (1982, 1983) and Burke (1984), completed dissertations in which they compared student learning under the following three conditions