Evaluating the Effectiveness of Developmental Mathematics by Embedding a Randomized Experiment Within a Regression Discontinuity Design

@article{Moss2014EvaluatingTE,
  title={Evaluating the Effectiveness of Developmental Mathematics by Embedding a Randomized Experiment Within a Regression Discontinuity Design},
  author={Brian G. Moss and William H. Yeaton and Jane E. LIoyd},
  journal={Educational Evaluation and Policy Analysis},
  year={2014},
  volume={36},
  pages={170 - 185}
}
Using a novel design approach, a randomized experiment (RE) was embedded within a regression discontinuity (RD) design (R-RE-D) to evaluate the impact of developmental mathematics at a large midwestern college (n = 2,122). Within a region of uncertainty near the cut-score, estimates of benefit from a prospective RE were closely comparable with those found using an RD design, based on retrospective data. Parametric and nonparametric analyses were conducted, yielding increases that generally… Expand

Figures and Tables from this paper

Evaluating Effects of Developmental Education for College Students Using a Regression Discontinuity Design
TLDR
The cumulative, within-study comparison, replication approach offers considerable potential for the evaluation of new and existing policies, particularly when effects are relatively small, as is often the case in applied settings. Expand
A Multiple-Design, Experimental Strategy: Academic Probation Warning Letter's Impact on Student Achievement
Abstract We aimed to compare the findings of three research designs to bracket effect estimates of a strongly worded warning letter delivered by certified mail to students on academic probation. WeExpand
THE INTERNAL AND EXTERNAL VALIDITY OF THE REGRESSION DISCONTINUITY DESIGN: A META‐ANALYSIS OF 15 WITHIN‐STUDY COMPARISONS
Theory predicts that regression discontinuity (RD) provides valid causal inference at the cutoff score that determines treatment assignment. One purpose of this paper is to test RD's internalExpand
THE INTERNAL AND EXTERNAL VALIDITY OF THE REGRESSION DISCONTINUITY DESIGN: A METAANALYSIS OF 15 WITHINSTUDY COMPARISONS
Theory predicts that regression discontinuity (RD) provides valid causal inference at the cutoff score that determines treatment assignment. One purpose of this paper is to test RD’s internalExpand
RD or Not RD: Using Experimental Studies to Assess the Performance of the Regression Discontinuity Approach
TLDR
Examining the performance of the RD estimator with the design is well implemented and the extent of bias introduced by manipulation of the assignment variable in an RD design finds impact estimates that are not significantly different from one another and have a similar magnitude, on average. Expand
Using a Regression Discontinuity Design to Estimate the Impact of Placement Decisions in Developmental Math
This study evaluates the effectiveness of math placement policies for entering community college students on these students’ academic success in math. We estimate the impact of placement decisions byExpand
What Can Be Learned From Empirical Evaluations of Nonexperimental Methods?
TLDR
This essay provides an overview of recent efforts to empirically evaluate NE method performance in field settings and describes papers that are included in this two-volume special issue on WSC approaches and suggests future areas for consideration in the design, implementation, and analysis of WSCs. Expand
What Happens to Students Placed Into Developmental Education? A Meta-Analysis of Regression Discontinuity Studies
This article reports a systematic review and meta-analysis of studies that use regression discontinuity to examine the effects of placement into developmental education. Results suggest thatExpand
Alternatives to Randomized Control Trials: A Review of Three Quasi-experimental Designs for Causal Inference
Abstract. The Randomized Control Trial (RCT) design is typically seen as the gold standard in psychological research. As it is not always possible to conform to RCT specifications, many studies areExpand
Limitless Regression Discontinuity
Conventionally, regression discontinuity analysis contrasts a univariate regression’s limits as its independent variable, R, approaches a cut point, c, from either side. Alternative methods targetExpand
...
1
2
3
4
...

References

SHOWING 1-10 OF 43 REFERENCES
Shaping Policies Related to Developmental Education: An Evaluation Using the Regression-Discontinuity Design
Utilizing the regression-discontinuity research design, this article explores the effectiveness of a developmental English program in a large, multicampus community college. Routinely collected dataExpand
Statistical Power for Regression Discontinuity Designs in Education Evaluations
This article examines theoretical and empirical issues related to the statistical power of impact estimates under clustered regression discontinuity (RD) designs. The theory is grounded in the causalExpand
Comparison of a Randomized and Two Quasi-Experimental Designs in a Single Outcome Evaluation
TLDR
The authors assessed the impact of three designs (randomized experiment, nonequivalent control group design, regression discontinuity design) on estimates of effect size of a university-level freshman remedial writing program, which led to highly similar effect size estimates. Expand
A randomized experiment comparing random and cutoff-based assignment.
TLDR
This study randomly assigns 588 participants to be in a randomized experiment or a regression discontinuity design in which they are otherwise treated identically, comparing results estimating both the same and different parameters. Expand
Applying the Regression-Discontinuity Design to Infer Causality with Non-Random Assignment
The impact of developmental mathematics programs on student success in college-level mathematics has been a controversial topic in higher education. One limitation of the current research is thatExpand
Do remedial English programs impact first‐year success in college? An illustration of the regression‐discontinuity design
This paper presents an illustration of how the regression‐discontinuity design can be used to obtain an unbiased estimate of the causal effect of participating in a university remedial EnglishExpand
Development, Discouragement, or Diversion? New Evidence on the Effects of College Remediation
Half of all college students take at least one remedial course as part of their postsecondary experience, despite mixed evidence on the effectiveness of this intervention. Using aExpand
Does Mathematics Remediation Work?: A Comparative Analysis of Academic Attainment among Community College Students
Postsecondary remediation is a controversial topic. On one hand, it fills an important and sizeable niche in higher education. On the other hand, critics argue that it wastes tax dollars, diminishesExpand
Modern Regression Discontinuity Analysis
Abstract This article provides a detailed discussion of the theory and practice of modern regression discontinuity (RD) analysis for estimating the effects of interventions or treatments. Part 1Expand
"Waiting for Life to Arrive": A history of the regression-discontinuity design in Psychology, Statistics and Economics
This paper reviews the history of the regression discontinuity design in three academic disciplines. It describes the design's birth and subsequent demise in Psychology even though most problems withExpand
...
1
2
3
4
5
...