Evaluating the Effectiveness of Developmental Mathematics by Embedding a Randomized Experiment Within a Regression Discontinuity Design

@article{Moss2014EvaluatingTE,
  title={Evaluating the Effectiveness of Developmental Mathematics by Embedding a Randomized Experiment Within a Regression Discontinuity Design},
  author={Brian G. Moss and William H. Yeaton and Jane E. LIoyd},
  journal={Educational Evaluation and Policy Analysis},
  year={2014},
  volume={36},
  pages={170 - 185}
}
Using a novel design approach, a randomized experiment (RE) was embedded within a regression discontinuity (RD) design (R-RE-D) to evaluate the impact of developmental mathematics at a large midwestern college (n = 2,122). Within a region of uncertainty near the cut-score, estimates of benefit from a prospective RE were closely comparable with those found using an RD design, based on retrospective data. Parametric and nonparametric analyses were conducted, yielding increases that generally… 

Figures and Tables from this paper

Evaluating Effects of Developmental Education for College Students Using a Regression Discontinuity Design

The cumulative, within-study comparison, replication approach offers considerable potential for the evaluation of new and existing policies, particularly when effects are relatively small, as is often the case in applied settings.

A Multiple-Design, Experimental Strategy: Academic Probation Warning Letter's Impact on Student Achievement

Abstract We aimed to compare the findings of three research designs to bracket effect estimates of a strongly worded warning letter delivered by certified mail to students on academic probation. We

THE INTERNAL AND EXTERNAL VALIDITY OF THE REGRESSION DISCONTINUITY DESIGN: A META‐ANALYSIS OF 15 WITHIN‐STUDY COMPARISONS

Theory predicts that regression discontinuity (RD) provides valid causal inference at the cutoff score that determines treatment assignment. One purpose of this paper is to test RD's internal

THE INTERNAL AND EXTERNAL VALIDITY OF THE REGRESSION DISCONTINUITY DESIGN: A METAANALYSIS OF 15 WITHINSTUDY COMPARISONS

Theory predicts that regression discontinuity (RD) provides valid causal inference at the cutoff score that determines treatment assignment. One purpose of this paper is to test RD’s internal

RD or Not RD: Using Experimental Studies to Assess the Performance of the Regression Discontinuity Approach

Examining the performance of the RD estimator with the design is well implemented and the extent of bias introduced by manipulation of the assignment variable in an RD design finds impact estimates that are not significantly different from one another and have a similar magnitude, on average.

Using a Regression Discontinuity Design to Estimate the Impact of Placement Decisions in Developmental Math

This study evaluates the effectiveness of math placement policies for entering community college students on these students’ academic success in math. We estimate the impact of placement decisions by

What Can Be Learned From Empirical Evaluations of Nonexperimental Methods?

This essay provides an overview of recent efforts to empirically evaluate NE method performance in field settings and describes papers that are included in this two-volume special issue on WSC approaches and suggests future areas for consideration in the design, implementation, and analysis of WSCs.

What Happens to Students Placed Into Developmental Education? A Meta-Analysis of Regression Discontinuity Studies

This article reports a systematic review and meta-analysis of studies that use regression discontinuity to examine the effects of placement into developmental education. Results suggest that

Alternatives to Randomized Control Trials: A Review of Three Quasi-experimental Designs for Causal Inference

Abstract. The Randomized Control Trial (RCT) design is typically seen as the gold standard in psychological research. As it is not always possible to conform to RCT specifications, many studies are

Limitless Regression Discontinuity

Conventionally, regression discontinuity analysis contrasts a univariate regression’s limits as its independent variable, R, approaches a cut point, c, from either side. Alternative methods target

References

SHOWING 1-10 OF 43 REFERENCES

Shaping Policies Related to Developmental Education: An Evaluation Using the Regression-Discontinuity Design

Utilizing the regression-discontinuity research design, this article explores the effectiveness of a developmental English program in a large, multicampus community college. Routinely collected data

Statistical Power for Regression Discontinuity Designs in Education Evaluations

This article examines theoretical and empirical issues related to the statistical power of impact estimates under clustered regression discontinuity (RD) designs. The theory is grounded in the causal

Comparison of a Randomized and Two Quasi-Experimental Designs in a Single Outcome Evaluation

The authors assessed the impact of three designs (randomized experiment, nonequivalent control group design, regression discontinuity design) on estimates of effect size of a university-level freshman remedial writing program, which led to highly similar effect size estimates.

A randomized experiment comparing random and cutoff-based assignment.

This study randomly assigns 588 participants to be in a randomized experiment or a regression discontinuity design in which they are otherwise treated identically, comparing results estimating both the same and different parameters.

Applying the Regression-Discontinuity Design to Infer Causality with Non-Random Assignment

The impact of developmental mathematics programs on student success in college-level mathematics has been a controversial topic in higher education. One limitation of the current research is that

Do remedial English programs impact first‐year success in college? An illustration of the regression‐discontinuity design

This paper presents an illustration of how the regression‐discontinuity design can be used to obtain an unbiased estimate of the causal effect of participating in a university remedial English

Does Mathematics Remediation Work?: A Comparative Analysis of Academic Attainment among Community College Students

Postsecondary remediation is a controversial topic. On one hand, it fills an important and sizeable niche in higher education. On the other hand, critics argue that it wastes tax dollars, diminishes

Modern Regression Discontinuity Analysis

Abstract This article provides a detailed discussion of the theory and practice of modern regression discontinuity (RD) analysis for estimating the effects of interventions or treatments. Part 1

Help or Hindrance? The Effects of College Remediation on Academic and Labor Market Outcomes

Abstract Providing remedial (also known as developmental) education is the primary way colleges cope with students who do not have the academic preparation needed to succeed in college-level courses.