# A Cautionary Note on the Use of Matching to Estimate Causal Effects: An Empirical Example Comparing Matching Estimates to an Experimental Benchmark

@article{Arceneaux2010ACN, title={A Cautionary Note on the Use of Matching to Estimate Causal Effects: An Empirical Example Comparing Matching Estimates to an Experimental Benchmark}, author={Kevin Arceneaux and Alan S. Gerber and Donald P. Green}, journal={Sociological Methods \& Research}, year={2010}, volume={39}, pages={256 - 282} }

In recent years, social scientists have increasingly turned to matching as a method for drawing causal inferences from observational data. Matching compares those who receive a treatment to those with similar background attributes who do not receive a treatment. Researchers who use matching frequently tout its ability to reduce bias, particularly when applied to data sets that contain extensive background information. Drawing on a randomized voter mobilization experiment, the authors compare… Expand

#### 49 Citations

Optimizing matching and analysis combinations for estimating causal effects

- Computer Science, Medicine
- Scientific reports
- 2016

Simulation results indicate that combining full matching with double robust analysis performed best in both the simulations and the applied example, particularly when combined with machine learning estimation methods. Expand

Dude, Where’s My Treatment Effect? Errors in Administrative Data Linking and the Destruction of Statistical Power in Randomized Experiments

- Computer Science
- 2020

It is demonstrated that probabilistic linking substantially outperforms stringent linking criteria and allows researchers to recover a considerable share of the statistical power lost under stringent data-linking rules. Expand

Performance of Matching Methods to Unmatched Ordinary Least Squares Regression Under Constant Effects.

- Medicine, Mathematics
- American journal of epidemiology
- 2019

Comparing inferences from propensity score matching, coarsened exact matching, and un-matched covariate-adjusted ordinary least squares regression (OLS) to identify which methods produced unbiased inferences at the expected type I error rate suggests when estimates from matching and OLS are similar, OLS inferences are unbiased more often than matching inferences. Expand

Causal Effects Under Self-Selection

- 2013

The randomized assignment of units to treatment status provided by an experiment provides unparalleled leverage to understand the average causal effect of an assigned treatment. Yet political… Expand

A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook

- Computer Science
- 2017

It is suggested that commonly used observational approaches based on the data usually available in the advertising industry often fail to accurately measure the true effect of advertising, and the incremental explanatory power the authors' data would require to enable observational methods to successfully measure advertising effects. Expand

Advancements for Functional Form Robustness

- 2020

Social scientists face a dual problem of model uncertainty and methodological abundance. There are many different ways to conduct an analysis, but the true model is unknown. This uncertainty among… Expand

Administrative Data Linking and Statistical Power Problems in Randomized Experiments

- Computer Science
- 2019

This paper derives an analytic result for the consequences of linking errors on statistical power and shows how the problem varies across different combinations of relevant inputs, including the matching error rate, the outcome density and the sample size. Expand

Empirical Performance of Covariates in Education Observational Studies

- Mathematics
- 2017

ABSTRACT This article summarizes results from 12 empirical evaluations of observational methods in education contexts. We look at the performance of three common covariate-types in observational… Expand

Bias Reduction in Quasi-Experiments With Little Selection Theory but Many Covariates

- Mathematics
- 2015

Abstract: In observational studies, selection bias will be completely removed only if the selection mechanism is ignorable, namely, all confounders of treatment selection and potential outcomes are… Expand

Assessing Correspondence Between Experimental and Nonexperimental Estimates in Within-Study Comparisons

- Computer Science, Medicine
- Evaluation review
- 2018

Different distance-based correspondence measures for assessing correspondence in experimental and nonexperimental estimates are examined and a new and straightforward approach that combines traditional significance testing and equivalence testing in the same framework is recommended. Expand

#### References

SHOWING 1-10 OF 72 REFERENCES

Comparing Experimental and Matching Methods Using a Large-Scale Voter Mobilization Experiment

- Computer Science, Mathematics
- Political Analysis
- 2006

Testing the performance of matching by gauging the success with which matching approximates experimental results shows that brief paid get-out-the-vote phone calls do not increase turnout, while matching and regression show a large and significant effect. Expand

Characterizing Selection Bias Using Experimental Data

- Computer Science
- 1998

Semiparametric econometric methods are applied to estimate the form of selection bias that arises from using nonexperimental comparison groups to evaluate social programs and to test the identifying assumptions that justify three widely-used classes of estimators. Expand

Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference

- Computer Science
- Political Analysis
- 2007

A unified approach is proposed that makes it possible for researchers to preprocess data with matching and then to apply the best parametric techniques they would have used anyway and this procedure makes parametric models produce more accurate and considerably less model-dependent causal inferences. Expand

The bias due to incomplete matching.

- Mathematics, Medicine
- Biometrics
- 1985

A practical example shows that the bias due to incomplete matching can be severe, and moreover, can be avoided entirely by using an appropriate multivariate nearest available matching algorithm, which, in the example, leaves only a small residual biasDue to inexact matching. Expand

Practical propensity score matching: a reply to Smith and Todd

- Psychology
- 2005

Abstract This paper discusses propensity score matching in the context of Smith and Todd's (Does matching overcome Lalonde's critique of nonexperimental estimators, J. Econom., in press) reanalysis… Expand

Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme

- Economics
- 1997

This paper considers whether it is possible to devise a nonexperimental procedure for evaluating a prototypical job training programme. Using rich nonexperimental data, we examine the performance of… Expand

Estimating causal effects of treatments in randomized and nonrandomized studies.

- Psychology
- 1974

A discussion of matching, randomization, random sampling, and other methods of controlling extraneous variation is presented. The objective is to specify the benefits of randomization in estimating… Expand

Matching Estimators of Causal Effects

- 2006

As the counterfactual model of causality has increased in popularity, sociologists have returned to matching as a research methodology. In this article, advances over the past two decades in matching… Expand

Identification of Causal Effects Using Instrumental Variables

- Mathematics, Computer Science
- 1993

It is shown that the instrumental variables (IV) estimand can be embedded within the Rubin Causal Model (RCM) and that under some simple and easily interpretable assumptions, the IV estimand is the average causal effect for a subgroup of units, the compliers. Expand

Detecting selection bias, using propensity score matching, and estimating treatment effects: an application to the private returns to a master’s degree

- Mathematics, Economics
- 2006

Most research in the area of higher education is plagued by the problem of endogeneity or self-selection bias. Unlike ordinary least squares (OLS) regression, propensity score matching addresses the… Expand