Corpus ID: 202539260

Covariate Selection for Generalizing Experimental Results

  title={Covariate Selection for Generalizing Experimental Results},
  author={Naoki Egami and E. Hartman},
  journal={arXiv: Methodology},
Scientists are interested in generalizing causal effects estimated in an experiment to a target population. However, analysts are often constrained by available covariate information, which has limited applicability of existing approaches that assume rich covariate data from both experimental and population samples. As a concrete context, we focus on a large-scale development program, called the Youth Opportunities Program (YOP), in Uganda. Although more than 40 pre-treatment covariates are… Expand

Figures and Tables from this paper

Conducting Experiments in Multiple Contexts
In an effort to assess the generalizability of treatment effects across contexts, scholars (or teams of scholars) are increasingly conducting experiments around the same research questions inExpand
Knowledge Decays: Temporal Validity and Social Science in a Changing World
The “credibility revolution” has forced social scientists to confront the limits of our methods for creating general knowledge. The current approach aims to aggregate valid but local knowledge. AtExpand
The findings of a surgical hip fracture trial were generalisable to the UK national hip fracture database.
This generalisability analysis of the WHiTE-3 trial found that the inferences from the trial can be generalised to the group of individuals in the UK NHFD and the WH iTE-cohort who met the inclusion criteria for WHi TE-3. Expand
Temporal Validity in Online Social Science
The centrality of the internet to modern life means that the social and political world is changing faster and less predictably than ever before. At the same time, the “credibility revolution” hasExpand
Identification of Causal Diffusion Effects Using Stationary Causal Directed Acyclic Graphs
Although social scientists have long been interested in the process through which ideas and behavior diffuse, the identification of causal diffusion effects, also known as peer effects, remainsExpand


Assessing Methods for Generalizing Experimental Impact Estimates to Target Populations
This article examines whether generalization may be assisted by statistical methods that adjust for observed differences between the experimental participants and members of a target population and suggests that flexible modeling techniques can aid generalization. Expand
Partially Identified Treatment Effects for Generalizability
ABSTRACT Recent methods to improve generalizations from nonrandom samples typically invoke assumptions such as the strong ignorability of sample selection, which is challenging to meet in practice.Expand
Weighting for External Validity
External validity is a fundamental challenge in treatment effect estimation. Even when researchers credibly identify average treatment effects – for example through randomized experiments – theExpand
Sensitivity analysis for an unobserved moderator in RCT-to-target-population generalization of treatment effects
This paper considers sensitivity analyses for two situations: (1) where the authors cannot adjust for a specific moderator $V$ observed in the RCT because they do not observe it in the target population; and (2) where they are concerned that the treatment effect may be moderated by factors not observed even in theRCT, which they represent as a composite moderator $U$. Expand
Improving Generalizations From Experiments Using Propensity Score Subclassification
As a result of the use of random assignment to treatment, randomized experiments typically have high internal validity. However, units are very rarely randomly selected from a well-defined populationExpand
Assessing the Generalizability of Randomized Trial Results to Target Populations
It is illustrated that statistical methods can be used to assess and enhance the external validity of randomized trials, making the results more applicable to policy and clinical questions. Expand
The use of propensity scores to assess the generalizability of results from randomized trials.
These metrics can serve as a first step in assessing the generalizability of results from randomized trials to target populations, and are illustrated using data on the evaluation of a schoolwide prevention program called Positive Behavioral Interventions and Supports. Expand
The use of propensity scores and observational data to estimate randomized controlled trial generalizability bias.
It is found that a class of estimators can be unbiased for the true generalizability bias even when all potentially confounding variables are not measured, and the proposed doubly robust estimator performs well even for mis-specified models. Expand
External Validity: From Do-Calculus to Transportability Across Populations
A formal representation called "selection diagrams" for expressing knowledge about differences and commonalities between populations of interest is introduced and questions of transportability are reduced to symbolic derivations in the do-calculus. Expand
Generalizing Evidence from Randomized Trials using Inverse Probability of Sampling Weights.
An inverse probability of sampling weighted (IPSW) estimator is considered for generalizing trial results to a target population and is shown to be consistent and asymptotically normal. Expand