Design and Analysis of Experiments in Networks: Reducing Bias from Interference

  title={Design and Analysis of Experiments in Networks: Reducing Bias from Interference},
  author={Dean Eckles and Brian Karrer and Johan Ugander},
  journal={Journal of Causal Inference},
Abstract Estimating the effects of interventions in networks is complicated due to interference, such that the outcomes for one experimental unit may depend on the treatment assignments of other units. [] Key Method In design, we consider the ability to perform random assignment to treatments that is correlated in the network, such as through graph cluster randomization. In analysis, we consider incorporating information about the treatment assignment of network neighbors.

Figures from this paper

Model-assisted design of experiments in the presence of network-correlated outcomes
This work develops a class of models that posit a correlation structure among the outcomes of a randomized experiment, and leverages these models to develop restricted randomization strategies for allocating treatment optimally, by minimizing the mean square error of the estimated average treatment effect.
Graph Agnostic Randomized Experimental Design under Heterogeneous Linear Network Interference
This work considers a heterogeneous linear outcomes model that can capture network interference that arises from spillover, peer effects, and contagion and proposes simple estimators and randomized designs that output unbiased estimates with low variance for these three estimands.
Minimizing Interference and Selection Bias in Network Experiment Design
It is shown that cluster randomization does not ensure sufficient node randomization and it can lead to selection bias in which treatment and control nodes represent different populations of users.
Identification and Estimation of Treatment and Interference Effects in Observational Studies on Networks
An extended unconfoundedness assumption that accounts for interference is proposed, and new covariate-adjustment methods are developed that lead to valid estimates of treatment and interference effects in observational studies on networks.
Optimal model-assisted design of experiments for network correlated outcomes suggests new notions of network balance
We consider the problem of how to assign treatment in a randomized experiment, in which the correlation among the outcomes is informed by a network available pre-intervention. Working within the
Random Graph Asymptotics for Treatment Effect Estimation under Network Interference
The network interference model for causal inference places all experimental units at the vertices of an undirected exposure graph, such that treatment assigned to one unit may affect the outcome of
Estimating average causal effects under general interference, with application to a social network experiment
This paper develops the case of estimating average unit-level causal effects from a randomized experiment with interference of arbitrary but known form and provides randomization-based variance estimators that account for the complex clustering that can occur when interference is present.
Graph Agnostic Randomized Experimental Design
This work considers a heterogeneous linear outcomes model that can capture network interference that arises from spillover, peer effects, and contagion and proposes simple estimators and randomized designs that output unbiased estimates with low variance for these three estimands.
Designs for estimating the treatment effect in networks with interference
In this paper we introduce new, easily implementable designs for drawing causal inference from randomized experiments on networks with interference. Inspired by the idea of matching in observational


Exact p-Values for Network Interference
ABSTRACT We study the calculation of exact p-values for a large class of nonsharp null hypotheses about treatment effects in a setting with data from experiments involving members of a single
Graph cluster randomization: network exposure to multiple universes
It is shown that proper cluster randomization can lead to exponentially lower estimator variance when experimentally measuring average treatment effects under interference, and if a graph satisfies a restricted-growth condition on the growth rate of neighborhoods, then there exists a natural clustering algorithm, based on vertex neighborhoods, for which the variance of the estimator can be upper bounded by a linear function of the degrees.
Estimation of Monotone Treatment Effects in Network Experiments
New methods for finding confidence intervals on the attributable treatment effect in Randomized experiments on social networks, which do not require partial interference, but instead require an identifying assumption that is similar to requiring nonnegative treatment effects.
Toward Causal Inference With Interference
This article considers a population of groups of individuals where interference is possible between individuals within the same group, and proposes estimands for direct, indirect, total, and overall causal effects of treatment strategies in this setting.
A General Method for Detecting Interference Between Units in Randomized Experiments
Interference between units may pose a threat to unbiased causal inference in randomized controlled experiments. Although the assumption of no interference is often necessary for causal inference, few
Design of Randomized Experiments in Networks
This work categorize and review the emerging strategies to design and analyze experiments in networks and discuss their strengths and weaknesses.
Interference Between Units in Randomized Experiments
In a randomized experiment comparing two treatments, there is interference between units if applying the treatment to one unit may affect other units. Interference implies that treatment effects are
Estimating causal effects of treatments in randomized and nonrandomized studies.
A discussion of matching, randomization, random sampling, and other methods of controlling extraneous variation is presented. The objective is to specify the benefits of randomization in estimating
What Do Randomized Studies of Housing Mobility Demonstrate?
A frame-work for causal inference when interference is present is developed and a number of causal estimands of interest are defined, which are of great importance for a researcher who fails to recognize that a treatment is beneficial when in fact it is universally harmful.
Network A/B Testing: From Sampling to Estimation
The existence of network effect is examined in a recent online experiment conducted at LinkedIn, an efficient and effective estimator for Average Treatment Effect (ATE) considering the interference between users in real online experiments is proposed, and the method is applied in both simulations and a real world online experiment.