Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability

  title={Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability},
  author={Jonathan Crabbe and Alicia Curth and Ioana Bica and Mihaela van der Schaar},
Estimating personalized effects of treatments is a complex, yet pervasive problem. To tackle it, recent developments in the machine learning (ML) literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools: due to their flexibility, modularity and ability to learn constrained representations, neural networks in particular have become central to this literature. Unfortunately, the assets of such black boxes come at a cost: models typically involve… 

Figures and Tables from this paper

Exploring Transformer Backbones for Heterogeneous Treatment Effect Estimation

It is shown empirically that TransTEE can serve as a general purpose treatment effect estimator that outperforms competitive baselines in a variety of challenging TEE problems and yield multiple advantages: compatibility with propensity score modeling, parameter efficiency, robustness to continuous treatment value distribution shifts, explainable in covariate adjustment, and real-world utility in auditing pre-trained language models.



Metalearners for estimating heterogeneous treatment effects using machine learning

A metalearner, the X-learner, is proposed, which can adapt to structural properties, such as the smoothness and sparsity of the underlying treatment effect, and is shown to be easy to use and to produce results that are interpretable.

On Inductive Biases for Heterogeneous Treatment Effect Estimation

Three end-to-end learning strategies are investigated based on regularization, reparametrization and a flexible multi-task architecture – each encoding inductive bias favoring shared behavior across POs and observing that all three approaches can lead to substantial improvements upon numerous baselines and gain insight into performance differences across various experimental settings.

Really Doing Great at Estimating CATE? A Critical Look at ML Benchmarking Practices in Treatment Effect Estimation

This paper investigates current benchmarking practices for ML-based conditional average treatment effect (CATE) estimators, with special focus on empirical evaluation based on the popular semi-synthetic IHDP benchmark.

Limits of Estimating Heterogeneous Treatment Effects: Guidelines for Practical Algorithm Design

This paper characterizes the fundamental limits of estimating heterogeneous treatment effects, and establishes conditions under which these limits can be achieved, and builds a practical algorithm for estimating treatment effects using a non-stationary Gaussian processes with doubly-robust hyperparameters.

Covariate-Balancing-Aware Interpretable Deep Learning models for Treatment Effect Estimation

A theoretical analysis is provided and an upper bound for the bias of average treatment effect (ATE) estimation under the strong ignorability assumption is derived, which is tighter than what has been reported in the literature.

Learning Interpretable Models with Causal Guarantees

This work proposes a framework for learning causal interpretable models---from observational data---that can be used to predict individual treatment effects and proves an error bound on the treatment effects predicted by the model.

From Real‐World Patient Data to Individualized Treatment Effects Using Machine Learning: Current and Future Methods to Address Underlying Challenges

The modeling choices of the state‐of‐the‐art machine learning methods for causal inference, developed for estimating treatment effects both in the cross‐section and longitudinal settings are described.

Estimating individual treatment effect: generalization bounds and algorithms

A novel, simple and intuitive generalization-error bound is given showing that the expected ITE estimation error of a representation is bounded by a sum of the standard generalized-error of that representation and the distance between the treated and control distributions induced by the representation.

Learning Disentangled Representations for CounterFactual Regression

This work proposes an algorithm to identify disentangled representations of the above-mentioned underlying factors from any given observational dataset D and leverage this knowledge to reduce, as well as account for, the negative impact of selection bias on estimating the treatment effects from D.

A Unified Approach to Interpreting Model Predictions

A unified framework for interpreting predictions, SHAP (SHapley Additive exPlanations), which unifies six existing methods and presents new methods that show improved computational performance and/or better consistency with human intuition than previous approaches.