A causal framework for distribution generalization.

@article{Christiansen2021ACF,
  title={A causal framework for distribution generalization.},
  author={Rune Christiansen and Niklas Pfister and Martin Emil Jakobsen and Nicola Gnecco and Jonas Peters},
  journal={IEEE transactions on pattern analysis and machine intelligence},
  year={2021},
  volume={PP}
}
We consider the problem of predicting a response Y from a set of covariates X when test- and training distributions differ. Since such differences may have causal explanations, we consider test distributions that emerge from interventions in a structural causal model, and focus on minimizing the worst-case risk. Causal regression models, which regress the response on its direct causes, remain unchanged under arbitrary interventions on the covariates, but they are not always optimal in the above… Expand

Figures and Tables from this paper

Invariant Policy Learning: A Causal Perspective
TLDR
This paper views the environmental shift problem through the lens of causality and proposes multi-environment contextual bandits that allow for changes in the underlying mechanisms, and adopts the concept of invariance from the causality literature and introduces the notion of policy invariance. Expand
An Online Learning Approach to Interpolation and Extrapolation in Domain Generalization
TLDR
This work rigorously demonstrate that extrapolation is computationally much harder than interpolation, though their statistical complexity is not significantly different, and shows that ERM—or a noisy variant—is provably minimax-optimal for both tasks. Expand
Distributional Anchor Regression
Prediction models often fail if train and test data do not stem from the same distribution. Out-of-distribution (OOD) generalization to unseen, perturbed test data is a desirable butExpand
Domain Generalization using Causal Matching
TLDR
An iterative algorithm called MatchDG is proposed that approximates base object similarity by using a contrastive loss formulation adapted for multiple domains and learns matches that have over 25\% overlap with ground-truth object matches in MNIST and Fashion-MNIST. Expand
Learning Under Adversarial and Interventional Shifts
TLDR
This work proposes a new formulation, RISe, for designing robust models against a set of distribution shifts that are at the intersection of adversarial and interventional shifts and employs the distributionally robust optimization framework to optimize the resulting objective in both supervised and reinforcement learning settings. Expand
Optimization-based Causal Estimation from Heterogenous Environments
TLDR
Compared to classical ML and existing methods, CoCo provides more accurate estimates of the causal model, and the theoretical foundations of this approach are described and its effectiveness on simulated and real datasets are described. Expand
Robustness in Deep Learning for Computer Vision: Mind the gap?
TLDR
A structural causal model of the data generating process is introduced and non-adversarial robustness is interpreted as pertaining to a model’s behavior on corrupted images which correspond to low-probability samples from the unaltered data distribution, revealing common practices in the current literature correspond to causal concepts. Expand
The Risks of Invariant Risk Minimization
TLDR
In this setting, the first analysis of classification under the IRM objective is presented, and it is found that IRM and its alternatives fundamentally do not improve over standard Empirical Risk Minimization. Expand
Causality in Cognitive Neuroscience: Concepts, Challenges, and Distributional Robustness
TLDR
The concept of causality is introduced in the context of cognitive neuroscience and existing methods for inferring causal relationships from data are reviewed and argued for distributional robustness as a guiding principle to tackle problems of scarcity of interventional data and challenge of finding the right variables. Expand
Deconfounding and Causal Regularisation for Stability and External Validity
TLDR
How simple and user-friendly techniques improve stability, replicability and distributional robustness in heterogeneous data and additional thoughts to the issue on concept drift, raised by Efron (2020), when the data generating distribution is changing. Expand
...
1
2
...

References

SHOWING 1-10 OF 72 REFERENCES
Conditional variance penalties and domain shift robustness
TLDR
Using a causal framework, this conditional variance regularization (CoRe) is shown to protect asymptotically against shifts in the distribution of the style variables and improves predictive accuracy substantially in settings where domain changes occur in terms of image quality, brightness and color. Expand
Distributional Robustness of K-class Estimators and the PULSE
TLDR
It is proved that the classical K- class estimator satisfies such optimality by establishing a connection between K-class estimators and anchor regression and motivates a novel estimator in instrumental variable settings that minimizes the mean squared prediction error subject to the constraint that the estimator lies in an asymptotically valid confidence region of the causal coefficient. Expand
Foundations of structural causal models with cycles and latent variables
TLDR
The foundations for a general theory of statistical causal modeling with SCMs are provided, allowing for the presence of both latent confounders and cycles, and a class of simple SCMs is introduced that extends the class of acyclic SCMs to the cyclic setting, while preserving many of the convenient properties. Expand
Stabilizing variable selection and regression
We consider regression in which one predicts a response $Y$ with a set of predictors $X$ across different experiments or environments. This is a common setup in many data-driven scientific fields andExpand
Identification and Estimation of Causal Effects Defined by Shift Interventions
TLDR
The utility of the method is demonstrated by using an electronic health record dataset to estimate the effect of extending the length of stay in the intensive care unit (ICU) in a hospital by an extra day on patient ICU readmission probability. Expand
A survey on Image Data Augmentation for Deep Learning
TLDR
This survey will present existing methods for Data Augmentation, promising developments, and meta-level decisions for implementing DataAugmentation, a data-space solution to the problem of limited data. Expand
Data-Driven Optimal Transport Cost Selection For Distributionally Robust Optimization
TLDR
This paper shows rigorously that its framework encompasses adaptive regularization as a particular case, and demonstrates empirically that the proposed methodology is able to improve upon a wide range of popular machine learning estimators. Expand
Invariant Risk Minimization
TLDR
This work introduces Invariant Risk Minimization, a learning paradigm to estimate invariant correlations across multiple training distributions and shows how the invariances learned by IRM relate to the causal structures governing the data and enable out-of-distribution generalization. Expand
Kernel Instrumental Variable Regression
TLDR
KIV is proposed, a nonparametric generalization of 2SLS, modeling relations among X, Y, and Z as nonlinear functions in reproducing kernel Hilbert spaces (RKHSs), and it is proved the consistency of KIV under mild assumptions, and conditions under which convergence occurs at the minimax optimal rate for unconfounded, single-stage RKHS regression. Expand
Learning stable and predictive structures in kinetic systems
TLDR
Results on both simulated and real-world examples suggest that learning the structure of kinetic systems benefits from a causal perspective, and an approach is proposed, CausalKinetiX, that explicitly takes into account stability across different experiments. Expand
...
1
2
3
4
5
...