# CASTLE: Regularization via Auxiliary Causal Graph Discovery

@article{Kyono2020CASTLERV, title={CASTLE: Regularization via Auxiliary Causal Graph Discovery}, author={Trent Kyono and Yao Zhang and Mihaela van der Schaar}, journal={ArXiv}, year={2020}, volume={abs/2009.13180} }

Regularization improves generalization of supervised models to out-of-sample data. Prior works have shown that prediction in the causal direction (effect from cause) results in lower testing error than the anti-causal direction. However, existing regularization methods are agnostic of causality. We introduce Causal Structure Learning (CASTLE) regularization and propose to regularize a neural network by jointly learning the causal relationships between variables. CASTLE learns the causal…

## 30 Citations

### GFlowCausal: Generative Flow Networks for Causal Discovery

- Computer ScienceArXiv
- 2022

This work proposes a novel approach to learning a DAG from observational data called GFlowCausal, which converts the graph search problem to a generation problem, in which direct edges are added gradually, and proposes a plug-and-play module based on transitive closure to ensure efﬁcient sampling.

### Invariant Structure Learning for Better Generalization and Causal Explainability

- Computer ScienceArXiv
- 2022

This work proposes a novel framework, Invariant Structure Learning (ISL), that is designed to improve causal structure discovery by utilizing generalization as an indication, and extends ISL to a self-supervised learning setting where accurate causal structureiscovery does not rely on any labels.

### DARING: Differentiable Causal Discovery with Residual Independence

- Computer ScienceKDD
- 2021

A novel differentiable method DARING is proposed by imposing explicit residual independence constraint in an adversarial way that can significantly improve the causal discovery performances in various scientific and industrial scenarios.

### Causal Regularization Using Domain Priors

- Computer ScienceArXiv
- 2021

This work proposes a causal regularization method that can incorporate such causal domain priors into the network and which supports both direct and total causal effects, and shows that this approach can generalize to various kinds of specifications of causal priors.

### Deep Causal Learning: Representation, Discovery and Inference

- Computer ScienceArXiv
- 2022

It is pointed out that deep causal learning is important for the theoretical extension and application expansion of causal science and is also an indispensable part of general artificial intelligence.

### On the Generalization and Adaption Performance of Causal Models

- Computer ScienceArXiv
- 2022

This work systematically study the generalization and adaption performance of such modular neural causal models by comparing it to monolithic models and structured models where the set of predictors is not constrained to causal parents.

### Matching Learned Causal Effects of Neural Networks with Domain Priors

- Computer ScienceICML
- 2022

This work proposes a regularization method that aligns the learned causal effects of a neural network with domain priors, including both direct and total causal effects, and shows that it is robust and gets improved accuracy on noisy inputs.

### Incorporating Causal Graphical Prior Knowledge into Predictive Modeling via Simple Data Augmentation

- Computer ScienceUAI
- 2021

This work proposes a model-agnostic data augmentation method that allows us to exploit the prior knowledge of the CI encoded in a CG for supervised machine learning and experimentally shows that the proposed method is effective in improving the prediction accuracy, especially in the small-data regime.

### Multivariable Causal Discovery with General Nonlinear Relationships

- Computer Science
- 2022

This work shows that causal models resolve the permutation indeterminacy of ICA and proves that under strong identifiability, the inference function’s Jacobian captures the sparsity structure of the causal graph.

### DAPDAG: Domain Adaptation via Perturbed DAG Reconstruction

- Computer ScienceArXiv
- 2022

This work proposes to learn an auto-encoder that undertakes inference on population statistics given features and reconstructing a directed acyclic graph (DAG) as an auxiliary task, and demonstrates that reconstructing the DAG beneﬁts the approximate inference.

## References

SHOWING 1-10 OF 53 REFERENCES

### Causal Discovery with Reinforcement Learning

- Computer ScienceICLR
- 2020

This work proposes to use Reinforcement Learning (RL) to search for a Directed Acyclic Graph (DAG) according to a predefined score function and shows that the proposed approach not only has an improved search ability but also allows a flexible score function under the acyclicity constraint.

### Triad Constraints for Learning Causal Structure of Latent Variables

- Computer ScienceNeurIPS
- 2019

This paper designs a form of "pseudo-residual" from three variables, and shows that when causal relations are linear and noise terms are non-Gaussian, the causal direction between the latent variables for the three observed variables is identifiable by checking a certain kind of independence relationship.

### High-dimensional learning of linear causal networks via inverse covariance estimation

- Computer Science, MathematicsJ. Mach. Learn. Res.
- 2014

It is shown that when the error variances are known or estimated to close enough precision, the true DAG is the unique minimizer of the score computed using the reweighted squared l2-loss.

### Learning Directed Acyclic Graphs with Penalized Neighbourhood Regression

- Mathematics, Computer ScienceArXiv
- 2015

The main results establish support recovery guarantees and deviation bounds for a family of penalized least-squares estimators under concave regularization without assuming prior knowledge of a variable ordering.

### Invariant Models for Causal Transfer Learning

- Computer ScienceJ. Mach. Learn. Res.
- 2018

This work relaxes the usual covariate shift assumption and assumes that it holds true for a subset of predictor variables: the conditional distribution of the target variable given this subset of predictors is invariant over all tasks.

### Semi-supervised interpolation in an anticausal learning scenario

- Computer ScienceJ. Mach. Learn. Res.
- 2015

It is proved that unlabelled data help for the problem of interpolating a monotonically increasing function if and only if the orthogonality conditions are violated - which the authors only expect for the anticausal direction.

### Causal Regularization

- Computer ScienceArXiv
- 2017

The proposed causal regularizer is proposed to steer predictive models towards causally-interpretable solutions and theoretically study its properties and is shown to outperforms its L1-regularized counterpart in causal accuracy and is competitive in predictive performance.

### Learning Sparse Nonparametric DAGs

- Computer ScienceAISTATS
- 2020

A completely general framework for learning sparse nonparametric directed acyclic graphs (DAGs) from data is developed that can be applied to general nonlinear models, general differentiable loss functions, and generic black-box optimization routines.

### Causal inference by using invariant prediction: identification and confidence intervals

- Computer Science, Mathematics
- 2015

This work proposes to exploit invariance of a prediction under a causal model for causal inference: given different experimental settings (e.g. various interventions) the authors collect all models that do show invariance in their predictive accuracy across settings and interventions, and yields valid confidence intervals for the causal relationships in quite general scenarios.

### Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions

- Computer ScienceNeurIPS
- 2018

This work proposes an approach for solving causal domain adaptation problems that exploits causal inference and does not rely on prior knowledge of the causal graph, the type of interventions or the intervention targets, and demonstrates a possible implementation on simulated and real world data.