• Corpus ID: 246680177

Generative multitask learning mitigates target-causing confounding

@article{Makino2022GenerativeML,
  title={Generative multitask learning mitigates target-causing confounding},
  author={Taro Makino and Krzysztof J Geras and Kyunghyun Cho},
  journal={ArXiv},
  year={2022},
  volume={abs/2202.04136}
}
We propose a simple and scalable approach to causal representation learning for multitask learning. Our approach requires minimal modification to existing ML systems, and improves robustness to target shift. The improvement comes from mitigating unobserved confounders that cause the targets, but not the input. We refer to them as target-causing confounders. These confounders induce spurious dependencies between the input and targets. This poses a problem for the conventional approach to… 
1 Citations

taochen at SemEval-2022 Task 5: Multimodal Multitask Learning and Ensemble Learning

TLDR
A multi-modal deep learning system for the Multimedia Automatic Misogyny Identification (MAMI) challenge, a SemEval task of identifying and classifying misogynistic messages in online memes, and applies multi-task learning for the multimodal subtasks to transfer knowledge among the correlated subtasks.

References

SHOWING 1-10 OF 49 REFERENCES

Nonlinear Invariant Risk Minimization: A Causal Approach

TLDR
Invariant Causal Representation Learning is proposed, a learning paradigm that enables out-of-distribution (OOD) generalization in the nonlinear setting (i.e., nonlinear representations and nonlinear classifiers) and builds upon a practical and general assumption: the prior over the data representation factorizes when conditioning on the target and the environment.

Domain Adaptation under Target and Conditional Shift

TLDR
This work considers domain adaptation under three possible scenarios, kernel embedding of conditional as well as marginal distributions, and proposes to estimate the weights or transformations by reweighting or transforming training data to reproduce the covariate distribution on the test domain.

Muppet: Massive Multi-task Representations with Pre-Finetuning

TLDR
It is shown that pre-finetuning consistently improves performance for pretrained discriminators and generation models on a wide range of tasks while also significantly improving sample efficiency during fine-tuning, and that large-scale multi-tasking is crucial.

On causal and anticausal learning

TLDR
The problem of function estimation in the case where an underlying causal model can be inferred is considered, and a hypothesis for when semi-supervised learning can help is formulated, and corroborate it with empirical results.

Cross-Stitch Networks for Multi-task Learning

TLDR
This paper proposes a principled approach to learn shared representations in Convolutional Networks using multitask learning using a new sharing unit: "cross-stitch" unit that combines the activations from multiple networks and can be trained end-to-end.

Multi-Task Deep Neural Networks for Natural Language Understanding

TLDR
A Multi-Task Deep Neural Network (MT-DNN) for learning representations across multiple natural language understanding (NLU) tasks that allows domain adaptation with substantially fewer in-domain labels than the pre-trained BERT representations.

Taskonomy: Disentangling Task Transfer Learning

TLDR
This work proposes a fully computational approach for modeling the structure of space of visual tasks via finding (first and higher-order) transfer learning dependencies across a dictionary of twenty six 2D, 2.5D, 3D, and semantic tasks in a latent space and provides a computational taxonomic map for task transfer learning.

Multitask Learning

TLDR
Prior work on MTL is reviewed, new evidence that MTL in backprop nets discovers task relatedness without the need of supervisory signals is presented, and new results for MTL with k-nearest neighbor and kernel regression are presented.

Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts

TLDR
This work proposes a novel multi-task learning approach, Multi-gate Mixture-of-Experts (MMoE), which explicitly learns to model task relationships from data and demonstrates the performance improvements by MMoE on real tasks including a binary classification benchmark, and a large-scale content recommendation system at Google.

A Brief Review of Deep Multi-task Learning and Auxiliary Task Learning

TLDR
A brief review on the recent deep multi-task learning approaches followed by methods on selecting useful auxiliary tasks that can be used in dMTL to improve the performance of the model for the main task is provided.