MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learning

@inproceedings{Vandenhende2020MTINetMT,
  title={MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learning},
  author={Simon Vandenhende and Stamatios Georgoulis and Luc Van Gool},
  booktitle={ECCV},
  year={2020}
}
In this paper, we argue about the importance of considering task interactions at multiple scales when distilling task information in a multi-task learning setup. In contrast to common belief, we show that tasks with high affinity at a certain scale are not guaranteed to retain this behaviour at other scales, and vice versa. We propose a novel architecture, namely MTI-Net, that builds upon this finding in three ways. First, it explicitly models task interactions at every scale via a multi-scale… 
Multi-Task Learning with Deep Neural Networks: A Survey
TLDR
An overview of multi-task learning methods for deep neural networks is given, with the aim of summarizing both the well-established and most recent directions within the field.
Revisiting Multi-Task Learning in the Deep Learning Era
TLDR
This survey provides a well-rounded view on state-of-the-art MTL techniques within the context of deep neural networks and examines various optimization methods to tackle the joint learning of multiple tasks.
Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference
TLDR
The reparameterization enables the model to learn new tasks without adversely affecting the performance of existing ones and achieves state-of-the-art on two challenging multi-task learning benchmarks, PASCAL-Context and NYUD, and also demonstrates superior incremental learning capability as compared to its close competitors.
Maximum Roaming Multi-Task Learning
TLDR
Maximum Roaming is proposed, a method inspired by dropout that randomly varies the parameter partitioning, while forcing them to visit as many tasks as possible at a regulated frequency, so that the network fully adapts to each update.
InverseForm: A Loss Function for Structured Boundary-Aware Segmentation
TLDR
A novel boundary-aware loss term for semantic segmentation using an inverse-transformation network, which efficiently learns the degree of parametric transformations between estimated and target boundaries, which allows consistent and significant performance improvement on segmentation backbone models without increasing their size and computational complexity.
Medusa: Universal Feature Learning via Attentional Multitasking
TLDR
It is argued that MTL is a stepping stone towards universal feature learning (UFL), which is the ability to learn generic features that can be applied to new tasks without retraining, and Medusa is proposed to realize this goal, designing task heads with dual attention mechanisms.
Lifting the Veil of Frequency in Joint Segmentation and Depth Estimation
TLDR
A Frequency Aware Feature Enhancement (FAFE) network that can effectively enhance the reciprocal relationship whereas avoiding the competition is proposed, and a novel local-aware consistency loss function is proposed to impose on the predicted segmentation and depth so as to strengthen the cooperation.
Inverted Pyramid Multi-task Transformer for Dense Scene Understanding
TLDR
This paper proposes a novel end-to-end Inverted Pyramid multi- task (InvPT) Transformer to perform a simultaneous modeling of spatial positions and multiple tasks in a unified framework and achieves superior multi-task performance on NYUD-v2 and PASCAL-Context datasets respectively, and significantly outperforms previous state-of-the-arts.
Multi-Task Classification of Sewer Pipe Defects and Properties using a Cross-Task Graph Neural Network Decoder
TLDR
This work classifies sewer pipe defects and properties concurrently and presents a novel decoder-focused multi-task classification architecture Cross-Task Graph Neural Network (CT-GNN), which refines the disjointed per-task predictions using cross-task information.
On Steering Multi-Annotations per Sample for Multi-Task Learning
TLDR
Stochastic Task Allocation (STA) is introduced, a mechanism that addresses the issue of optimally learning different tasks simultaneously by a task allocation approach, in which each sample is randomly allocated a subset of tasks.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 54 REFERENCES
End-To-End Multi-Task Learning With Attention
TLDR
The proposed Multi-Task Attention Network (MTAN) consists of a single shared network containing a global feature pool, together with a soft-attention module for each task, which allows learning of task-specific feature-level attention.
Branched Multi-Task Networks: Deciding what layers to share
TLDR
This paper proposes an approach to automatically construct branched multi-task networks, by leveraging the employed tasks' affinities, given a specific budget, and generates architectures, in which shallow layers are task-agnostic, whereas deeper ones gradually grow more task-specific.
Attentive Single-Tasking of Multiple Tasks
In this work we address task interference in universal networks by considering that a network is trained on multiple tasks, but performs one task at a time, an approach we refer to as
Which Tasks Should Be Learned Together in Multi-task Learning?
TLDR
This framework offers a time-accuracy trade-off and can produce better accuracy using less inference time than not only a single large multi-task neural network but also many single-task networks.
Fully-Adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification
TLDR
Evaluation on person attributes classification tasks involving facial and clothing attributes suggests that the models produced by the proposed method are fast, compact and can closely match or exceed the state-of-the-art accuracy from strong baselines by much more expensive models.
PAD-Net: Multi-tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing
TLDR
This paper proposes a novel multi-task guided prediction-and-distillation network (PAD-Net), which first predicts a set of intermediate auxiliary tasks ranging from low level to high level, and then the predictions from these intermediate Auxiliary tasks are utilized as multi-modal input via the authors' proposed multi- modal distillation modules for the final tasks.
Multi-Task Learning as Multi-Objective Optimization
TLDR
This paper proposes an upper bound for the multi-objective loss and shows that it can be optimized efficiently, and proves that optimizing this upper bound yields a Pareto optimal solution under realistic assumptions.
Cross-Stitch Networks for Multi-task Learning
TLDR
This paper proposes a principled approach to learn shared representations in Convolutional Networks using multitask learning using a new sharing unit: "cross-stitch" unit that combines the activations from multiple networks and can be trained end-to-end.
GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
TLDR
A gradient normalization (GradNorm) algorithm that automatically balances training in deep multitask models by dynamically tuning gradient magnitudes is presented, showing that for various network architectures, for both regression and classification tasks, and on both synthetic and real datasets, GradNorm improves accuracy and reduces overfitting across multiple tasks.
Dynamic Task Prioritization for Multitask Learning
TLDR
This work proposes a notion of dynamic task prioritization to automatically prioritize more difficult tasks by adaptively adjusting the mixing weight of each task’s loss objective and outperforms existing multitask methods and demonstrates competitive results with modern single-task models on the COCO and MPII datasets.
...
1
2
3
4
5
...