Budget-Aware Adapters for Multi-Domain Learning

@article{Berriel2019BudgetAwareAF,
  title={Budget-Aware Adapters for Multi-Domain Learning},
  author={Rodrigo Berriel and St{\'e}phane Lathuili{\`e}re and Moin Nabi and Tassilo Klein and Thiago Oliveira-Santos and N. Sebe and Elisa Ricci},
  journal={2019 IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2019},
  pages={382-391}
}
Multi-Domain Learning (MDL) refers to the problem of learning a set of models derived from a common deep architecture, each one specialized to perform a task in a certain domain (e.g., photos, sketches, paintings. [] Key Method To implement this idea we derive specialized deep models for each domain by adapting a pre-trained architecture but, differently from other methods, we propose a novel strategy to automatically adjust the computational complexity of the network.

Figures and Tables from this paper

An Alternating Training Method of Attention-Based Adapters for Visual Explanation of Multi-Domain Satellite Images
TLDR
A multi-domain learning method based on attention-based adapters to resolve the two issues in the development of a deep model for satellite image analytics and its enhancement in visual explanability is validated.
Incremental multi-domain learning with network latent tensor factorization
TLDR
This paper presents a method to learn new-domains and tasks incrementally, building on prior knowledge from already learned tasks and without catastrophic forgetting by jointly parametrizing weights across layers using low-rank Tucker structure and shows that leveraging tensor structure enables better performance than simply using matrix operations.
Boosting binary masks for multi-domain learning through affine transformations
TLDR
This work provides a general formulation of binary mask-based models for multi-domain learning by affine transformations of the original network parameters by solving the challenge of producing a single model performing a task in all the domains together.
Task Adaptive Parameter Sharing for Multi-Task Learning
TLDR
Task Adaptive Parameter Sharing (TAPS), a general method for tuning a base model to a new task by adaptively modifying a small, task-specific subset of layers, enables multi-task learning while minimizing resources used and competition between tasks.
Towards Recognizing New Semantic Concepts in New Visual Domains
TLDR
This thesis argues that it is crucial to design deep architectures that can operate in previously unseen visual domains and recognize novel semantic concepts, and proposes an approach based on domain and semantic mixing of inputs and features, which is a first, promising step towards solving this problem.
$DA^3$:Dynamic Additive Attention Adaption for Memory-EfficientOn-Device Multi-Domain Learning
TLDR
Dynamic Additive Attention Adaption (DA), a novel memory-efficient on-device multi-domain learning method that mitigates activation memory buffering for reducing memory usage during training, and serves as dynamic gating mechanism to reduce the computation cost for fast inference.
KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning
TLDR
The model, Knowledge-Guided Simple Primitives (KG-SP), achieves state of the art in both OW-CZ SL and pCZSL, sur-passing most recent competitors even when coupled with semi-supervised learning techniques.
Learning to Balance Specificity and Invariance for In and Out of Domain Generalization
TLDR
This work introduces Domain-specific Masks for Generalization, a model for improving both in-domain and out-of-domain generalization performance and encourages the masks to learn a balance of domain-invariant and domain-specific features, thus enabling a model which can benefit from the predictive power of specialized features while retaining the universal applicability of Domain-Invariant features.
Disentangling Transfer and Interference in Multi-Domain Learning
TLDR
This work proposes new metrics disentangling interference and transfer, set up experimental protocols, and examines the roles of network capacity, task grouping, and dynamic loss weighting in reducing interference and facilitating transfer in multi-domain learning.
Unidirectional Thin Adapter for Efficient Adaptation of Deep Neural Networks
TLDR
Since UDTA learns the target task without modifying the backbone, a single backbone can adapt to multiple tasks by learning only UDTAs separately, which saves computation for training significantly.
...
1
2
...

References

SHOWING 1-10 OF 41 REFERENCES
Efficient Parametrization of Multi-domain Deep Neural Networks
TLDR
This paper proposes to consider universal parametric families of neural networks, which still contain specialized problem-specific models, but differing only by a small number of parameters, and shows that these universal parametrization are very effective for transfer learning, where they outperform traditional fine-tuning techniques.
Learning multiple visual domains with residual adapters
TLDR
This paper develops a tunable deep network architecture that, by means of adapter residual modules, can be steered on the fly to diverse visual domains and introduces the Visual Decathlon Challenge, a benchmark that evaluates the ability of representations to capture simultaneously ten very differentVisual domains and measures their ability to recognize well uniformly.
Incremental Learning Through Deep Adaptation
TLDR
This work proposes a method called Deep Adaptation Modules (DAM) that constrains newly learned filters to be linear combinations of existing ones, and reduces the parameter cost to around 3 percent of the original with negligible or no loss in accuracy.
Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights
TLDR
This work learns binary masks that “piggyback” on an existing network, or are applied to unmodified weights of that network to provide good performance on a new task, and shows performance comparable to dedicated fine-tuned networks for a variety of classification tasks.
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
  • Arun Mallya, S. Lazebnik
  • Computer Science
    2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
  • 2018
TLDR
This paper is able to add three fine-grained classification tasks to a single ImageNet-trained VGG-16 network and achieve accuracies close to those of separately trained networks for each task.
Adding New Tasks to a Single Network with Weight Trasformations using Binary Masks
TLDR
This work shows that with the generalization of this approach it is possible to achieve significantly higher levels of adaptation to new tasks, enabling the approach to compete with fine tuning strategies by requiring slightly more than 1 bit per network parameter per additional task.
MobileNetV2: Inverted Residuals and Linear Bottlenecks
TLDR
A new mobile architecture, MobileNetV2, is described that improves the state of the art performance of mobile models on multiple tasks and benchmarks as well as across a spectrum of different model sizes and allows decoupling of the input/output domains from the expressiveness of the transformation.
Multi-scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation
TLDR
This paper addresses the problem of depth estimation from a single still image by designing a novel CNN implementation of mean-field updates for continuous CRFs and demonstrates the effectiveness of the proposed approach and establishes new state of the art results on publicly available datasets.
SpotTune: Transfer Learning Through Adaptive Fine-Tuning
TLDR
In SpotTune, given an image from the target task, a policy network is used to make routing decisions on whether to pass the image through the fine-tuned layers or the pre-trained layers, which outperforms the traditional fine- Tuning approach on 12 out of 14 standard datasets.
Learning without Forgetting
TLDR
This work proposes the Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities, and performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques.
...
1
2
3
4
5
...