Budget-Aware Adapters for Multi-Domain Learning

@article{Berriel2019BudgetAwareAF,
  title={Budget-Aware Adapters for Multi-Domain Learning},
  author={Rodrigo Berriel and St{\'e}phane Lathuili{\`e}re and Moin Nabi and Tassilo Klein and Thiago Oliveira-Santos and N. Sebe and Elisa Ricci},
  journal={2019 IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2019},
  pages={382-391}
}
Multi-Domain Learning (MDL) refers to the problem of learning a set of models derived from a common deep architecture, each one specialized to perform a task in a certain domain (e.g., photos, sketches, paintings. [...] Key Method To implement this idea we derive specialized deep models for each domain by adapting a pre-trained architecture but, differently from other methods, we propose a novel strategy to automatically adjust the computational complexity of the network.Expand
An Alternating Training Method of Attention-Based Adapters for Visual Explanation of Multi-Domain Satellite Images
TLDR
A multi-domain learning method based on attention-based adapters to resolve the two issues in the development of a deep model for satellite image analytics and its enhancement in visual explanability is validated.
Incremental multi-domain learning with network latent tensor factorization
TLDR
This paper presents a method to learn new-domains and tasks incrementally, building on prior knowledge from already learned tasks and without catastrophic forgetting by jointly parametrizing weights across layers using low-rank Tucker structure and shows that leveraging tensor structure enables better performance than simply using matrix operations.
Boosting binary masks for multi-domain learning through affine transformations
TLDR
This work provides a general formulation of binary mask-based models for multi-domain learning by affine transformations of the original network parameters by solving the challenge of producing a single model performing a task in all the domains together.
Towards Recognizing New Semantic Concepts in New Visual Domains
TLDR
This thesis argues that it is crucial to design deep architectures that can operate in previously unseen visual domains and recognize novel semantic concepts, and proposes an approach based on domain and semantic mixing of inputs and features, which is a first, promising step towards solving this problem.
$DA^3$:Dynamic Additive Attention Adaption for Memory-EfficientOn-Device Multi-Domain Learning
TLDR
Dynamic Additive Attention Adaption (DA), a novel memory-efficient on-device multi-domain learning method that mitigates activation memory buffering for reducing memory usage during training, and serves as dynamic gating mechanism to reduce the computation cost for fast inference.
What and Where: Learn to Plug Adapters via NAS for Multi-Domain Learning
  • Hanbin Zhao, Hao Zeng, +4 authors Xi Li
  • Computer Science, Medicine
    IEEE transactions on neural networks and learning systems
  • 2021
TLDR
This work proposes a data-driven adapter plugging strategy with neural architecture search (NAS), which automatically determines where to plug for those adapter modules, and proposes an NAS-adapter module for adapter structure design in anNAS-driven learning scheme, which automatically discovers effective adapter module structures for different domains.
Disentangling Transfer and Interference in Multi-Domain Learning
TLDR
This work deciphers the conditions where interference and knowledge transfer occur in multi-domain learning, and proposes new metrics disentangling interference and transfer and sets up experimental protocols.
Learning to Balance Specificity and Invariance for In and Out of Domain Generalization
TLDR
This work introduces Domain-specific Masks for Generalization, a model for improving both in-domain and out-of-domain generalization performance and encourages the masks to learn a balance of domain-invariant and domain-specific features, thus enabling a model which can benefit from the predictive power of specialized features while retaining the universal applicability of Domain-Invariant features.
Product Categorization by Title Using Deep Neural Networks as Feature Extractor
TLDR
This work proposes an automatic system for product categorization using only their titles that employs a state-of-the-art deep neural network as a tool to extract features from the titles to be used as input in different machine learning models.
...
1
2
...

References

SHOWING 1-10 OF 41 REFERENCES
Efficient Parametrization of Multi-domain Deep Neural Networks
TLDR
This paper proposes to consider universal parametric families of neural networks, which still contain specialized problem-specific models, but differing only by a small number of parameters, and shows that these universal parametrization are very effective for transfer learning, where they outperform traditional fine-tuning techniques.
Learning multiple visual domains with residual adapters
TLDR
This paper develops a tunable deep network architecture that, by means of adapter residual modules, can be steered on the fly to diverse visual domains and introduces the Visual Decathlon Challenge, a benchmark that evaluates the ability of representations to capture simultaneously ten very differentVisual domains and measures their ability to recognize well uniformly.
Incremental Learning Through Deep Adaptation
TLDR
This work proposes a method called Deep Adaptation Modules (DAM) that constrains newly learned filters to be linear combinations of existing ones, and reduces the parameter cost to around 3 percent of the original with negligible or no loss in accuracy.
Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights
TLDR
This work learns binary masks that “piggyback” on an existing network, or are applied to unmodified weights of that network to provide good performance on a new task, and shows performance comparable to dedicated fine-tuned networks for a variety of classification tasks.
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
  • Arun Mallya, S. Lazebnik
  • Computer Science
    2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
  • 2018
TLDR
This paper is able to add three fine-grained classification tasks to a single ImageNet-trained VGG-16 network and achieve accuracies close to those of separately trained networks for each task.
Adding New Tasks to a Single Network with Weight Trasformations using Binary Masks
TLDR
This work shows that with the generalization of this approach it is possible to achieve significantly higher levels of adaptation to new tasks, enabling the approach to compete with fine tuning strategies by requiring slightly more than 1 bit per network parameter per additional task.
MobileNetV2: Inverted Residuals and Linear Bottlenecks
TLDR
A new mobile architecture, MobileNetV2, is described that improves the state of the art performance of mobile models on multiple tasks and benchmarks as well as across a spectrum of different model sizes and allows decoupling of the input/output domains from the expressiveness of the transformation.
Multi-scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation
TLDR
This paper addresses the problem of depth estimation from a single still image by designing a novel CNN implementation of mean-field updates for continuous CRFs and demonstrates the effectiveness of the proposed approach and establishes new state of the art results on publicly available datasets.
SpotTune: Transfer Learning Through Adaptive Fine-Tuning
TLDR
In SpotTune, given an image from the target task, a policy network is used to make routing decisions on whether to pass the image through the fine-tuned layers or the pre-trained layers, which outperforms the traditional fine- Tuning approach on 12 out of 14 standard datasets.
Learning without Forgetting
  • Zhizhong Li, Derek Hoiem
  • Computer Science, Mathematics
    IEEE Transactions on Pattern Analysis and Machine Intelligence
  • 2018
TLDR
This work proposes the Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities, and performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques.
...
1
2
3
4
5
...