Multi-path Neural Networks for On-device Multi-domain Visual Classification

  title={Multi-path Neural Networks for On-device Multi-domain Visual Classification},
  author={Qifei Wang and Junjie Ke and Joshua Greaves and Grace Chu and Gabriel Bender and Luciano Sbaiz and Alec Go and Andrew G. Howard and Feng Yang and Ming-Hsuan Yang and Jeff Gilbert and Peyman Milanfar},
  journal={2021 IEEE Winter Conference on Applications of Computer Vision (WACV)},
  • Qifei Wang, Junjie Ke, +9 authors P. Milanfar
  • Published 10 October 2020
  • Computer Science
  • 2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
Learning multiple domains/tasks with a single model is important for improving data efficiency and lowering inference cost for numerous vision tasks, especially on resource-constrained mobile devices. However, hand-crafting a multi-domain/task model can be both tedious and challenging. This paper proposes a novel approach to automatically learn a multi-path network for multi-domain visual classification on mobile devices. The proposed multi-path network is learned from neural architecture… Expand
1 Citations

Figures and Tables from this paper

Memory Efficient Adaptive Attention For Multiple Domain Learning
This work suggests that a further reduction in the number of trainable parameters by an order of magnitude is possible and suggests that new modularization techniques for multi-domain learning should also be compared on other realistic metrics. Expand


End-To-End Multi-Task Learning With Attention
The proposed Multi-Task Attention Network (MTAN) consists of a single shared network containing a global feature pool, together with a soft-attention module for each task, which allows learning of task-specific feature-level attention. Expand
Fully-Adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification
Evaluation on person attributes classification tasks involving facial and clothing attributes suggests that the models produced by the proposed method are fast, compact and can closely match or exceed the state-of-the-art accuracy from strong baselines by much more expensive models. Expand
Revisiting Multi-Task Learning in the Deep Learning Era
This survey provides a well-rounded view on state-of-the-art MTL techniques within the context of deep neural networks and examines various optimization methods to tackle the joint learning of multiple tasks. Expand
Budget-Aware Adapters for Multi-Domain Learning
Budget-Aware Adapters are introduced that select the most relevant feature channels to better handle data from a novel domain and propose a novel strategy to automatically adjust the computational complexity of the network. Expand
Continual and Multi-Task Architecture Search
This work introduces a novel continual architecture search (CAS) approach, so as to continually evolve the model parameters during the sequential training of several tasks, without losing performance on previously learned tasks, thus enabling life-long learning. Expand
Learning to Branch for Multi-Task Learning
This work proposes a novel tree-structured design space that casts a tree branching operation as a gumbel-softmax sampling procedure that enables differentiable network splitting that is end-to-end trainable. Expand
Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
A principled approach to multi-task deep learning is proposed which weighs multiple loss functions by considering the homoscedastic uncertainty of each task, allowing us to simultaneously learn various quantities with different units or scales in both classification and regression settings. Expand
Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning
A collaborative multi-agent reinforcement learning (MARL) approach is employed to jointly train the router and function blocks of a routing network, a kind of self-organizing neural network consisting of a router and a set of one or more function blocks. Expand
Learning multiple visual domains with residual adapters
This paper develops a tunable deep network architecture that, by means of adapter residual modules, can be steered on the fly to diverse visual domains and introduces the Visual Decathlon Challenge, a benchmark that evaluates the ability of representations to capture simultaneously ten very differentVisual domains and measures their ability to recognize well uniformly. Expand
Efficient Parametrization of Multi-domain Deep Neural Networks
This paper proposes to consider universal parametric families of neural networks, which still contain specialized problem-specific models, but differing only by a small number of parameters, and shows that these universal parametrization are very effective for transfer learning, where they outperform traditional fine-tuning techniques. Expand