Corpus ID: 235446370

Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components

@article{Garciarena2021RedefiningNA,
  title={Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components},
  author={Unai Garciarena and Roberto Santana and A. Mendiburu},
  journal={ArXiv},
  year={2021},
  volume={abs/2106.08972}
}
With neural architecture search methods gaining ground on manually designed deep neural networks -even more rapidly as model sophistication escalates-, the research trend shifts towards arranging different and often increasingly complex neural architecture search spaces. In this conjuncture, delineating algorithms which can efficiently explore these search spaces can result in a significant improvement over currently used methods, which, in general, randomly select the structural variation… Expand

Figures from this paper

References

SHOWING 1-10 OF 69 REFERENCES
Automatic Structural Search for Multi-task Learning VALPs
TLDR
This work proposes a set of model structure-modifying operators designed specifically for the VALP, a recently introduced multi-network model for heterogeneous multi-task problems and indicates that the modifiers can indeed form part of intelligent searches over the space of VALP structures, which encourages more research in this direction. Expand
Rethinking Architecture Selection in Differentiable NAS
TLDR
This work proposes an alternative perturbation-based architecture selection that directly measures each operation’s influence on the supernet and finds that it is able to extract significantly improved architectures from the underlying supernets consistently and re-evaluate several differentiable NAS methods with the proposed architecture selection. Expand
Evolutionary architecture search for deep multitask networks
TLDR
A synergetic approach of evolving custom routings with evolved, shared modules for each task is found to be very powerful, significantly improving the state of the art in the Omniglot multitask, multialphabet character recognition domain. Expand
SMASH: One-Shot Model Architecture Search through HyperNetworks
TLDR
A technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture is proposed, achieving competitive performance with similarly-sized hand-designed networks. Expand
Towards Automatic Construction of Multi-Network Models for Heterogeneous Multi-Task Learning
TLDR
This work formally defines a multi-network model, identifying the necessary components and characteristics to allow different adaptations of said model depending on the tasks it is required to fulfill, and develops an illustrative model example consisting of three different tasks. Expand
ModuleNet: Knowledge-inherited Neural Architecture Search
TLDR
A new NAS algorithm is proposed, namely, ModuleNet, which can fully inherit knowledge from the existing convolutional neural networks, and is able to directly search for architectures in the macrospace by the NSGA-II algorithm without tuning parameters in these modules. Expand
Regularized Evolution for Image Classifier Architecture Search
TLDR
This work evolves an image classifier---AmoebaNet-A---that surpasses hand-designs for the first time and gives evidence that evolution can obtain results faster with the same hardware, especially at the earlier stages of the search. Expand
Neural Architecture Search with Reinforcement Learning
TLDR
This paper uses a recurrent network to generate the model descriptions of neural networks and trains this RNN with reinforcement learning to maximize the expected accuracy of the generated architectures on a validation set. Expand
Expanding variational autoencoders for learning and exploiting latent representations in search distributions
TLDR
It is shown that VAE can capture dependencies between decision variables and objectives, which is proven to improve the sampling capacity of model based EAs and represents a promising direction for the application of generative models within EDAs. Expand
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
TLDR
A new scaling method is proposed that uniformly scales all dimensions of depth/width/resolution using a simple yet highly effective compound coefficient and is demonstrated the effectiveness of this method on scaling up MobileNets and ResNet. Expand
...
1
2
3
4
5
...