Corpus ID: 236881447

Elastic Architecture Search for Diverse Tasks with Different Resources

  title={Elastic Architecture Search for Diverse Tasks with Different Resources},
  author={Jing Liu and Bohan Zhuang and Mingkui Tan and Xu Liu and Dinh Q. Phung and YuanQing Li and Jianfei Cai},
We study a new challenging problem of efficient deployment for diverse tasks with different resources, where the resource constraint and task of interest corresponding to a group of classes are dynamically specified at testing time. Previous NAS approaches seek to design architectures for all classes simultaneously, which may not be optimal for some individual tasks. A straightforward solution is to search an architecture from scratch for each deployment scenario, which however is computation… Expand


InstaNAS: Instance-aware Neural Architecture Search
InstaNAS is proposed, an instance-aware NAS framework that employs a controller trained to search for a "distribution of architectures" instead of a single architecture, and can achieve up to 48.8% latency reduction without compromising accuracy on a series of datasets against MobileNetV2. Expand
Pareto-Frontier-aware Neural Architecture Generation for Diverse Budgets
A Pareto-Frontier-aware Neural Architecture Generator (NAG) is proposed which takes an arbitrary budget as input and produces the Pare to optimal architecture for the target budget. Expand
Once for All: Train One Network and Specialize it for Efficient Deployment
This work proposes to train a once-for-all (OFA) network that supports diverse architectural settings by decoupling training and search, to reduce the cost and propose a novel progressive shrinking algorithm, a generalized pruning method that reduces the model size across many more dimensions than pruning. Expand
Efficient Neural Architecture Search via Parameter Sharing
Efficient Neural Architecture Search is a fast and inexpensive approach for automatic model design that establishes a new state-of-the-art among all methods without post-training processing and delivers strong empirical performances using much fewer GPU-hours. Expand
Rethinking the Value of Network Pruning
It is found that with optimal learning rate, the "winning ticket" initialization as used in Frankle & Carbin (2019) does not bring improvement over random initialization, and the need for more careful baseline evaluations in future research on structured pruning methods is suggested. Expand
BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models
The proposed BigNAS, an approach that challenges the conventional wisdom that post-processing of the weights is necessary to get good prediction accuracies, is proposed, able to train a single set of shared weights on ImageNet and use these weights to obtain child models whose sizes range from 200 to 1000 MFLOPs. Expand
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
ProxylessNAS is presented, which can directly learn the architectures for large-scale target tasks and target hardware platforms and apply ProxylessNAS to specialize neural architectures for hardware with direct hardware metrics (e.g. latency) and provide insights for efficient CNN architecture design. Expand
NSGANetV2: Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Search
An efficient NAS algorithm for generating task-specific models that are competitive under multiple competing objectives that comprises of two surrogates, one at the architecture level to improve sample efficiency and one through a supernet, to improve gradient descent training efficiency. Expand
Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search
The Curriculum Neural Architecture Search (CNAS) method significantly improves the search efficiency and finds better architectures than existing NAS methods. Expand
Single Path One-Shot Neural Architecture Search with Uniform Sampling
A Single Path One-Shot model is proposed to construct a simplified supernet, where all architectures are single paths so that weight co-adaption problem is alleviated. Expand