ReNAS: Relativistic Evaluation of Neural Architecture Search

@article{Xu2021ReNASRE,
  title={ReNAS: Relativistic Evaluation of Neural Architecture Search},
  author={Yixing Xu and Yunhe Wang and Kai Han and Shangling Jui and Chunjing Xu and Qi Tian and Chang Xu},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2021},
  pages={4409-4418}
}
  • Yixing Xu, Yunhe Wang, +4 authors Chang Xu
  • Published 30 September 2019
  • Computer Science, Mathematics
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
An effective and efficient architecture performance evaluation scheme is essential for the success of Neural Architecture Search (NAS). To save computational cost, most of existing NAS algorithms often train and evaluate intermediate neural architectures on a small proxy dataset with limited training epochs. But it is difficult to expect an accurate performance estimation of an architecture in such a coarse evaluation way. This paper advocates a new neural architecture evaluation scheme, which… Expand

Figures and Tables from this paper

Homogeneous Architecture Augmentation for Neural Predictor
TLDR
A homogeneous architecture augmentation algorithm is proposed in HAAP to generate sufficient training data taking the use of homogeneous representation of DNN architectures to address the issue of shortage of annotated DNN architecture for effectively training the neural predictors. Expand
Weak NAS Predictors Are All You Need
TLDR
This paper shifts the paradigm from finding a complicated predictor that covers the whole architecture space to a set of weaker predictors that progressively move towards the high-performance sub-space, and achieves the state-of-the-art ImageNet performance on the NASNet search space. Expand
Evolving graph convolutional networks for neural architecture search
TLDR
An evolutionary-based NAS method is employed to evolve graph convolutional networks for the problem of predicting the relative performance of various architectures included in the NAS-Bench-101 dataset, achieving a Kendall’s tau correlation coefficient of 0.907. Expand
AceNAS: Learning to Rank Ace Neural Architectures with Weak Supervision of Weight Sharing
TLDR
This paper proposes to use Normalized Discounted Cumulative Gain as the target metric and LambdaRank as the training algorithm and proposes to leverage weak supervision from weight sharing by pretraining architecture representation on weak labels obtained from the super-net and then finetuning the ranking model using a small number of architectures trained from scratch. Expand
DC-NAS: Divide-and-Conquer Neural Architecture Search
TLDR
This work presents a divide-and-conquer (DC) approach to effectively and efficiently search deep neural architectures, and demonstrates that DC-NAS can overcome the inaccurate evaluation problem, achieving a top-1 accuracy on the ImageNet dataset, which is higher than that of state-of-the-art methods using the same search space. Expand
Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
TLDR
A literature review on the application of NAS to computer vision problems is provided and existing approaches are summarized into several categories according to their efforts in bridging the gap. Expand
A Learned Performance Model for the Tensor Processing Unit
TLDR
A method of learning performance models from a corpus of tensor computation graph programs for the Tensor Processing Unit (TPU) is demonstrated and it is found that the learned model is competitive to a heavily-optimized analytical cost model used in the production XLA compiler. Expand
Contrastive Neural Architecture Search with Neural Architecture Comparators
TLDR
This paper designs and learns a Neural Architecture Comparator (NAC) to compute the probability of candidate architectures being better than a baseline one and theoretically shows that learning NAC is equivalent to optimizing the ranking over architectures. Expand
A Generic Graph-based Neural Architecture Encoding Scheme for Predictor-based NAS
TLDR
This work proposes a novel Graph-based neural ArchiTecture Encoding Scheme, a.k.a GATES, to improve the predictor-based Neural architecture search, which models the operations as the transformation of the propagating information, which mimics the actual data processing of neural architecture. Expand
A LEARNED PERFORMANCE MODEL FOR TENSOR PROCESSING UNITS
Accurate hardware performance models are critical to efficient code generation. They can be used by compilers to make heuristic decisions, by superoptimizers as a minimization objective, or byExpand
...
1
2
...

References

SHOWING 1-10 OF 53 REFERENCES
NAS-Bench-101: Towards Reproducible Neural Architecture Search
TLDR
This work introduces NAS-Bench-101, the first public architecture dataset for NAS research, which allows researchers to evaluate the quality of a diverse range of models in milliseconds by querying the pre-computed dataset. Expand
Efficient Neural Architecture Search via Parameter Sharing
TLDR
Efficient Neural Architecture Search is a fast and inexpensive approach for automatic model design that establishes a new state-of-the-art among all methods without post-training processing and delivers strong empirical performances using much fewer GPU-hours. Expand
Evaluating the Search Phase of Neural Architecture Search
TLDR
This paper finds that on average, the state-of-the-art NAS algorithms perform similarly to the random policy; the widely-used weight sharing strategy degrades the ranking of the NAS candidates to the point of not reflecting their true performance, thus reducing the effectiveness of the search process. Expand
TAPAS: Train-less Accuracy Predictor for Architecture Search
TLDR
A new deep neural network accuracy predictor, that estimates in fractions of a second classification performance for unseen input datasets, without training is proposed, which achieves a performance which exceeds 100 networks per second on a single GPU, thus creating the opportunity to perform large-scale architecture search within a few minutes. Expand
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
TLDR
ProxylessNAS is presented, which can directly learn the architectures for large-scale target tasks and target hardware platforms and apply ProxylessNAS to specialize neural architectures for hardware with direct hardware metrics (e.g. latency) and provide insights for efficient CNN architecture design. Expand
DSNAS: Direct Neural Architecture Search Without Parameter Retraining
  • Shou-Yong Hu, Sirui Xie, +4 authors Dahua Lin
  • Computer Science, Mathematics
  • 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2020
TLDR
DSNAS is proposed, an efficient differentiable NAS framework that simultaneously optimizes architecture and parameters with a low-biased Monte Carlo estimate and successfully discovers networks with comparable accuracy on ImageNet. Expand
NAS-Bench-102: Extending the Scope of Reproducible Neural Architecture Search
Accelerating Neural Architecture Search using Performance Prediction
TLDR
Standard frequentist regression models can predict the final performance of partially trained model configurations using features based on network architectures, hyperparameters, and time-series validation performance data and an early stopping method is proposed, which obtains a speedup of a factor up to 6x in both hyperparameter optimization and meta-modeling. Expand
Surrogate-Assisted Evolutionary Deep Learning Using an End-to-End Random Forest-Based Performance Predictor
TLDR
An end-to-end offline performance predictor based on the random forest is proposed to accelerate the fitness evaluation in EDL and not only significantly speeds up the fitness evaluations but also achieves the best prediction among the peer performance predictors. Expand
NSGANetV2: Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Search
TLDR
An efficient NAS algorithm for generating task-specific models that are competitive under multiple competing objectives that comprises of two surrogates, one at the architecture level to improve sample efficiency and one through a supernet, to improve gradient descent training efficiency. Expand
...
1
2
3
4
5
...