Binarizing MobileNet via Evolution-Based Searching

@article{Phan2020BinarizingMV,
  title={Binarizing MobileNet via Evolution-Based Searching},
  author={Hai T. Phan and Zechun Liu and Dang The Huynh and Marios Savvides and Kwang-Ting Cheng and Zhiqiang Shen},
  journal={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020},
  pages={13417-13426}
}
Binary Neural Networks (BNNs), known to be one among the effectively compact network architectures, have achieved great outcomes in the visual tasks. Designing efficient binary architectures is not trivial due to the binary nature of the network. In this paper, we propose a use of evolutionary search to facilitate the construction and training scheme when binarizing MobileNet, a compact network with separable depth-wise convolution. Being inspired by one-shot architecture search frameworks, we… Expand
Bi-Real Net V2: Rethinking Non-linearity for 1-bit CNNs and Going Beyond
TLDR
This paper argues that the binarized convolution process owns an increasing linearity towards the target of minimizing such error, which hampers BNN's discriminative ability, and investigates and tunes proper non-linear modules to fix that contradiction, leading to a strong baseline which achieves state-of-the-art performance on the large-scale ImageNet dataset. Expand
Training Multi-Bit Quantized and Binarized Networks with a Learnable Symmetric Quantizer
TLDR
This work examines binarization difficulties in a quantization framework and finds that all it need to enable the binary training are a symmetric quantizer, good initialization, and careful hyperparameter selection, which leads to substantial improvements in multi-bit quantization. Expand
A Comprehensive Survey on Hardware-Aware Neural Architecture Search
TLDR
This survey provides a detailed review of existing HW-NAS research and categorizes them according to four key dimensions: the search space, the search strategy, the acceleration technique, and the hardware cost estimation strategies. Expand
Dynamic Network Quantization for Efficient Video Inference
TLDR
A dynamic network quantization framework, that selects optimal precision for each frame conditioned on the input for efficient video recognition is proposed, that provides significant savings in computation and memory usage while outperforming the existing state-of-the-art methods. Expand
A comprehensive review of Binary Neural Network
  • Chunyu Yuan, S. Agaian
  • Computer Science
  • ArXiv
  • 2021
TLDR
This paper focuses exclusively on 1-bit activations and weights networks, as opposed to previous surveys in which low-bit works are mixed in, and discusses potential directions and future research opportunities for the latest BNN algorithms and techniques. Expand
Distribution-sensitive Information Retention for Accurate Binary Neural Network
  • Haotong Qin, Xiangguo Zhang, Ruihao Gong, Yifu Ding, Yi Xu, Xianglong Liu
  • Computer Science
  • ArXiv
  • 2021
TLDR
The DIR-Net investigates both forward and backward processes of BNNs from the unified information perspective, thereby provides new insight into the mechanism of network binarization. Expand
Once Quantization-Aware Training: High Performance Extremely Low-bit Architecture Search
TLDR
An architecture perspective is taken to investigate the potential of high-performance QNN, and a bit-inheritance scheme is introduced to transfer the quantized models to the lower bit, which further reduces the time cost and meanwhile improves the quantization accuracy. Expand
BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer
TLDR
The approach is the first to seamlessly extend one-shot weight-sharing NAS supernet to support subnets with arbitrary ultra-low bitwidth mixed-precision quantization policies without retraining and opens up new possibilities in joint hardware-aware neural architecture search and quantization. Expand
Once Quantized for All: Progressively Searching for Quantized Efficient Models
TLDR
Once Quantized for All (OQA) is presented, a novel framework that searches for quantized efficient models and deploys their quantized weights at the same time without additional post-process without affecting the quality of the search-retrain schema. Expand
How Do Adam and Training Strategies Help BNNs Optimization?
TLDR
This paper derives a simple training scheme, building on existing Adam-based optimization, which achieves 70.5% top-1 accuracy on the ImageNet dataset using the same architecture as the state-of-the-art ReActNet (Liu et al., 2020) while achieving 1.1% higher accuracy. Expand
...
1
2
...

References

SHOWING 1-10 OF 58 REFERENCES
MoBiNet: A Mobile Binary Network for Image Classification
TLDR
A novel neural network architecture, namely MoBi- Net - Mobile Binary Network in which skip connections are manipulated to prevent information loss and vanishing gradient, thus facilitate the training process and results in an effectively small model while keeping the accuracy comparable to existing ones. Expand
Towards Accurate Binary Convolutional Neural Network
TLDR
The implementation of the resulting binary CNN, denoted as ABC-Net, is shown to achieve much closer performance to its full-precision counterpart, and even reach the comparable prediction accuracy on ImageNet and forest trail datasets, given adequate binary weight bases and activations. Expand
Training Competitive Binary Neural Networks from Scratch
TLDR
This work is the first to successfully adopt a network architecture with dense connections for binary networks, which lets the performance of binary neural networks improve the state-of-the-art even further. Expand
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
TLDR
The Binary-Weight-Network version of AlexNet is compared with recent network binarization methods, BinaryConnect and BinaryNets, and outperform these methods by large margins on ImageNet, more than \(16\,\%\) in top-1 accuracy. Expand
Binarized Neural Networks
TLDR
A binary matrix multiplication GPU kernel is written with which it is possible to run the MNIST BNN 7 times faster than with an unoptimized GPU kernel, without suffering any loss in classification accuracy. Expand
Learning Efficient Convolutional Networks through Network Slimming
TLDR
The approach is called network slimming, which takes wide and large networks as input models, but during training insignificant channels are automatically identified and pruned afterwards, yielding thin and compact models with comparable accuracy. Expand
Genetic CNN
  • Lingxi Xie, A. Yuille
  • Computer Science
  • 2017 IEEE International Conference on Computer Vision (ICCV)
  • 2017
TLDR
The core idea is to propose an encoding method to represent each network structure in a fixed-length binary string to efficiently explore this large search space. Expand
Single Path One-Shot Neural Architecture Search with Uniform Sampling
TLDR
A Single Path One-Shot model is proposed to construct a simplified supernet, where all architectures are single paths so that weight co-adaption problem is alleviated. Expand
Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm
TLDR
A novel model, dubbed Bi-Real net, which connects the real activations (after the 1-bit convolution and/or BatchNorm layer, before the sign function) to activations of the consecutive block, through an identity shortcut is proposed, which achieves up to 10% higher top-1 accuracy with more memory saving and lower computational cost. Expand
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
TLDR
An extremely computation-efficient CNN architecture named ShuffleNet is introduced, which is designed specially for mobile devices with very limited computing power (e.g., 10-150 MFLOPs), to greatly reduce computation cost while maintaining accuracy. Expand
...
1
2
3
4
5
...