XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
@inproceedings{Rastegari2016XNORNetIC, title={XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks}, author={Mohammad Rastegari and Vicente Ordonez and Joseph Redmon and Ali Farhadi}, booktitle={European Conference on Computer Vision}, year={2016} }
We propose two efficient approximations to standard convolutional neural networks: Binary-Weight-Networks and XNOR-Networks. [] Key Result This results in 58\(\times \) faster convolutional operations (in terms of number of the high precision operations) and 32\(\times \) memory savings. XNOR-Nets offer the possibility of running state-of-the-art networks on CPUs (rather than GPUs) in real-time. Our binary networks are simple, accurate, efficient, and work on challenging visual tasks. We evaluate our…
3,086 Citations
Structured Binary Neural Networks for Image Recognition
- Computer ScienceInternational Journal of Computer Vision
- 2022
This paper proposes a “group decomposition” strategy, termed GroupNet, which divides a network into desired groups, and extends the GroupNet for accurate semantic segmentation by embedding the rich context into the binary structure.
FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations
- Computer ScienceFPGA
- 2021
The proposed FracBNN exploits fractional activations to substantially improve the accuracy of BNNs, and implements the entire optimized network architecture on an embedded FPGA (Xilinx Ultra96 v2) with the ability of real-time image classification.
Binary neural networks
- Computer ScienceHardware Architectures for Deep Learning
- 2020
A survey on the state-of-the-art researches on the design and hardware implementation of the BNN models is conducted.
Towards Lossless Binary Convolutional Neural Networks Using Piecewise Approximation
- Computer ScienceECAI
- 2020
A Piecewise Approximation (PA) scheme for multiple binary CNNs which lessens accuracy loss by approximating full precision weights and activations efficiently and maintains parallelism of bitwise operations to guarantee efficiency.
Capacity Limits of Fully Binary CNN
- Computer Science2020 IEEE 50th International Symposium on Multiple-Valued Logic (ISMVL)
- 2020
The aim of the paper is to provide the further exploration of the binarization effect on the model capacity and show that while for MNIST the accuracy is very close to the full precision counterpart, for the more complex dataset, CIFAR-10, thebinarization and the representational power of CNNs is strongly affected.
Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagation
- Computer ScienceAAAI
- 2019
This paper introduces projection convolutional neural networks with a discrete back propagation via projection (DBPP) to improve the performance of binarized neural networks (BNNs) and learns a set of diverse quantized kernels that compress the full-precision kernels in a more efficient way than those proposed previously.
SATB-Nets: Training Deep Neural Networks with Segmented Asymmetric Ternary and Binary Weights
- Computer ScienceICONIP
- 2018
SATB-Nets, a method which trains CNNs with segmented asymmetric ternary weights for convolutional layers and binary weights for the fully-connected layers, outperforms full precision model VGG16 on CIFAR-10 and ImageNet datasets.
Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networks
- Computer Science2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- 2021
Sub-bit Neural Networks are introduced, a new type of binary quantization design tailored to compress and accelerate BNNs that is inspired by an empirical observation, showing that binary kernels learnt at convolutional layers of a BNN model are likely to be distributed over kernel subsets.
A Convolutional Result Sharing Approach for Binarized Neural Network Inference
- Computer Science2020 Design, Automation & Test in Europe Conference & Exhibition (DATE)
- 2020
The binary-weight-binary-input binarized neural network (BNN) allows a much more efficient way to implement convolutional neural networks (CNNs) on mobile platforms and the number of operations in convolution layers of BNNs can be reduced effectively.
Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detection
- Computer ScienceJournal of Computational Mathematics
- 2019
LBW-Net is presented, an efficient optimization based method for quantization and training of the low bit-width convolutional neural networks (CNNs) that is nearly lossless in the object detection tasks, and can even do better in some real world visual scenes.
References
SHOWING 1-10 OF 44 REFERENCES
ImageNet classification with deep convolutional neural networks
- Computer ScienceCommun. ACM
- 2012
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.
Fixed point optimization of deep convolutional neural networks for object recognition
- Computer Science2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- 2015
The results indicate that quantization induces sparsity in the network which reduces the effective number of network parameters and improves generalization, and reduces the required memory storage by a factor of 1/10 and achieves better classification results than the high precision networks.
BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- Computer ScienceArXiv
- 2016
BinaryNet, a method which trains DNNs with binary weights and activations when computing parameters’ gradient is introduced, which drastically reduces memory usage and replaces most multiplications by 1-bit exclusive-not-or (XNOR) operations, which might have a big impact on both general-purpose and dedicated Deep Learning hardware.
BinaryConnect: Training Deep Neural Networks with binary weights during propagations
- Computer ScienceNIPS
- 2015
BinaryConnect is introduced, a method which consists in training a DNN with binary weights during the forward and backward propagations, while retaining precision of the stored weights in which gradients are accumulated, and near state-of-the-art results with BinaryConnect are obtained on the permutation-invariant MNIST, CIFAR-10 and SVHN.
Compressing Deep Convolutional Networks using Vector Quantization
- Computer ScienceArXiv
- 2014
This paper is able to achieve 16-24 times compression of the network with only 1% loss of classification accuracy using the state-of-the-art CNN, and finds in terms of compressing the most storage demanding dense connected layers, vector quantization methods have a clear gain over existing matrix factorization methods.
Bitwise Neural Networks
- Computer ScienceArXiv
- 2016
The proposed Bitwise Neural Network (BNN) is especially suitable for resource-constrained environments, since it replaces either floating or fixed-point arithmetic with significantly more efficient bitwise operations.
Deep Residual Learning for Image Recognition
- Computer Science2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- 2016
This work presents a residual learning framework to ease the training of networks that are substantially deeper than those used previously, and provides comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth.
Very Deep Convolutional Networks for Large-Scale Image Recognition
- Computer ScienceICLR
- 2015
This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
Speeding up Convolutional Neural Networks with Low Rank Expansions
- Computer ScienceBMVC
- 2014
Two simple schemes for drastically speeding up convolutional neural networks are presented, achieved by exploiting cross-channel or filter redundancy to construct a low rank basis of filters that are rank-1 in the spatial domain.
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Computer ScienceIEEE Transactions on Pattern Analysis and Machine Intelligence
- 2015
This work introduces a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals and further merge RPN and Fast R-CNN into a single network by sharing their convolutionAL features.