Corpus ID: 3656432

From Hashing to CNNs: Training BinaryWeight Networks via Hashing

@inproceedings{Hu2018FromHT,
  title={From Hashing to CNNs: Training BinaryWeight Networks via Hashing},
  author={Qinghao Hu and Peisong Wang and Jian Cheng},
  booktitle={AAAI},
  year={2018}
}
Deep convolutional neural networks (CNNs) have shown appealing performance on various computer vision tasks in recent years. This motivates people to deploy CNNs to realworld applications. However, most of state-of-art CNNs require large memory and computational resources, which hinders the deployment on mobile devices. Recent studies show that low-bit weight representation can reduce much storage and memory demand, and also can achieve efficient network inference. To achieve this goal, we… Expand
Accurate and Compact Convolutional Neural Networks with Trained Binarization
TLDR
An improved training approach towards compact binary CNNs with higher accuracy is proposed including tight approximation for derivative of discontinuous binarization function and $L_2$ regularization acting on weight scaling factors. Expand
Training Low Bitwidth Model with Weight Normalization for Convolutional Neural Networks
TLDR
A method to train convolutional neural networks with low bit width by performing weight normalization, which enables the low bitwidth network to achieve a good trade-off between range and precision. Expand
Learning Compression from Limited Unlabeled Data
TLDR
By employing the re-estimated statistics in batch normalization, this paper significantly improves the accuracy of compressed CNNs and is comparable to label-based methods. Expand
Deep Polarized Network for Supervised Learning of Accurate Binary Hashing Codes
This paper proposes a novel deep polarized network (DPN) for learning to hash, in which each channel in the network outputs is pushed far away from zero by employing a differentiable bit-wiseExpand
Binary Neural Networks: A Survey
TLDR
A comprehensive survey of algorithms proposed for binary neural networks, mainly categorized into the native solutions directly conducting binarization, and the optimized ones using techniques like minimizing the quantization error, improving the network loss function, and reducing the gradient error are presented. Expand
Unsupervised Network Quantization via Fixed-Point Factorization
TLDR
This article proposes an efficient framework, namely, fixed-point factorized network (FFN), to turn all weights into ternary values, i.e., {−1, 0, 1}, and highlights that the proposed FFN framework can achieve negligible degradation even without any supervised retraining on the labeled data. Expand
Group binary weight networks
In recent years, quantizing the weights of a deep neural network draws increasing attention in the area of network compression. An efficient and popular way to quantize the weight parameters is toExpand
Collaborative Learning for Extremely Low Bit Asymmetric Hashing
TLDR
An asymmetric hashing framework with two variants of multi-head embedding structures is derived, termed as Multi-head Asymmetric Hashing (MAH), leading to great efficiency of training and querying and significantly surpasses the performance of the 48-bit codes by the state-of-the-arts in image retrieval tasks. Expand
Quantized Neural Networks via {-1, +1} Encoding Decomposition and Acceleration
TLDR
A novel encoding scheme using {−1,+1} to decompose quantized neural networks (QNNs) into multi-branch binary networks, which can be efficiently implemented by bitwise operations (i.e., xnor and bitcount) to achieve model compression, computational acceleration, and resource saving. Expand
Structured Probabilistic Pruning for Convolutional Neural Network Acceleration
TLDR
A novel progressive parameter pruning method, named Structured Probabilistic Pruning (SPP), which effectively prunes weights of convolutional layers in a probabilistic manner and can be directly applied to accelerate multi-branch CNN networks, such as ResNet, without specific adaptations. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 37 REFERENCES
Scalable and Sustainable Deep Learning via Randomized Hashing
TLDR
This work presents a novel hashing-based technique to drastically reduce the amount of computation needed to train and test neural networks, and demonstrates the scalability and sustainability (energy efficiency) of the proposed algorithm via rigorous experimental evaluations on several datasets. Expand
Compressing Deep Convolutional Networks using Vector Quantization
TLDR
This paper is able to achieve 16-24 times compression of the network with only 1% loss of classification accuracy using the state-of-the-art CNN, and finds in terms of compressing the most storage demanding dense connected layers, vector quantization methods have a clear gain over existing matrix factorization methods. Expand
Compressing Neural Networks with the Hashing Trick
TLDR
This work presents a novel network architecture, HashedNets, that exploits inherent redundancy in neural networks to achieve drastic reductions in model sizes, and demonstrates on several benchmark data sets that HashingNets shrink the storage requirements of neural networks substantially while mostly preserving generalization performance. Expand
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
TLDR
The Binary-Weight-Network version of AlexNet is compared with recent network binarization methods, BinaryConnect and BinaryNets, and outperform these methods by large margins on ImageNet, more than \(16\,\%\) in top-1 accuracy. Expand
Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications
TLDR
A simple and effective scheme to compress the entire CNN, called one-shot whole network compression, which addresses the important implementation level issue on 1?1 convolution, which is a key operation of inception module of GoogLeNet as well as CNNs compressed by the proposed scheme. Expand
Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
TLDR
Extensive experiments on the ImageNet classification task using almost all known deep CNN architectures including AlexNet, VGG-16, GoogleNet and ResNets well testify the efficacy of the proposed INQ, showing that at 5-bit quantization, models have improved accuracy than the 32-bit floating-point references. Expand
Quantized Convolutional Neural Networks for Mobile Devices
TLDR
This paper proposes an efficient framework, namely Quantized CNN, to simultaneously speed-up the computation and reduce the storage and memory overhead of CNN models. Expand
Supervised Discrete Hashing
TLDR
This work proposes a new supervised hashing framework, where the learning objective is to generate the optimal binary hash codes for linear classification, and introduces an auxiliary variable to reformulate the objective such that it can be solved substantially efficiently by employing a regularization algorithm. Expand
Fixed Point Quantization of Deep Convolutional Networks
TLDR
This paper proposes a quantizer design for fixed point implementation of DCNs, formulate and solve an optimization problem to identify optimal fixed point bit-width allocation across DCN layers, and demonstrates that fine-tuning can further enhance the accuracy of fixed point DCNs beyond that of the original floating point model. Expand
Accelerating Convolutional Neural Networks for Mobile Applications
TLDR
An efficient and effective approach is proposed to accelerate the test-phase computation of CNNs based on low-rank and group sparse tensor decomposition, which achieves significant reduction in computational complexity, at the cost of negligible loss in accuracy. Expand
...
1
2
3
4
...