Corpus ID: 221802416

Searching for Low-Bit Weights in Quantized Neural Networks

  title={Searching for Low-Bit Weights in Quantized Neural Networks},
  author={Z. Yang and Yunhe Wang and Kai Han and Chunjing Xu and Chao Xu and Dacheng Tao and C. Xu},
  • Z. Yang, Yunhe Wang, +4 authors C. Xu
  • Published 2020
  • Computer Science
  • ArXiv
  • Quantized neural networks with low-bit weights and activations are attractive for developing AI accelerators. However, the quantization functions used in most conventional quantization methods are non-differentiable, which increases the optimization difficulty of quantized networks. Compared with full-precision parameters (i.e., 32-bit floating numbers), low-bit values are selected from a much smaller set. For example, there are only 16 possibilities in 4-bit space. Thus, we present to regard… CONTINUE READING
    6 Citations

    Tables from this paper

    Kernel Based Progressive Distillation for Adder Neural Networks
    • 2
    • PDF
    Training Stronger Baselines for Learning to Optimize
    • PDF
    GhostSR: Learning Ghost Features for Efficient Image Super-Resolution
    • Ying Nie, Kai Han, +4 authors Yunhe Wang
    • Engineering, Computer Science
    • 2021
    • PDF
    A Survey on Visual Transformer
    • 1
    • PDF
    SCOP: Scientific Control for Reliable Neural Network Pruning
    • 1
    • PDF


    SYQ: Learning Symmetric Quantization for Efficient Deep Neural Networks
    • 73
    • PDF
    Trained Ternary Quantization
    • 607
    • PDF
    Binarized Neural Networks for Resource-Efficient Hashing with Minimizing Quantization Loss
    • 2
    • PDF
    BinaryConnect: Training Deep Neural Networks with binary weights during propagations
    • 1,615
    • Highly Influential
    • PDF
    Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks
    • 67
    • PDF
    LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
    • 206
    • PDF
    Deep Learning with Low Precision by Half-Wave Gaussian Quantization
    • 253
    • PDF
    Proximal Mean-Field for Neural Network Quantization
    • 7
    • PDF
    XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
    • 1,626
    • PDF