Corpus ID: 195601375

Low precision storage for deep learning

@article{Courbariaux2014LowPS,
  title={Low precision storage for deep learning},
  author={Matthieu Courbariaux and Yoshua Bengio and J. David},
  journal={arXiv: Learning},
  year={2014}
}
  • Matthieu Courbariaux, Yoshua Bengio, J. David
  • Published 2014
  • Computer Science
  • arXiv: Learning
  • We train a set of state of the art neural networks, the Maxout networks (Goodfellow et al., 2013a), on three benchmark datasets: the MNIST, CIFAR10 and SVHN, with three distinct storing formats: floating point, fixed point and dynamic fixed point. [...] Key Result For example, Maxout networks state-of-the-art results are nearly maintained with 10 bits for storing activations and gradients, and 12 bits for storing parameters.Expand Abstract
    40 Citations
    Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations
    • 58
    • PDF
    Overcoming Challenges in Fixed Point Training of Deep Convolutional Networks
    • 27
    • PDF
    Compressing Convolutional Neural Networks
    • 39
    • PDF
    Hardware-oriented Approximation of Convolutional Neural Networks
    • 233
    • PDF
    Trained Biased Number Representation for ReRAM-Based Neural Network Accelerators
    • 2
    • Highly Influenced
    Structured Transforms for Small-Footprint Deep Learning
    • 166
    • PDF
    Compressing Convolutional Neural Networks in the Frequency Domain
    • 60
    • PDF
    Compressing Neural Networks with the Hashing Trick
    • 807
    • Highly Influenced
    • PDF
    Deep Neural Network Approximation for Custom Hardware
    • 38
    • PDF
    Soft Threshold Ternary Networks
    • PDF

    References

    SHOWING 1-10 OF 27 REFERENCES
    DaDianNao: A Machine-Learning Supercomputer
    • Yunji Chen, T. Luo, +8 authors O. Temam
    • Computer Science
    • 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture
    • 2014
    • 883
    • PDF
    ImageNet classification with deep convolutional neural networks
    • 59,458
    • Highly Influential
    • PDF
    A highly scalable Restricted Boltzmann Machine FPGA implementation
    • 68
    • PDF
    DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
    • 1,005
    • PDF
    Learning Multiple Layers of Features from Tiny Images
    • 10,432
    • PDF
    Stochastic Pooling for Regularization of Deep Convolutional Neural Networks
    • 710
    • PDF
    Large Scale Distributed Deep Networks
    • 2,535
    • PDF
    Backpropagation without Multiplication
    • 25
    • PDF
    Maxout Networks
    • 1,610
    • PDF