Demystifying differentiable programming: shift/reset the penultimate backpropagator

@article{Wang2019DemystifyingDP,
  title={Demystifying differentiable programming: shift/reset the penultimate backpropagator},
  author={Fei Wang and Xilun Wu and Gr{\'e}gory M. Essertel and James M. Decker and Tiark Rompf},
  journal={Proceedings of the ACM on Programming Languages},
  year={2019},
  volume={3},
  pages={1 - 31}
}
  • Fei Wang, Xilun Wu, +2 authors Tiark Rompf
  • Published 2019
  • Mathematics, Computer Science
  • Proceedings of the ACM on Programming Languages
  • Deep learning has seen tremendous success over the past decade in computer vision, machine translation, and gameplay. This success rests crucially on gradient-descent optimization and the ability to “learn” parameters of a neural network by backpropagating observed errors. However, neural network architectures are growing increasingly sophisticated and diverse, which motivates an emerging quest for even more general forms of differentiable programming, where arbitrary parameterized computations… CONTINUE READING
    The Differentiable Curry
    Deep learning for pedestrians: backpropagation in CNNs
    • L. Boué
    • Computer Science, Mathematics
    • 2018
    A Differentiable Programming System to Bridge Machine Learning and Scientific Computing
    • 28
    • PDF
    AutoGraph: Imperative-style Coding with Graph-based Performance
    • 17
    • PDF
    Differentiating a Tensor Language
    Relay: A High-Level Compiler for Deep Learning.
    • 2
    • Highly Influenced
    Flare & Lantern: Efficiently Swapping Horses Midstream
    • 2
    • PDF

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 14 REFERENCES
    Automatic differentiation in machine learning: a survey
    • 470
    • Highly Influential
    • PDF
    The simple essence of automatic differentiation
    • 29
    • Highly Influential
    • PDF
    Learning representations by back-propagating errors
    • 16,006
    • Highly Influential
    • PDF
    TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
    • 7,512
    • Highly Influential
    • PDF
    Deep Learning with Dynamic Computation Graphs
    • 91
    • Highly Influential
    • PDF
    Finding Structure in Time
    • 8,412
    • Highly Influential
    • PDF
    Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
    • 1,857
    • Highly Influential
    • PDF
    The essence of compiling with continuations
    • 524
    • Highly Influential
    • PDF
    Handwritten Digit Recognition with a Back-Propagation Network
    • 2,257
    • Highly Influential
    • PDF
    An Operational Foundation for Delimited Continuations in the CPS Hierarchy
    • 87
    • Highly Influential
    • PDF