Corpus ID: 4944472

NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations

@inproceedings{Ciccone2018NAISNetSD,
  title={NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations},
  author={Marco Ciccone and Marco Gallieri and J. Masci and Christian Osendorfer and F. Gomez},
  booktitle={NeurIPS},
  year={2018}
}
  • Marco Ciccone, Marco Gallieri, +2 authors F. Gomez
  • Published in NeurIPS 2018
  • Computer Science, Mathematics
  • This paper introduces "Non-Autonomous Input-Output Stable Network" (NAIS-Net), a very deep architecture where each stacked processing block is derived from a time-invariant non-autonomous dynamical system. Non-autonomy is implemented by skip connections from the block input to each of the unrolled processing stages and allows stability to be enforced so that blocks can be unrolled adaptively to a pattern-dependent processing depth. We prove that the network is globally asymptotically stable so… CONTINUE READING
    Invertible Residual Networks
    • 144
    • Highly Influenced
    • PDF
    Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network
    • 108
    • PDF
    ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs
    • 38
    • PDF
    IMEXnet: A Forward Stable Deep Neural Network
    • 13
    • PDF
    Neural Controlled Differential Equations for Irregular Time Series
    • 14
    • PDF
    ANODEV2: A Coupled Neural ODE Evolution Framework
    • 13
    • PDF
    SNODE: Spectral Discretization of Neural ODEs for System Identification
    • 9
    • PDF
    ANODEV2: A Coupled Neural ODE Framework
    • 7
    • PDF
    Continuous-time Models for Stochastic Optimization Algorithms
    • 7
    • PDF

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 61 REFERENCES
    ImageNet classification with deep convolutional neural networks
    • 52,359
    • PDF
    Deep Residual Learning for Image Recognition
    • 50,397
    • Highly Influential
    • PDF
    Long Short-Term Memory
    • 30,975
    • Highly Influential
    • PDF
    Densely Connected Convolutional Networks
    • 9,725
    • PDF
    Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
    • 8,022
    • PDF
    Intriguing properties of neural networks
    • 5,184
    • PDF
    Sequence to Sequence Learning with Neural Networks
    • 10,529
    • PDF
    Learning Multiple Layers of Features from Tiny Images
    • 9,081
    • Highly Influential
    • PDF
    Matrix analysis
    • 27,486
    • PDF