Corpus ID: 3067546

FractalNet: Ultra-Deep Neural Networks without Residuals

@article{Larsson2017FractalNetUN,
  title={FractalNet: Ultra-Deep Neural Networks without Residuals},
  author={Gustav Larsson and M. Maire and Gregory Shakhnarovich},
  journal={ArXiv},
  year={2017},
  volume={abs/1605.07648}
}
  • Gustav Larsson, M. Maire, Gregory Shakhnarovich
  • Published 2017
  • Computer Science
  • ArXiv
  • We introduce a design strategy for neural network macro-architecture based on self-similarity. Repeated application of a simple expansion rule generates deep networks whose structural layouts are precisely truncated fractals. These networks contain interacting subpaths of different lengths, but do not include any pass-through or residual connections; every internal signal is transformed by a filter and nonlinearity before being seen by subsequent layers. In experiments, fractal networks match… CONTINUE READING
    464 Citations
    SoFAr: Shortcut-based Fractal Architectures for Binary Convolutional Neural Networks
    • PDF
    Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations
    • 179
    • Highly Influenced
    • PDF
    Slim, Sparse, and Shortcut Networks
    • 3
    • PDF
    Correlated Noise in Deep Convolutional Neural Networks
    Continuous-in-Depth Neural Networks
    • 1
    • PDF
    Beyond Dropout: Feature Map Distortion to Regularize Deep Neural Networks
    • Yehui Tang, Yunhe Wang, +4 authors C. Xu
    • Computer Science, Mathematics
    • AAAI
    • 2020
    • 5
    • PDF
    PolyNet: A Pursuit of Structural Diversity in Very Deep Networks
    • 139
    • Highly Influenced
    • PDF
    Sequentially Aggregated Convolutional Networks
    • PDF
    Sparsely Aggregated Convolutional Networks
    • 29
    • PDF

    References

    SHOWING 1-10 OF 52 REFERENCES
    Network In Network
    • 3,222
    • PDF
    Wide Residual Networks
    • 2,655
    • PDF
    Understanding the difficulty of training deep feedforward neural networks
    • 9,218
    • PDF
    Densely Connected Convolutional Networks
    • 11,396
    • PDF
    Deep Networks with Stochastic Depth
    • 910
    • Highly Influential
    • PDF
    Identity Mappings in Deep Residual Networks
    • 4,287
    • PDF
    Deep Residual Learning for Image Recognition
    • 57,619
    • Highly Influential
    • PDF
    Recurrent convolutional neural network for object recognition
    • M. Liang, Xiaolin Hu
    • Computer Science
    • 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
    • 2015
    • 610
    • PDF