Corpus ID: 14499411

Architectural Complexity Measures of Recurrent Neural Networks

@article{Zhang2016ArchitecturalCM,
  title={Architectural Complexity Measures of Recurrent Neural Networks},
  author={Saizheng Zhang and Yuhuai Wu and Tong Che and Zhouhan Lin and Roland Memisevic and Ruslan Salakhutdinov and Yoshua Bengio},
  journal={ArXiv},
  year={2016},
  volume={abs/1602.08210}
}
  • Saizheng Zhang, Yuhuai Wu, +4 authors Yoshua Bengio
  • Published 2016
  • Mathematics, Computer Science
  • ArXiv
  • In this paper, we systematically analyze the connecting architectures of recurrent neural networks (RNNs). Our main contribution is twofold: first, we present a rigorous graph-theoretic framework describing the connecting architectures of RNNs in general. Second, we propose three architecture complexity measures of RNNs: (a) the recurrent depth, which captures the RNN's over-time nonlinear complexity, (b) the feedforward depth, which captures the local input-output nonlinearity (similar to the… CONTINUE READING

    Figures, Tables, and Topics from this paper.

    Citations

    Publications citing this paper.
    SHOWING 1-10 OF 93 CITATIONS

    FILTER CITATIONS BY YEAR

    2016
    2020

    CITATION STATISTICS

    • 19 Highly Influenced Citations

    • Averaged 20 Citations per year from 2018 through 2020