Architectural Complexity Measures of Recurrent Neural Networks

  title={Architectural Complexity Measures of Recurrent Neural Networks},
  author={Saizheng Zhang and Yuhuai Wu and Tong Che and Zhouhan Lin and Roland Memisevic and Ruslan Salakhutdinov and Yoshua Bengio},
In this paper, we systematically analyse the connecting architectures of recurrent neural networks (RNNs). Our main contribution is twofold: first, we present a rigorous graphtheoretic framework describing the connecting architectures of RNNs in general. Second, we propose three architecture complexity measures of RNNs: (a) the recurrent depth, which captures the RNN’s over-time nonlinear complexity, (b) the feedforward depth, which captures the local input-output nonlinearity (similar to the… CONTINUE READING
Highly Influential
This paper has highly influenced 13 other papers. REVIEW HIGHLY INFLUENTIAL CITATIONS
Highly Cited
This paper has 66 citations. REVIEW CITATIONS
Related Discussions
This paper has been referenced on Twitter 62 times. VIEW TWEETS


Publications citing this paper.
Showing 1-10 of 48 extracted citations

Recurrent Neural Networks with Flexible Gates Using Kernel Activation Functions

2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP) • 2018
View 12 Excerpts
Highly Influenced

67 Citations

Citations per Year
Semantic Scholar estimates that this publication has 67 citations based on the available data.

See our FAQ for additional information.


Publications referenced by this paper.
Showing 1-10 of 31 references

A Clockwork RNN

View 5 Excerpts
Highly Influenced

On the difficulty of training recurrent neural networks

Pascanu, Razvan, +3 authors Yoshua
In Proceedings of The 30th International Conference on Machine Learning, • 2013
View 5 Excerpts
Highly Influenced

LSTM: A Search Space Odyssey

IEEE Transactions on Neural Networks and Learning Systems • 2017
View 2 Excerpts

Keras. GitHub repository: https://

Chollet, François

Skip-Thought Vectors

NIPS • 2015
View 1 Excerpt

Similar Papers

Loading similar papers…