Unitary Evolution Recurrent Neural Networks

@inproceedings{Arjovsky2016UnitaryER,
  title={Unitary Evolution Recurrent Neural Networks},
  author={Mart{\'i}n Arjovsky and Amar Shah and Yoshua Bengio},
  booktitle={ICML},
  year={2016}
}
Recurrent neural networks (RNNs) are notoriously difficult to train. When the eigenvalues of the hidden to hidden weight matrix deviate from absolute value 1, optimization becomes difficult due to the well studied issue of vanishing and exploding gradients, especially when trying to learn long-term dependencies. To circumvent this problem, we propose a new architecture that learns a unitary weight matrix, with eigenvalues of absolute value exactly 1. The challenge we address is that of… CONTINUE READING
Highly Influential
This paper has highly influenced 22 other papers. REVIEW HIGHLY INFLUENTIAL CITATIONS
Highly Cited
This paper has 133 citations. REVIEW CITATIONS
94 Citations
23 References
Similar Papers

Citations

Publications citing this paper.

133 Citations

0204060802015201620172018
Citations per Year
Semantic Scholar estimates that this publication has 133 citations based on the available data.

See our FAQ for additional information.

Similar Papers

Loading similar papers…