Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflections

@inproceedings{Mhammedi2017EfficientOP,
  title={Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflections},
  author={Zakaria Mhammedi and Andrew D. Hellicar and Ashfaqur Rahman and James Bailey},
  booktitle={ICML},
  year={2017}
}
Recurrent Neural Networks (RNNs) have been successfully used in many applications. However, the problem of learning long-term dependencies in sequences using these networks is still a major challenge. Recent methods have been suggested to solve this problem by constraining the transition matrix to be unitary during training, which ensures that its norm is exactly equal to one. These methods either have limited expressiveness or scale poorly with the size of the network when compared with the… CONTINUE READING
Highly Cited
This paper has 23 citations. REVIEW CITATIONS
Recent Discussions
This paper has been referenced on Twitter 29 times over the past 90 days. VIEW TWEETS

7 Figures & Tables

Topics

Statistics

0102020172018
Citations per Year

Citation Velocity: 13

Averaging 13 citations per year over the last 2 years.

Learn more about how we calculate this metric in our FAQ.