Corpus ID: 1101453

Recurrent Highway Networks

@article{Zilly2017RecurrentHN,
  title={Recurrent Highway Networks},
  author={Julian G. Zilly and R. Srivastava and J. Koutn{\'i}k and J. Schmidhuber},
  journal={ArXiv},
  year={2017},
  volume={abs/1607.03474}
}

Julian G. Zilly, R. Srivastava, +1 author J. Schmidhuber

Published 2017

Computer Science, Mathematics

ArXiv

Many sequential processing tasks require complex nonlinear transition functions from one step to the next. However, recurrent neural networks with 'deep' transition functions remain difficult to train, even when using Long Short-Term Memory (LSTM) networks. We introduce a novel theoretical analysis of recurrent networks based on Gersgorin's circle theorem that illuminates several modeling and optimization issues and improves our understanding of the LSTM cell. Based on this analysis we propose… CONTINUE READING

View PDF on arXiv

Save to Library

Create Alert

Cite

Launch Research Feed