Long Short-Term Memory

@article{Hochreiter1997LongSM,
  title={Long Short-Term Memory},
  author={S. Hochreiter and J. Schmidhuber},
  journal={Neural Computation},
  year={1997},
  volume={9},
  pages={1735-1780}
}
Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. [...] Key Method Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space…Expand
On the importance of sluggish state memory for learning long term dependency
Language Modeling through Long-Term Memory Network
Learning Sparse Hidden States in Long Short-Term Memory
Learning Longer Memory in Recurrent Neural Networks
On Extended Long Short-term Memory and Dependent Bidirectional Recurrent Neural Network
Internal Memory Gate for Recurrent Neural Networks with Application to Spoken Language Understanding
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 69 REFERENCES
Learning long-term dependencies in NARX recurrent neural networks
Learning Unambiguous Reduced Sequence Descriptions
Induction of Multiscale Temporal Structure
Learning Sequential Structure with the Real-Time Recurrent Learning Algorithm
A Local Learning Algorithm for Dynamic Feedforward and Recurrent Networks
Continuous history compression
Learning long-term dependencies with gradient descent is difficult
Learning Complex, Extended Sequences Using the Principle of History Compression
Generalization of backpropagation with application to a recurrent gas market model
...
1
2
3
4
5
...