Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets

@article{PrezOrtiz2003KalmanFI,
  title={Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets},
  author={Juan Antonio P{\'e}rez-Ortiz and Felix A. Gers and Douglas Eck and J{\"u}rgen Schmidhuber},
  journal={Neural networks : the official journal of the International Neural Network Society},
  year={2003},
  volume={16 2},
  pages={241-50}
}
The long short-term memory (LSTM) network trained by gradient descent solves difficult problems which traditional recurrent neural networks in general cannot. We have recently observed that the decoupled extended Kalman filter training algorithm allows for even better performance, reducing significantly the number of training steps when compared to the original gradient descent training algorithm. In this paper we present a set of experiments which are unsolvable by classical recurrent networks… CONTINUE READING
Related Discussions
This paper has been referenced on Twitter 1 time. VIEW TWEETS

Citations

Publications citing this paper.
Showing 1-10 of 33 extracted citations

Efficient Online Learning Algorithms Based on LSTM Neural Networks

IEEE Transactions on Neural Networks and Learning Systems • 2018
View 10 Excerpts
Highly Influenced

Online Training of LSTM Networks in Distributed Systems for Variable Length Data Sequences

IEEE Transactions on Neural Networks and Learning Systems • 2018
View 3 Excerpts

References

Publications referenced by this paper.
Showing 1-10 of 23 references

A recurrent error propagation speech recognition system

A. J. Robinson, F. Fallside
Computer Speech and Language • 1991
View 7 Excerpts
Highly Influenced

Hill climbing in recurrent neural networks for learning the anbncn language

S. Chalup, A. Blair
Proceedings of the 6 th Conference on Neural Information Processing • 1999
View 1 Excerpt

Similar Papers

Loading similar papers…