# Fast training of recurrent networks based on the EM algorithm

@article{Ma1998FastTO, title={Fast training of recurrent networks based on the EM algorithm}, author={Sheng-Fei Ma and C. Ji}, journal={IEEE transactions on neural networks}, year={1998}, volume={9 1}, pages={ 11-26 } }

In this work, a probabilistic model is established for recurrent networks. The expectation-maximization (EM) algorithm is then applied to derive a new fast training algorithm for recurrent networks through mean-field approximation. This new algorithm converts training a complicated recurrent network into training an array of individual feedforward neurons. These neurons are then trained via a linear weighted regression algorithm. The training time has been improved by five to 15 times on… Expand

#### Figures, Tables, and Topics from this paper

#### 37 Citations

A unified approach on fast training of feedforward and recurrent networks using EM algorithm

- Computer Science
- IEEE Trans. Signal Process.
- 1998

A learning algorithm for recurrent neural networks and its application to nonlinear identification

- Computer Science
- Proceedings of the 1999 IEEE International Symposium on Computer Aided Control System Design (Cat. No.99TH8404)
- 1999

New results on recurrent network training: unifying the algorithms and accelerating convergence

- Computer Science, Medicine
- IEEE Trans. Neural Networks Learn. Syst.
- 2000

A new EM-based training algorithm for RBF networks

- Mathematics, Computer Science
- Neural Networks
- 2003

Improving Generalization Capabilities of Dynamic Neural Networks

- Computer Science, Medicine
- Neural Computation
- 2004

A new supervised learning algorithm for multilayered and interconnected neural networks

- Computer Science, Medicine
- IEEE Trans. Neural Networks Learn. Syst.
- 2000

Learning Long-Term Dependencies in Segmented Memory Recurrent Neural Networks

- Computer Science
- ISNN
- 2004

Accelerating the Convergence of EM-Based Training Algorithms for RBF Networks

- Computer Science
- IWANN
- 2001

On the improvement of the real time recurrent learning algorithm for recurrent neural networks

- Computer Science
- Neurocomputing
- 1999

Learning long-term dependencies in segmented-memory recurrent neural networks with backpropagation of error

- Computer Science
- Neurocomputing
- 2014

#### References

SHOWING 1-10 OF 44 REFERENCES

An Efficient EM-based Training Algorithm for Feedforward Neural Networks

- Computer Science, Medicine
- Neural Networks
- 1997

An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories

- Computer Science
- Neural Computation
- 1990

A Learning Algorithm for Continually Running Fully Recurrent Neural Networks

- Computer Science
- Neural Computation
- 1989

The problem of learning long-term dependencies in recurrent networks

- Computer Science
- IEEE International Conference on Neural Networks
- 1993

A statistical approach to learning and generalization in layered neural networks

- Computer Science
- COLT '89
- 1989

Hierarchical Mixtures of Experts and the EM Algorithm

- Mathematics
- Neural Computation
- 1994

A statistical approach to learning and generalization in layered neural networks

- Computer Science
- COLT 1989
- 1989