# Experimental Analysis of the Real-time Recurrent Learning Algorithm

@article{Williams1989ExperimentalAO, title={Experimental Analysis of the Real-time Recurrent Learning Algorithm}, author={Ronald J. Williams and David Zipser}, journal={Connection Science}, year={1989}, volume={1}, pages={87-111} }

Abstract The real-time recurrent learning algorithm is a gradient-following learning algorithm for completely recurrent networks running in continually sampled time. Here we use a series of simulation experiments to investigate the power and properties of this algorithm. In the recurrent networks studied here, any unit can be connected to any other, and any unit can receive external input. These networks run continually in the sense that they sample their inputs on every update cycle, and any…

## 342 Citations

### A method for improving the real-time recurrent learning algorithm

- Computer ScienceNeural Networks
- 1993

### A Comparison of Selected Training Algorithms for Recurrent Neural Networks

- Computer SciencePRICAI
- 2006

This work focuses on comparison of the selected and proposed training algorithms for RNNs and to evaluate the performance of the algorithms in the daily stock price forecasting in terms of efficiency index and computational time.

### 13 Gradient-Based Learning Algorithms for Recurrent Networks and Thei r Computational Complexity

- Computer Science
- 2008

Algorithms for training recurrent networks to perform temporal supervised learning tasks, in which the specification of desired behavior is in the form of specific examples of input and desired output trajectories are considered.

### Convergence of gradient method for a fully recurrent neural network

- Computer Science, MathematicsSoft Comput.
- 2010

This paper uses an off-line gradient training algorithm for the fully recurrent neural network, and proves some convergence results of deterministic nature, including the monotonicity of the error function in the iteration.

### Recurrent learning algorithms for designing optimal controllers of continuous systems

- Computer ScienceIEEE Trans. Syst. Man Cybern. Part A
- 2000

A recurrent learning algorithm for designing the controllers of continuous dynamical systems in optimal control problems, in the form of unfolded recurrent neural nets embedded with physical laws from classical control techniques, is proposed.

### A conjugate gradient learning algorithm for recurrent neural networks

- Computer ScienceNeurocomputing
- 1999

### A real-time recurrent learning network structure for data reconciliation

- Computer ScienceArtif. Intell. Eng.
- 1998

### A learning algorithm for Recurrent Radial Basis Function Networks

- Computer ScienceNeural Processing Letters
- 2006

A Recurrent Radial Basis Function network that can be applied to temporal pattern classifications and predictions and can approximate the filter more accurate than the Continually Running Fully Recurrent networks trained by the Real-Time Recurrent Learning algorithm.

### Modelling Temporal Series Through Synaptic Delay-based Neural Networks

- Computer ScienceNeural Computing & Applications
- 2003

The capacity of the neural network and training algorithm in the modeling of time series is tested in the prediction of future values of chaotic time series using iterative multistep prediction.

## References

SHOWING 1-10 OF 19 REFERENCES

### A Learning Algorithm for Continually Running Fully Recurrent Neural Networks

- Computer ScienceNeural Computation
- 1989

The exact form of a gradient-following learning algorithm for completely recurrent networks running in continually sampled time is derived and used as the basis for practical algorithms for temporal…

### Learning State Space Trajectories in Recurrent Neural Networks

- Computer ScienceNeural Computation
- 1989

A procedure for finding E/wij, where E is an error functional of the temporal trajectory of the states of a continuous recurrent network and wij are the weights of that network, which seems particularly suited for temporally continuous domains.

### Learning Subsequential Structure in Simple Recurrent Networks

- Computer ScienceNIPS
- 1988

A network architecture introduced by Elman (1988) for predicting successive elements of a sequence using the pattern of activation over a set of hidden units to be illustrated with cluster analyses performed at different points during training.

### A Dynamical Approach to Temporal Pattern Processing

- Computer ScienceNIPS
- 1987

This work proposes an architecture in which time serves as its own representation, and temporal context is encoded in the state of the nodes, and contrasts this with the approach of replicating portions of the architecture to represent time.

### The theory of computer science: A programming approach

- Computer Science
- 1977

A meta computer science approach to the science of computation based on McCarthy's pioneering studies and evidence in support of Turing's thesis.