Experimental Analysis of the Real-time Recurrent Learning Algorithm

  title={Experimental Analysis of the Real-time Recurrent Learning Algorithm},
  author={Ronald J. Williams and David Zipser},
  journal={Connection Science},
Abstract The real-time recurrent learning algorithm is a gradient-following learning algorithm for completely recurrent networks running in continually sampled time. Here we use a series of simulation experiments to investigate the power and properties of this algorithm. In the recurrent networks studied here, any unit can be connected to any other, and any unit can receive external input. These networks run continually in the sense that they sample their inputs on every update cycle, and any… 

Figures and Tables from this paper

A Comparison of Selected Training Algorithms for Recurrent Neural Networks

This work focuses on comparison of the selected and proposed training algorithms for RNNs and to evaluate the performance of the algorithms in the daily stock price forecasting in terms of efficiency index and computational time.

13 Gradient-Based Learning Algorithms for Recurrent Networks and Thei r Computational Complexity

Algorithms for training recurrent networks to perform temporal supervised learning tasks, in which the specification of desired behavior is in the form of specific examples of input and desired output trajectories are considered.

Convergence of gradient method for a fully recurrent neural network

This paper uses an off-line gradient training algorithm for the fully recurrent neural network, and proves some convergence results of deterministic nature, including the monotonicity of the error function in the iteration.

Recurrent learning algorithms for designing optimal controllers of continuous systems

A recurrent learning algorithm for designing the controllers of continuous dynamical systems in optimal control problems, in the form of unfolded recurrent neural nets embedded with physical laws from classical control techniques, is proposed.

A conjugate gradient learning algorithm for recurrent neural networks

A learning algorithm for Recurrent Radial Basis Function Networks

  • M. Mak
  • Computer Science
    Neural Processing Letters
  • 2006
A Recurrent Radial Basis Function network that can be applied to temporal pattern classifications and predictions and can approximate the filter more accurate than the Continually Running Fully Recurrent networks trained by the Real-Time Recurrent Learning algorithm.

Modelling Temporal Series Through Synaptic Delay-based Neural Networks

The capacity of the neural network and training algorithm in the modeling of time series is tested in the prediction of future values of chaotic time series using iterative multistep prediction.



A Learning Algorithm for Continually Running Fully Recurrent Neural Networks

The exact form of a gradient-following learning algorithm for completely recurrent networks running in continually sampled time is derived and used as the basis for practical algorithms for temporal

Learning State Space Trajectories in Recurrent Neural Networks

A procedure for finding E/wij, where E is an error functional of the temporal trajectory of the states of a continuous recurrent network and wij are the weights of that network, which seems particularly suited for temporally continuous domains.

Dynamics and architecture for neural computation

Learning Subsequential Structure in Simple Recurrent Networks

A network architecture introduced by Elman (1988) for predicting successive elements of a sequence using the pattern of activation over a set of hidden units to be illustrated with cluster analyses performed at different points during training.

A Dynamical Approach to Temporal Pattern Processing

This work proposes an architecture in which time serves as its own representation, and temporal context is encoded in the state of the nodes, and contrasts this with the approach of replicating portions of the architecture to represent time.

The theory of computer science: A programming approach

A meta computer science approach to the science of computation based on McCarthy's pioneering studies and evidence in support of Turing's thesis.

Adaptive Signal Processing

  • S. Alexander
  • Computer Science
    Texts and Monographs in Computer Science
  • 1986