Learning to Control Fast-Weight Memories: An Alternative to Dynamic Recurrent Networks

@article{Schmidhuber1992LearningTC,
  title={Learning to Control Fast-Weight Memories: An Alternative to Dynamic Recurrent Networks},
  author={J{\"u}rgen Schmidhuber},
  journal={Neural Computation},
  year={1992},
  volume={4},
  pages={131-139}
}
  • J. Schmidhuber
  • Published 3 January 1992
  • Computer Science
  • Neural Computation
Previous algorithms for supervised sequence learning are based on dynamic recurrent networks. This paper describes an alternative class of gradient-based systems consisting of two feedforward nets that learn to deal with temporal sequences using fast weights: The first net learns to produce context-dependent weight changes for the second net whose weights may vary very quickly. The method offers the potential for STM storage efficiency: A single weight (instead of a full-fledged unit) may be… 
GATED FAST WEIGHTS FOR ASSOCIATIVE RETRIEVAL
TLDR
This work improves previous end-to-end differentiable neural networks with fast weight memories by trained on a complex sequence to sequence variation of the Associative Retrieval Problem with roughly 70 times more temporal memory than similar-sized standard recurrent NNs.
Using Fast Weights to Attend to the Recent Past
TLDR
These ``fast weights'' can be used to store temporary memories of the recent past and they provide a neurally plausible way of implementing the type of attention to the past that has recently proven helpful in sequence-to-sequence models.
Continual learning in recurrent neural networks
TLDR
This study shows that established CL methods can be successfully ported to the recurrent case, and that a recent regularization approach based on hypernetworks outperforms weight-importance methods, thus emerging as a promising candidate for CL in RNNs.
Learning Associative Inference Using Fast Weight Memory
TLDR
This model is trained end-to-end by gradient descent and yields excellent performance on compositional language reasoning problems, meta-reinforcement-learning for POMDPs, and small-scale word-level language modelling.
Learning Unambiguous Reduced Sequence Descriptions
TLDR
Experiments show that systems based on these principles can require less computation per time step and many fewer training sequences than conventional training algorithms for recurrent nets.
Continual learning with hypernetworks
TLDR
Insight is provided into the structure of low-dimensional task embedding spaces (the input space of the hypernetwork) and it is shown that task-conditioned hypernetworks demonstrate transfer learning.
Learning Complex, Extended Sequences Using the Principle of History Compression
TLDR
A simple principle for reducing the descriptions of event sequences without loss of information is introduced and this insight leads to the construction of neural architectures that learn to divide and conquer by recursively decomposing sequences.
Continual Learning in Recurrent Neural Networks with Hypernetworks
TLDR
This work demonstrates that high working memory requirements, but not necessarily sequence length, lead to an increased need for stability at the cost of decreased performance on subsequent tasks, and employs a recent method based on hypernetworks to address catastrophic forgetting on sequential data.
Sparse Meta Networks for Sequential Adaptation and its Application to Adaptive Language Modelling
TLDR
This work augments a deep neural network with a layer-specific fast-weight memory, generated sparsely at each time step and accumulated incrementally through time providing a useful inductive bias for online continual adaptation.
Metalearned Neural Memory
TLDR
This work augments recurrent neural networks with an external memory mechanism that builds upon recent progress in metalearning and achieves strong performance on a variety of learning problems, from supervised question answering to reinforcement learning.
...
...

References

SHOWING 1-10 OF 22 REFERENCES
Learning State Space Trajectories in Recurrent Neural Networks
TLDR
A procedure for finding E/wij, where E is an error functional of the temporal trajectory of the states of a continuous recurrent network and wij are the weights of that network, which seems particularly suited for temporally continuous domains.
A Local Learning Algorithm for Dynamic Feedforward and Recurrent Networks
TLDR
This paper proposes a parallel on-line learning algorithms which performs local computations only, yet still is designed to deal with hidden units and with units whose past activations are ‘hidden in time’.
Experimental Analysis of the Real-time Recurrent Learning Algorithm
TLDR
A series of simulation experiments are used to investigate the power and properties of the real-time recurrent learning algorithm, a gradient-following learning algorithm for completely recurrent networks running in continually sampled time.
Learning Algorithms for Networks with Internal and External Feedback
A Fixed Size Storage O(n3) Time Complexity Learning Algorithm for Fully Recurrent Continually Running Networks
TLDR
A method suited for on-line learning that computes exactly the same gradient and requires fixed-size storage of the same order but has an average time complexity per time step of O(n3).
Learning Algorithms for Networks with Internal and External Feedback
TLDR
This paper gives an overview of some novel algorithms for reinforcement learning in non-stationary possibly reactive environments and critisizes methods based on system identiication and adaptive critics, and describes an adaptive subgoal generator.
Learning to generate subgoals for action sequences
  • J. Schmidhuber
  • Business
    IJCNN-91-Seattle International Joint Conference on Neural Networks
  • 1991
TLDR
The author discusses a system which solves at least one problem associated with compositional learning with the help of 'time-bridging' adaptive models that predict the effects of the system's subprograms.
Dynamische neuronale Netze und das fundamentale raumzeitliche Lernproblem
TLDR
The improvement of the invention comprises a deflector fixed rearwardly of the sensor to prevent excessive displacement by individual stalks which may be out-of-place and thereby preclude an excessive response of the automatic steering system because of occasional out- of-place stalks.
...
...