Learning to Control Fast-Weight Memories: An Alternative to Dynamic Recurrent Networks

  title={Learning to Control Fast-Weight Memories: An Alternative to Dynamic Recurrent Networks},
  author={J{\"u}rgen Schmidhuber},
  journal={Neural Computation},
  • J. Schmidhuber
  • Published 3 January 1992
  • Computer Science
  • Neural Computation
Previous algorithms for supervised sequence learning are based on dynamic recurrent networks. This paper describes an alternative class of gradient-based systems consisting of two feedforward nets that learn to deal with temporal sequences using fast weights: The first net learns to produce context-dependent weight changes for the second net whose weights may vary very quickly. The method offers the potential for STM storage efficiency: A single weight (instead of a full-fledged unit) may be… 


This work improves previous end-to-end differentiable neural networks with fast weight memories by trained on a complex sequence to sequence variation of the Associative Retrieval Problem with roughly 70 times more temporal memory than similar-sized standard recurrent NNs.

Using Fast Weights to Attend to the Recent Past

These ``fast weights'' can be used to store temporary memories of the recent past and they provide a neurally plausible way of implementing the type of attention to the past that has recently proven helpful in sequence-to-sequence models.

Continual learning in recurrent neural networks

This study shows that established CL methods can be successfully ported to the recurrent case, and that a recent regularization approach based on hypernetworks outperforms weight-importance methods, thus emerging as a promising candidate for CL in RNNs.

Learning Unambiguous Reduced Sequence Descriptions

Experiments show that systems based on these principles can require less computation per time step and many fewer training sequences than conventional training algorithms for recurrent nets.

Continual learning with hypernetworks

Insight is provided into the structure of low-dimensional task embedding spaces (the input space of the hypernetwork) and it is shown that task-conditioned hypernetworks demonstrate transfer learning.

Sparse Meta Networks for Sequential Adaptation and its Application to Adaptive Language Modelling

This work augments a deep neural network with a layer-specific fast-weight memory, generated sparsely at each time step and accumulated incrementally through time providing a useful inductive bias for online continual adaptation.

Metalearned Neural Memory

This work augments recurrent neural networks with an external memory mechanism that builds upon recent progress in metalearning and achieves strong performance on a variety of learning problems, from supervised question answering to reinforcement learning.

Fast & Slow Learning: Incorporating Synthetic Gradients in Neural Memory Controllers

This work proposes to decouple the learning process of the NMN controllers to allow them to achieve flexible, rapid adaptation in the presence of new information, which is highly beneficial for meta-learning tasks where the memory controllers must quickly grasp abstract concepts in the target domain, and adapt stored knowledge.

Neural networks with late-phase weights

This work shows that augmenting standard models with late-phase weights improves generalization in established benchmarks such as CIFAR-10/100, ImageNet and enwik8 and investigates a family of low-dimensional late- phase weight models which interact multiplicatively with the remaining parameters.

Meta-Reinforcement Learning with Self-Modifying Networks

The resulting model, MetODS (for Meta-Optimized Dynamical Synapses) is a broadly applicable meta-reinforcement learning system able to learn efficient and powerful control rules in the agent policy space, comparing favorably with previous meta- rein reinforcement learning approaches.



Learning State Space Trajectories in Recurrent Neural Networks

A procedure for finding E/wij, where E is an error functional of the temporal trajectory of the states of a continuous recurrent network and wij are the weights of that network, which seems particularly suited for temporally continuous domains.

A Local Learning Algorithm for Dynamic Feedforward and Recurrent Networks

This paper proposes a parallel on-line learning algorithms which performs local computations only, yet still is designed to deal with hidden units and with units whose past activations are ‘hidden in time’.

Experimental Analysis of the Real-time Recurrent Learning Algorithm

A series of simulation experiments are used to investigate the power and properties of the real-time recurrent learning algorithm, a gradient-following learning algorithm for completely recurrent networks running in continually sampled time.

Learning Algorithms for Networks with Internal and External Feedback

A Fixed Size Storage O(n3) Time Complexity Learning Algorithm for Fully Recurrent Continually Running Networks

A method suited for on-line learning that computes exactly the same gradient and requires fixed-size storage of the same order but has an average time complexity per time step of O(n3).

Learning Algorithms for Networks with Internal and External Feedback

This paper gives an overview of some novel algorithms for reinforcement learning in non-stationary possibly reactive environments and critisizes methods based on system identiication and adaptive critics, and describes an adaptive subgoal generator.

Learning to generate subgoals for action sequences

  • J. Schmidhuber
  • Business
    IJCNN-91-Seattle International Joint Conference on Neural Networks
  • 1991
The author discusses a system which solves at least one problem associated with compositional learning with the help of 'time-bridging' adaptive models that predict the effects of the system's subprograms.

Task modularization by network modulation

  • Proceedings of Neuro - Nimes ‘ 90 ,
  • 1990