Learning to Control Fast-Weight Memories: An Alternative to Dynamic Recurrent Networks

@article{Schmidhuber1992LearningTC,
  title={Learning to Control Fast-Weight Memories: An Alternative to Dynamic Recurrent Networks},
  author={J. Schmidhuber},
  journal={Neural Computation},
  year={1992},
  volume={4},
  pages={131-139}
}
Previous algorithms for supervised sequence learning are based on dynamic recurrent networks. This paper describes an alternative class of gradient-based systems consisting of two feedforward nets that learn to deal with temporal sequences using fast weights: The first net learns to produce context-dependent weight changes for the second net whose weights may vary very quickly. The method offers the potential for STM storage efficiency: A single weight (instead of a full-fledged unit) may be… Expand
179 Citations
GATED FAST WEIGHTS FOR ASSOCIATIVE RETRIEVAL
  • Highly Influenced
Using Fast Weights to Attend to the Recent Past
  • 128
  • PDF
Continual Learning in Recurrent Neural Networks
  • PDF
Learning Unambiguous Reduced Sequence Descriptions
  • 44
  • PDF
Continual learning with hypernetworks
  • 47
  • PDF
Learning Complex, Extended Sequences Using the Principle of History Compression
  • 378
Continual Learning in Recurrent Neural Networks with Hypernetworks
  • 3
  • PDF
Sparse Meta Networks for Sequential Adaptation and its Application to Adaptive Language Modelling
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 22 REFERENCES
Learning State Space Trajectories in Recurrent Neural Networks
  • 710
  • PDF
A Local Learning Algorithm for Dynamic Feedforward and Recurrent Networks
  • 97
  • PDF
Learning Algorithms for Networks with Internal and External Feedback
  • 6
  • PDF
A Fixed Size Storage O(n3) Time Complexity Learning Algorithm for Fully Recurrent Continually Running Networks
  • 130
  • PDF
Generalization of backpropagation with application to a recurrent gas market model
  • 806
  • PDF
Learning to generate subgoals for action sequences
  • J. Schmidhuber
  • Computer Science
  • IJCNN-91-Seattle International Joint Conference on Neural Networks
  • 1991
  • 8
  • PDF
Dynamische neuronale Netze und das fundamentale raumzeitliche Lernproblem
  • 33
Learning internal representations by error propagation
  • 18,479
  • PDF
...
1
2
3
...