Corpus ID: 11649458

Improved multitask learning through synaptic intelligence

@article{Zenke2017ImprovedML,
  title={Improved multitask learning through synaptic intelligence},
  author={Friedemann Zenke and Ben Poole and S. Ganguli},
  journal={ArXiv},
  year={2017},
  volume={abs/1703.04200}
}
Deep learning has led to remarkable advances when applied to problems where the data distribution does not change over the course of learning. In stark contrast, biological neural networks continually adapt to changing domains, and solve a diversity of tasks simultaneously. Furthermore, synapses in biological neurons are not simply real-valued scalars, but possess complex molecular machinery enabling non-trivial learning dynamics. In this study, we take a first step toward bringing this… Expand
30 Citations
Overcoming catastrophic forgetting with hard attention to the task
  • 202
  • Highly Influenced
  • PDF
Memory Aware Synapses: Learning what (not) to forget
  • 286
  • Highly Influenced
  • PDF
Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning
  • 53
  • PDF
Generative replay with feedback connections as a general strategy for continual learning
  • 87
  • Highly Influenced
  • PDF
Efficient Continual Learning in Neural Networks with Embedding Regularization
  • 5
  • Highly Influenced
  • PDF
Modular Continual Learning in a Unified Visual Environment
  • PDF
Continual Learning in Neural Networks
  • 6
  • Highly Influenced
  • PDF
Task-Free Continual Learning
  • 74
  • PDF
Selfless Sequential Learning
  • 42
  • Highly Influenced
  • PDF
Gradient Episodic Memory for Continual Learning
  • 580
  • PDF
...
1
2
3
...

References

SHOWING 1-10 OF 29 REFERENCES
Overcoming catastrophic forgetting in neural networks
  • 1,597
  • Highly Influential
  • PDF
Progressive Neural Networks
  • 538
  • PDF
Less-forgetting Learning in Deep Neural Networks
  • 77
  • PDF
Learning without Forgetting
  • Zhizhong Li, Derek Hoiem
  • Computer Science, Mathematics
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • 2018
  • 938
  • PDF
How transferable are features in deep neural networks?
  • 4,500
  • PDF
Computational principles of synaptic memory consolidation
  • 104
Learning Multiple Layers of Features from Tiny Images
  • 10,930
  • PDF
A memory frontier for complex synapses
  • 43
  • PDF
Revisiting Natural Gradient for Deep Networks
  • 221
  • PDF
...
1
2
3
...