Online Continual Learning under Extreme Memory Constraints

@article{Fini2020OnlineCL,
  title={Online Continual Learning under Extreme Memory Constraints},
  author={Enrico Fini and St{\'e}phane Lathuili{\`e}re and E. Sangineto and Moin Nabi and E. Ricci},
  journal={ArXiv},
  year={2020},
  volume={abs/2008.01510}
}
Continual Learning (CL) aims to develop agents emulating the human ability to sequentially learn new tasks while being able to retain knowledge obtained from past experiences. In this paper, we introduce the novel problem of Memory-Constrained Online Continual Learning (MC-OCL) which imposes strict constraints on the memory overhead that a possible algorithm can use to avoid catastrophic forgetting. As most, if not all, previous CL methods violate these constraints, we propose an algorithmic… Expand

Figures and Tables from this paper

Rainbow Memory: Continual Learning with a Memory of Diverse Samples
Layerwise Optimization by Gradient Decomposition for Continual Learning
Towards Continual, Online, Unsupervised Depth

References

SHOWING 1-10 OF 32 REFERENCES
Meta-Learning Representations for Continual Learning
Gradient Episodic Memory for Continual Learning
Online Continual Learning with Maximally Interfered Retrieval
Gradient based sample selection for online continual learning
Learning to Learn without Forgetting By Maximizing Transfer and Minimizing Interference
Task-Free Continual Learning
Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning
Continual Learning with Deep Generative Replay
Learning a Unified Classifier Incrementally via Rebalancing
Progressive Neural Networks
...
1
2
3
4
...