Corpus ID: 12775860

Memory-Efficient Backpropagation Through Time

@inproceedings{Gruslys2016MemoryEfficientBT,
  title={Memory-Efficient Backpropagation Through Time},
  author={A. Gruslys and R. Munos and Ivo Danihelka and Marc Lanctot and A. Graves},
  booktitle={NIPS},
  year={2016}
}

• A. Gruslys, R. Munos, +2 authors A. Graves
• Published in NIPS 2016
• Computer Science

We propose a novel approach to reduce memory consumption of the backpropagation through time (BPTT) algorithm when training recurrent neural networks (RNNs). Our approach uses dynamic programming to balance a trade-off between caching of intermediate results and recomputation. The algorithm is capable of tightly fitting within almost any user-set memory budget while finding an optimal execution policy minimizing the computational cost. Computational devices have limited memory capacity and… Expand

View PDF on arXiv

Save to Library

Create Alert

Cite

Launch Research Feed