• DOI:10.1016/j.csl.2019.101034
Corpus ID: 211080668

State gradients for analyzing memory in LSTM language models

@article{Verwimp2020StateGF,
  title={State gradients for analyzing memory in LSTM language models},
  author={Lyan Verwimp and H. V. hamme and P. Wambacq},
  journal={Comput. Speech Lang.},
  year={2020},
  volume={61},
  pages={101034}
}

• Lyan Verwimp, H. V. hamme, P. Wambacq
• Published 2020
• Computer Science
• Comput. Speech Lang.

Abstract Gradients can be used to train neural networks, but they can also be used to interpret them. We investigate how well the inputs of RNNs are remembered by their state by calculating ‘state gradients’, and applying SVD on the gradient matrix to reveal which directions in embedding space are remembered and to what extent. Our method can be applied to any RNN and reveals which properties in the embedding space influence the state space, without the need to know and label the properties… Expand

View via Publisher

lirias.kuleuven.be

Save to Library

Create Alert

Cite

Launch Research Feed