Corpus ID: 22458497

Learning to update Auto-associative Memory in Recurrent Neural Networks for Improving Sequence Memorization

@article{Zhang2017LearningTU,
  title={Learning to update Auto-associative Memory in Recurrent Neural Networks for Improving Sequence Memorization},
  author={Wei Zhang and Bowen Zhou},
  journal={ArXiv},
  year={2017},
  volume={abs/1709.06493}
}
Learning to remember long sequences remains a challenging task for recurrent neural networks. Register memory and attention mechanisms were both proposed to resolve the issue with either high computational cost to retain memory differentiability, or by discounting the RNN representation learning towards encoding shorter local contexts than encouraging long sequence encoding. Associative memory, which studies the compression of multiple patterns in a fixed size memory, were rarely considered in… Expand
8 Citations
Rotational Unit of Memory
  • 7
  • PDF
Rotational Unit of Memory: A Novel Representation Unit for RNNs with Scalable Applications
  • 9
  • Highly Influenced
  • PDF
Self-Attentive Associative Memory
  • 4
  • PDF
Fast Weight Long Short-Term Memory
  • Highly Influenced
  • PDF
Self-Assttentive Associative Memory
Hopfield Networks is All You Need
  • 20
  • PDF
MIDS: End-to-End Personalized Response Generation in Untrimmed Multi-Role Dialogue*
Learning Associative Inference Using Fast Weight Memory
  • 1
  • PDF

References

SHOWING 1-10 OF 44 REFERENCES
Dense Associative Memory for Pattern Recognition
  • 66
  • PDF
Structured Memory for Neural Turing Machines
  • 11
  • PDF
Long Short-Term Memory
  • 36,782
  • Highly Influential
  • PDF
Learning dynamic Boltzmann machines with spike-timing dependent plasticity
  • 14
  • PDF
Using Fast Weights to Attend to the Recent Past
  • 127
  • PDF
Neural Architecture Search with Reinforcement Learning
  • 2,395
  • PDF
Using fast weights to deblur old memories
  • 185
Recurrent Highway Networks
  • 318
  • Highly Influential
  • PDF
Recurrent Models of Visual Attention
  • 1,911
  • PDF
...
1
2
3
4
5
...