Extensions of recurrent neural network language model

@article{Mikolov2011ExtensionsOR,
  title={Extensions of recurrent neural network language model},
  author={Tomas Mikolov and Stefan Kombrink and L. Burget and J. Cernock{\'y} and S. Khudanpur},
  journal={2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  year={2011},
  pages={5528-5531}
}
We present several modifications of the original recurrent neural network language model (RNN LM).While this model has been shown to significantly outperform many competitive language modeling techniques in terms of accuracy, the remaining problem is the computational complexity. [...] Key Method Next, we show importance of using a backpropagation through time algorithm. An empirical comparison with feedforward networks is also provided. In the end, we discuss possibilities how to reduce the amount of…Expand
1,271 Citations
Multiple parallel hidden layers and other improvements to recurrent neural network language modeling
  • D. Caseiro, A. Ljolje
  • Computer Science
  • 2013 IEEE International Conference on Acoustics, Speech and Signal Processing
  • 2013
  • 3
  • Highly Influenced
LSTM Neural Networks for Language Modeling
  • 1,264
  • PDF
Comparison of feedforward and recurrent neural network language models
  • 113
  • PDF
RECURRENT NEURAL NETWORKS
  • PDF
Learning Longer Memory in Recurrent Neural Networks
  • 195
  • PDF
Multiscale recurrent neural network based language model
  • 11
  • Highly Influenced
  • PDF
Recurrent Neural Networks and Related Models
  • 1
Higher Order Recurrent Neural Networks
  • 43
  • PDF
Unsupervised Adaptation of Recurrent Neural Network Language Models
  • 26
  • PDF
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 25 REFERENCES
Recurrent neural network based language model
  • 4,322
  • PDF
Hierarchical Probabilistic Neural Network Language Model
  • 870
  • Highly Influential
  • PDF
Adaptive Importance Sampling to Accelerate Training of a Neural Probabilistic Language Model
  • 169
  • PDF
Training Neural Network Language Models on Very Large Corpora
  • 125
  • PDF
Learning long-term dependencies with gradient descent is difficult
  • 5,108
  • PDF
A guide to recurrent neural networks and backpropagation
  • 212
  • PDF
Classes for fast maximum entropy training
  • Joshua Goodman
  • Computer Science
  • 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221)
  • 2001
  • 219
  • Highly Influential
  • PDF
Exact training of a neural syntactic language model
  • A. Emami, F. Jelinek
  • Computer Science
  • 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing
  • 2004
  • 26
  • PDF
...
1
2
3
...