Scaling recurrent neural network language models

@article{Williams2015ScalingRN,
  title={Scaling recurrent neural network language models},
  author={Will Williams and Niranjani Prasad and David Mrva and Tom Ash and Tony Robinson},
  journal={2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  year={2015},
  pages={5391-5395}
}
This paper investigates the scaling properties of Recurrent Neural Network Language Models (RNNLMs). We discuss how to train very large RNNs on GPUs and address the questions of how RNNLMs scale with respect to model size, training-set size, computational costs and memory. Our analysis shows that despite being more costly to train, RNNLMs obtain much lower perplexities on standard benchmarks than n-gram models. We train the largest known RNNs and present relative word error rates gains of 18… CONTINUE READING
Highly Cited
This paper has 57 citations. REVIEW CITATIONS
Recent Discussions
This paper has been referenced on Twitter 25 times over the past 90 days. VIEW TWEETS
36 Citations
23 References
Similar Papers

Citations

Publications citing this paper.
Showing 1-10 of 36 extracted citations

58 Citations

010203020142015201620172018
Citations per Year
Semantic Scholar estimates that this publication has 58 citations based on the available data.

See our FAQ for additional information.

References

Publications referenced by this paper.
Showing 1-10 of 23 references

Similar Papers

Loading similar papers…