Subword Language Modeling with Neural Networks

  title={Subword Language Modeling with Neural Networks},
  author={Tomas Mikolov and Ilya Sutskever and Anoop Deoras and Hai-Son Le and Stefan Kombrink and Jan Cernock{\'y}},
We explore the performance of several types of language mode ls on the word-level and the character-level language modelin g tasks. This includes two recently proposed recurrent neural netwo rk architectures, a feedforward neural network model, a maximum ent ropy model and the usual smoothed n-gram models. We then propose a simple technique for learning sub-word level units from th e data, and show that it combines advantages of both character and wo rdlevel models. Finally, we show that neural… CONTINUE READING
Highly Influential
This paper has highly influenced 13 other papers. REVIEW HIGHLY INFLUENTIAL CITATIONS
Highly Cited
This paper has 163 citations. REVIEW CITATIONS


Publications citing this paper.
Showing 1-10 of 109 extracted citations

Fast, Lean, and Accurate: Modeling Password Guessability Using Neural Networks

USENIX Security Symposium • 2016
View 4 Excerpts
Highly Influenced

Earning S Yntax and L Exicon

Y M Shen, Zhi Lin, C. L. Huang, aaron. courville
View 4 Excerpts
Highly Influenced

164 Citations

Citations per Year
Semantic Scholar estimates that this publication has 164 citations based on the available data.

See our FAQ for additional information.


Publications referenced by this paper.
Showing 1-10 of 22 references

SRILM - an extensible language modeling toolkit

View 3 Excerpts
Highly Influenced

Out-Of-Vocabulary Words For Robust Speech Recognition

Issam Bazzi, Modelling
Ph.D. thesis, • 2002
View 3 Excerpts
Highly Influenced

Extensions of recurrent neural network language model

2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) • 2011
View 1 Excerpt

Similar Papers

Loading similar papers…