On efficient training of word classes and their application to recurrent neural network language models

@inproceedings{Botros2015OnET,
  title={On efficient training of word classes and their application to recurrent neural network language models},
  author={Rami Botros and Kazuki Irie and Martin Sundermeyer and Hermann Ney},
  booktitle={INTERSPEECH},
  year={2015}
}
In this paper, we investigated various word clustering methods, by studying two clustering algorithms: Brown clustering and exchange algorithm, and three objective functions derived from different class-based language models (CBLM): two-sided, predictive and conditional models. In particular, we focused on the implementation of the exchange algorithm with improved speed. In total, we compared six clustering methods in terms of runtime and perplexity (PP) of the CBLM on a French corpus, and show… CONTINUE READING

Citations

Publications citing this paper.
Showing 1-10 of 11 extracted citations

Scaling Up Word Clustering

HLT-NAACL Demos • 2016
View 5 Excerpts
Highly Influenced

Aalto system for the 2017 Arabic multi-genre broadcast challenge

2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) • 2017
View 1 Excerpt

Character-based units for unlimited vocabulary continuous speech recognition

2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) • 2017
View 1 Excerpt

Improving N-gram language modeling for code-switching speech recognition

2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) • 2017
View 1 Excerpt

Recurrent Neural Network based language modeling with controllable external Memory

2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) • 2017
View 1 Excerpt

References

Publications referenced by this paper.
Showing 1-10 of 22 references

Maximum-likelihood-schätzung von wortkategorien mit verfahren der kombinatorischen optimierung

F. J. Och
Studienarbeit, Friedrich-Alexander-Universität, Erlangen-Nürnberg, Germany, 1995. • 1995
View 4 Excerpts
Highly Influenced

Investigation of multilingual deep neural networks for spoken term detection

2013 IEEE Workshop on Automatic Speech Recognition and Understanding • 2013
View 1 Excerpt

Speed regularization and optimality in word classing

2013 IEEE International Conference on Acoustics, Speech and Signal Processing • 2013
View 1 Excerpt

Similar Papers

Loading similar papers…