N-gram distribution based language model adaptation

  title={N-gram distribution based language model adaptation},
  author={Jianfeng Gao and Mingjing Li and Kai-Fu Lee},
This paper presents two techniques for language model (LM) adaptation. The first aims to build a more general LM. We propose a distribution-based pruning of n-gram LMs, where we prune n-grams that are likely to be infrequent in a new document. Experimental results show that the distribution-based pruning method performed up to 9% (word perplexity reduction) better than conventional cutoff methods. Moreover, the pruning method results in a more general ngram backoff model, in spite of the domain… CONTINUE READING

From This Paper

Figures, tables, and topics from this paper.
3 Citations
11 References
Similar Papers

Similar Papers

Loading similar papers…