Another Look at the Data Sparsity Problem

@inproceedings{Allison2006AnotherLA,
  title={Another Look at the Data Sparsity Problem},
  author={Ben Allison and David Guthrie and Louise Guthrie},
  booktitle={TSD},
  year={2006}
}

• Ben Allison, David Guthrie, Louise Guthrie
• Published in TSD 2006
• DOI:10.1007/11846406_41

Performance on a statistical language processing task relies upon accurate information being found in a corpus. However, it is known (and this paper will confirm) that many perfectly valid word sequences do not appear in training corpora. The percentage of n-grams in a test document which are seen in a training corpus is defined as n-gram coverage, and work in the speech processing community [7] has shown that there is a correlation between n-gram coverage and word error rate (WER) on a speech… CONTINUE READING

Save to Library

Create Alert

Cite

View via Publisher