Hierarchical Distributed Representations for Statistical Language Modeling

  title={Hierarchical Distributed Representations for Statistical Language Modeling},
  author={John Blitzer and Kilian Q. Weinberger and Lawrence K. Saul and Fernando Pereira},
Statistical language models estimate the probability of a word occurring in a given context. The most common language models rely on a discrete enumeration of predictive contexts (e.g., n-grams) and consequently fail to capture and exploit statistical regularities across these contexts. In this paper, we show how to learn hierarchical, distributed representations of word contexts that maximize the predictive value of a statistical language model. The representations are initialized by… CONTINUE READING
Highly Cited
This paper has 24 citations. REVIEW CITATIONS

Similar Papers

Loading similar papers…