Deep contextualized word representations

  title={Deep contextualized word representations},
  author={Matthew E. Peters and Mark Neumann and Mohit Iyyer and Matt Gardner and Christopher Clark and Kenton Lee and Luke S. Zettlemoyer},
We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). Our word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pretrained on a large text corpus. We show that these representations can be easily added to existing models and significantly improve the state… CONTINUE READING
Highly Influential
This paper has highly influenced 83 other papers. REVIEW HIGHLY INFLUENTIAL CITATIONS
Highly Cited
This paper has 410 citations. REVIEW CITATIONS
This paper has been referenced on Twitter 446 times. VIEW TWEETS


Publications citing this paper.
Showing 1-10 of 295 citations

410 Citations

Citations per Year
Semantic Scholar estimates that this publication has 410 citations based on the available data.

See our FAQ for additional information.


Publications referenced by this paper.
Showing 1-10 of 61 references