Deep contextualized word representations

@inproceedings{Peters2018DeepCW,
  title={Deep contextualized word representations},
  author={Matthew E. Peters and Mark Neumann and Mohit Iyyer and Matt Gardner and Christopher Clark and Kenton Lee and Luke S. Zettlemoyer},
  booktitle={NAACL-HLT},
  year={2018}
}

• Matthew E. Peters, Mark Neumann, +4 authors Luke S. Zettlemoyer
• Published in NAACL-HLT 2018
• DOI:10.18653/v1/N18-1202

We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). Our word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pretrained on a large text corpus. We show that these representations can be easily added to existing models and significantly improve the state… CONTINUE READING

View on ArXiv

Save to Library

Create Alert

Cite