Deep Contextualized Word Representations

@article{Peters2018DeepCW,
  title={Deep Contextualized Word Representations},
  author={Matthew E. Peters and Mark Neumann and Mohit Iyyer and Matt Gardner and Christopher Clark and Kenton Lee and Luke S. Zettlemoyer},
  journal={ArXiv},
  year={2018},
  volume={abs/1802.05365}
}
We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). Our word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a large text corpus. We show that these representations can be easily added to existing models and significantly improve the state… CONTINUE READING

Citations

Publications citing this paper.
SHOWING 1-10 OF 1,578 CITATIONS

Constituency Parsing with a Self-Attentive Encoder

VIEW 5 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

A Lightweight Recurrent Network for Sequence Modeling

VIEW 5 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

A Lightweight Recurrent Network for Sequence Modeling

VIEW 5 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

A Logic-Driven Framework for Consistency of Neural Models

VIEW 12 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

A Unified Neural Coherence Model

VIEW 9 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

An ELMo-inspired approach to SemDeep-5's Word-in-Context task

Alan Ansell, Felipe Bravo-Marquez, Bernhard Pfahringer
  • SemDeep@IJCAI
  • 2019
VIEW 11 EXCERPTS
HIGHLY INFLUENCED

ConSSED at SemEval-2019 Task 3: Configurable Semantic and Sentiment Emotion Detector

  • SemEval@NAACL-HLT
  • 2019
VIEW 5 EXCERPTS
CITES METHODS
HIGHLY INFLUENCED

FILTER CITATIONS BY YEAR

2017
2019

CITATION STATISTICS

  • 418 Highly Influenced Citations

  • Averaged 524 Citations per year from 2017 through 2019

  • 260% Increase in citations per year in 2019 over 2018

References

Publications referenced by this paper.
SHOWING 1-10 OF 60 REFERENCES