Corpus ID: 108300988

What do you learn from context? Probing for sentence structure in contextualized word representations

@article{Tenney2019WhatDY,
  title={What do you learn from context? Probing for sentence structure in contextualized word representations},
  author={Ian Tenney and Patrick Xia and B. Chen and Alex Wang and Adam Poliak and R. Thomas McCoy and Najoung Kim and Benjamin Van Durme and Samuel R. Bowman and Dipanjan Das and Ellie Pavlick},
  journal={ArXiv},
  year={2019},
  volume={abs/1905.06316}
}

Ian Tenney, Patrick Xia, +8 authors Ellie Pavlick

Published 2019

Computer Science

ArXiv

Contextualized representation models such as ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2018) have recently achieved state-of-the-art results on a diverse array of downstream NLP tasks. [...] Key Method We probe word-level contextual representations from four recent models and investigate how they encode sentence structure across a range of syntactic, semantic, local, and long-range phenomena. We find that existing models trained on language modeling and translation produce strong representations for…Expand Abstract

View PDF on arXiv

Save to Library

Create Alert

Cite

Launch Research Feed