• Publications
  • Influence
BERT Rediscovers the Classical NLP Pipeline
Pre-trained text encoders have rapidly advanced the state of the art on many NLP tasks. We focus on one such model, BERT, and aim to quantify where linguistic information is captured within theExpand
  • 217
  • 22
  • PDF
What do you learn from context? Probing for sentence structure in contextualized word representations
Contextualized representation models such as ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2018) have recently achieved state-of-the-art results on a diverse array of downstream NLP tasks.Expand
  • 203
  • 18
  • PDF
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling
Natural language understanding has recently seen a surge of progress with the use of sentence encoders like ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2019) which are pretrained on variantsExpand
  • 30
  • 6
  • PDF
WikiAtomicEdits: A Multilingual Corpus of Wikipedia Edits for Modeling Language and Discourse
We release a corpus of 43 million atomic edits across 8 languages. These edits are mined from Wikipedia edit history and consist of instances in which a human editor has inserted a single contiguousExpand
  • 10
  • 3
  • PDF
Ultrafast X-ray Auger probing of photoexcited molecular dynamics.
Molecules can efficiently and selectively convert light energy into other degrees of freedom. Disentangling the underlying ultrafast motion of electrons and nuclei of the photoexcited moleculeExpand
  • 91
  • 2
  • PDF
Ultrafast isomerization initiated by X-ray core ionization.
Rapid proton migration is a key process in hydrocarbon photochemistry. Charge migration and subsequent proton motion can mitigate radiation damage when heavier atoms absorb X-rays. If rapid enough,Expand
  • 65
  • 1
Probing What Different NLP Tasks Teach Machines about Function Word Comprehension
We introduce a set of nine challenge tasks that test for the understanding of function words. These tasks are created by structurally mutating sentences from existing datasets to target theExpand
  • 29
  • 1
  • PDF
Looking for ELMo's friends: Sentence-Level Pretraining Beyond Language Modeling
Work on the problem of contextualized word representation---the development of reusable neural network components for sentence understanding---has recently seen a surge of progress centered on theExpand
  • 23
  • 1
  • PDF
Semicrystalline Dihydroxyacetone Copolymers Derived from Glycerol
The ring-opening polymerization of glycerol-derived six-membered cyclic dimethylacetal dihydroxyacetone carbonate (MeO2DHAC) have been studied both in solution and bulk conditions with organicExpand
  • 22