• Publications
  • Influence
Annotation Artifacts in Natural Language Inference Data
TLDR
It is shown that a simple text categorization model can correctly classify the hypothesis alone in about 67% of SNLI and 53% of MultiNLI, and that specific linguistic phenomena such as negation and vagueness are highly correlated with certain inference classes. Expand
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks
TLDR
It is consistently found that multi-phase adaptive pretraining offers large gains in task performance, and it is shown that adapting to a task corpus augmented using simple data selection strategies is an effective alternative, especially when resources for domain-adaptive pretraining might be unavailable. Expand
Show Your Work: Improved Reporting of Experimental Results
TLDR
It is demonstrated that test-set performance scores alone are insufficient for drawing accurate conclusions about which model performs best, and a novel technique is presented: expected validation performance of the best-found model as a function of computation budget. Expand
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
TLDR
It is found that pretrained LMs can degenerate into toxic text even from seemingly innocuous prompts, and empirically assess several controllable generation methods find that while data- or compute-intensive methods are more effective at steering away from toxicity than simpler solutions, no current method is failsafe against neural toxic degeneration. Expand
Variational Pretraining for Semi-supervised Text Classification
TLDR
VAMPIRE is introduced, a lightweight pretraining framework for effective text classification when data and computing resources are limited and it is found that fine-tuning to in-domain data is crucial to achieving decent performance from contextual embeddings when working with limited supervision. Expand
Detoxifying Language Models Risks Marginalizing Minority Voices
TLDR
It is found that detoxification makes LMs more brittle to distribution shift, especially on language used by marginalized groups, and the tension between the controllability and distributional robustness of LMs is highlighted. Expand
Analysis of Graph Invariants in Functional Neocortical Circuitry Reveals Generalized Features Common to Three Areas of Sensory Cortex
TLDR
This study monitored spontaneous circuit dynamics in large, densely sampled neuronal populations within slices of mouse primary auditory, somatosensory, and visual cortex and revealed organizational features that generalized across functionally distinct cortical regions. Expand
Emergent coordination underlying learning to reach to grasp with a brain-machine interface.
TLDR
This work focuses on training nonhuman primates with chronic amputations to use neurons that are not reach or grasp related to control a robotic arm to reach to grasp through the use of operant conditioning, mimicking early development. Expand
Robust Question Answering System
Pretrained models like BERT achieves good performance when we fine-tune it to resourceful QA tasks like SQuAD. However, when we apply the model to out-ofdomain QA tasks with different question andExpand
...
1
2
...