Skip to search formSkip to main contentSkip to account menu

Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks

It is consistently found that multi-phase adaptive pretraining offers large gains in task performance, and it is shown that adapting to a task corpus augmented using simple data selection strategies is an effective alternative, especially when resources for domain-adaptive pretraining might be unavailable.
View on ACL (opens in a new tab)

Annotation Artifacts in Natural Language Inference Data

It is shown that a simple text categorization model can correctly classify the hypothesis alone in about 67% of SNLI and 53% of MultiNLI, and that specific linguistic phenomena such as negation and vagueness are highly correlated with certain inference classes.
View on ACL (opens in a new tab)

Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics

The results indicate that a shift in focus from quantity to quality of data could lead to robust models and improved out-of-distribution generalization, and a model-based tool to characterize and diagnose datasets.
View on ACL (opens in a new tab)

DyNet: The Dynamic Neural Network Toolkit

DyNet is a toolkit for implementing neural network models based on dynamic declaration of network structure that has an optimized C++ backend and lightweight graph representation and is designed to allow users to implement their models in a way that is idiomatic in their preferred programming language.
View PDF on arXiv (opens in a new tab)

A Dependency Parser for Tweets

A new dependency parser for English tweets, TWEEBOPARSER, which builds on several contributions: new syntactic annotations for a corpus of tweets, with conventions informed by the domain; adaptations to a statistical parsing algorithm; and a new approach to exploiting out-of-domain Penn Treebank data.
View on ACL (opens in a new tab)

DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts

This work highlights the promise of tuning small LMs on text with (un)desirable attributes for efficient decoding-time steering and applies DExperts to language detoxification and sentiment-controlled generation, where it outperform existing controllable generation methods on both automatic and human evaluations.
View on ACL (opens in a new tab)

Frame-Semantic Parsing with Softmax-Margin Segmental RNNs and a Syntactic Scaffold

A new, efficient frame-semantic parser that labels semantic arguments to FrameNet predicates, built using an extension to the segmental RNN that emphasizes recall, achieves competitive performance without any calls to a syntactic parser.
View PDF on arXiv (opens in a new tab)

Adversarial Filters of Dataset Biases

This work presents extensive supporting evidence that AFLite is broadly applicable for reduction of measurable dataset biases, and that models trained on the filtered datasets yield better generalization to out-of-distribution tasks.
View PDF on arXiv (opens in a new tab)

Transfer Learning in Natural Language Processing

An overview of modern transfer learning methods in NLP, how models are pre-trained, what information the representations they learn capture, and review examples and case studies on how these models can be integrated and adapted in downstream NLP tasks are presented.
View on ACL (opens in a new tab)

The Right Tool for the Job: Matching Model and Instance Complexities

This work proposes a modification to contextual representation fine-tuning which allows for an early (and fast) “exit” from neural network calculations for simple instances, and late (and accurate) exit for hard instances during inference.
View on ACL (opens in a new tab)
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy (opens in a new tab), Terms of Service (opens in a new tab), and Dataset License (opens in a new tab)