N. A. Smith

We propose a modification to contextual representation fine-tuning which, during inference, allows for an early (and fast) "exit" from neural network calculations for simple instances, and late (and accurate) exit for hard instances. Expand

We re-examine the trade-off and argue that transformer-based autoregressive models can be substantially sped up without loss in accuracy. Expand

We propose a new transformer stack, the sandwich transformer, and show that it improves perplexity on multiple word-level and character-level language modeling benchmarks, at no cost in parameters, memory, or training time. Expand

We present a novel training paradigm, Learn from EveryOne (LEO), which leverages multiple instructions (as different views) for the same trajectory to resolve language ambiguity and improve generalization. Expand

We explore the benefits of decreasing the input length of transformers. Expand

We study end-to-end segmental models with different weight functions, including ones based on frame-level neural classifiers and on segmental recurrent neural networks. Expand

We develop a formal hierarchy of the expressive capacity of RNN architectures based on two formal properties: space complexity, which measures the RNN's memory, and rational recurrence, defined as whether the recurrent update can be described by a weighted finite-state machine. Expand

We present Rationale^VT Transformer, an integrated model that learns to generate free-text rationales by combining pretrained language models with object recognition, grounded visual semantic frames, and visual commonsense graphs. Expand

We use multilingual BERT to create source and target sentence embeddings for nearest-neighbor search and adapt the model via self-training. Expand