Share This Author
Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics
Random Feature Attention
- Hao Peng, Nikolaos Pappas, Dani Yogatama, Roy Schwartz, Noah A. Smith, Lingpeng Kong
- Computer ScienceICLR
- 3 March 2021
RFA, a linear time and space attention that uses random feature methods to approximate the softmax function, is proposed and explored, showing that RFA is competitive in terms of both accuracy and efficiency on three long text classification datasets.
Diffusion of Lexical Change in Social Media
- Jacob Eisenstein, Brendan T. O'Connor, Noah A. Smith, E. Xing
- Computer SciencePloS one
- 18 October 2012
Using a latent vector autoregressive model to aggregate across thousands of words, high-level patterns in diffusion of linguistic change over the United States are identified and support for prior arguments that focus on geographical proximity and population size is offered.
Evaluating Models’ Local Decision Boundaries via Contrast Sets
A more rigorous annotation paradigm for NLP that helps to close systematic gaps in the test data, and recommends that the dataset authors manually perturb the test instances in small but meaningful ways that (typically) change the gold label, creating contrast sets.
The Right Tool for the Job: Matching Model and Instance Complexities
- Roy Schwartz, Gabriel Stanovsky, Swabha Swayamdipta, Jesse Dodge, Noah A. Smith
- Computer ScienceACL
- 16 April 2020
This work proposes a modification to contextual representation fine-tuning which allows for an early (and fast) “exit” from neural network calculations for simple instances, and late (and accurate) exit for hard instances during inference.
A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
- Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, Matt Gardner
- Computer ScienceNAACL
- 7 May 2021
Qasper is presented, a dataset of 5049 questions over 1585 Natural Language Processing papers that is designed to facilitate document-grounded, information-seeking QA, and finds that existing models that do well on other QA tasks do not perform well on answering these questions.
Bayesian Optimization of Text Representations
This work applies a sequential model-based optimization technique and shows that this method makes standard linear models competitive with more sophisticated, expensive state-of-the-art methods based on latent variable models or neural networks on various topic classification and sentiment analysis problems.
Shortformer: Better Language Modeling using Shorter Inputs
This work identifies conditions where shorter inputs are not harmful, and achieves perplexity and efficiency improvements through two new methods that decrease input length, and shows how to improve the efficiency of recurrence methods in transformers.