Share This Author
Pathologies of Neural Models Make Interpretations Difficult
- Shi Feng, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, Jordan L. Boyd-Graber
- Computer ScienceEMNLP
- 20 April 2018
This work uses input reduction, which iteratively removes the least important word from the input, to expose pathological behaviors of neural models: the remaining words appear nonsensical to humans and are not the ones determined as important by interpretation methods.
Trick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering
- Eric Wallace, Pedro Rodriguez, Shi Feng, Ikuya Yamada, Jordan L. Boyd-Graber
- Computer ScienceTACL
- 7 September 2018
This work proposes human- in-the-loop adversarial generation, where human authors are guided to break models through an interactive user interface, and applies this generation framework to a question answering task called Quizbowl, where trivia enthusiasts craft adversarial questions.
Quizbowl: The Case for Incremental Question Answering
- Pedro Rodriguez, Shi Feng, Mohit Iyyer, He He, Jordan L. Boyd-Graber
- Computer ScienceArXiv
- 9 April 2019
This work makes two key contributions to machine learning research through Quizbowl: collecting and curating a large factoid QA dataset and an accompanying gameplay dataset, and developing a computational approach to playing Quiz Bowl that involves determining both what to answer and when to answer.
Trick Me If You Can: Adversarial Writing of Trivia Challenge Questions
An adversarial writing setting is developed, where humans interact with trained models and try to break them, which yields a challenge set, which despite being easy for trivia players to answer, systematically stumps automated question answering systems.
Mitigating Noisy Inputs for Question Answering
- Denis Peskov, Joe Barrow, Pedro Rodriguez, Graham Neubig, Jordan L. Boyd-Graber
- Computer ScienceINTERSPEECH
- 8 August 2019
This work investigates and mitigate the effects of noise from Automatic Speech Recognition systems on two factoid Question Answering (QA) tasks, and empirically shown to improve the accuracy of downstream neural QA systems.
Information Seeking in the Spirit of Learning: A Dataset for Conversational Curiosity
A Wizard of Oz dialog task is designed that tests the hypothesis that engagement increases when users are presented with facts that relate to their existing knowledge, and shows that responses which incorporate a user's prior knowledge do increase engagement.
Introduction to NIPS 2017 Competition Track
Competitions have become a popular tool in the data science community to solve hard problems, assess the state of the art and spur new research directions. Companies like Kaggle and open source…
Evaluation Examples are not Equally Informative: How should that change NLP Leaderboards?
- Pedro Rodriguez, Joe Barrow, Alexander Miserlis Hoyle, John P. Lalor, Robin Jia, Jordan L. Boyd-Graber
- Computer ScienceACL
This work creates a Bayesian leaderboard model where latent subject skill and latent item difficulty predict correct responses and analyzes the ranking reliability of leaderboards to advocate a re-imagining.
Right Answer for the Wrong Reason: Discovery and Mitigation
- Shi Feng, Eric Wallace, Mohit Iyyer, Pedro Rodriguez, Alvin Grissom II, Jordan L. Boyd-Graber
- Computer ScienceArXiv
- 20 April 2018
This work introduces a simple training technique that mitigates this problem while maintaining performance on regular examples on natural language processing tasks by removing as many words as possible from the input without changing the model prediction.
Human-Computer Question Answering: The Case for Quizbowl
The setting: the game of quiz bowl is described, it is argued why it makes a suitable game for human-computer competition, and the logistics and preparation for the competition are described.