WINOGRANDE: An Adversarial Winograd Schema Challenge at Scale
- Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi
- Computer ScienceAAAI Conference on Artificial Intelligence
- 24 July 2019
This work introduces WinoGrande, a large-scale dataset of 44k problems, inspired by the original WSC design, but adjusted to improve both the scale and the hardness of the dataset, and establishes new state-of-the-art results on five related benchmarks.
COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs
- Jena D. Hwang, Chandra Bhagavatula, Yejin Choi
- Computer ScienceAAAI Conference on Artificial Intelligence
- 12 October 2020
It is proposed that manually constructed CSKGs will never achieve the coverage necessary to be applicable in all situations encountered by NLP agents, and a new evaluation framework for testing the utility of KGs based on how effectively implicit knowledge representations can be learned from them is proposed.
Ground Truth for Grammatical Error Correction Metrics
- Courtney Napoles, Keisuke Sakaguchi, Matt Post, J. Tetreault
- Computer ScienceAnnual Meeting of the Association for…
- 1 July 2015
The first human evaluation of GEC system outputs is conducted, and it is shown that the rankings produced by metrics such as MaxMatch and I-measure do not correlate well with this ground truth.
Abductive Commonsense Reasoning
- Chandra Bhagavatula, Ronan Le Bras, Yejin Choi
- Computer ScienceInternational Conference on Learning…
- 15 August 2019
This study introduces a challenge dataset, ART, that consists of over 20k commonsense narrative contexts and 200k explanations, and conceptualizes two new tasks -- Abductive NLI: a multiple-choice question answering task for choosing the more likely explanation, and Abduction NLG: a conditional generation task for explaining given observations in natural language.
JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction
- J. Tetreault, Keisuke Sakaguchi, Courtney Napoles
- LinguisticsConference of the European Chapter of the…
- 14 February 2017
A new parallel corpus, JHU FLuency-Extended GUG corpus (JFLEG), which represents a broad range of language proficiency levels and uses holistic fluency edits to not only correct grammatical errors but also make the original text more native sounding.
Universal Decompositional Semantics on Universal Dependencies
- Aaron Steven White, D. Reisinger, Benjamin Van Durme
- Computer ScienceConference on Empirical Methods in Natural…
- 1 November 2016
A framework for augmenting data sets from the Universal Dependencies project with Universal Decompositional Semantics, and describes results from annotating the English Universal Dependency treebank, dealing with word senses, semantic roles, and event properties.
There’s No Comparison: Reference-less Evaluation Metrics in Grammatical Error Correction
- Courtney Napoles, Keisuke Sakaguchi, J. Tetreault
- Computer ScienceConference on Empirical Methods in Natural…
- 7 October 2016
It is shown that reference-less grammaticality metrics correlate very strongly with human judgments and are competitive with the leading reference-based evaluation metrics.
Robsut Wrod Reocginiton via Semi-Character Recurrent Neural Network
- Keisuke Sakaguchi, Kevin Duh, Matt Post, Benjamin Van Durme
- Computer ScienceAAAI Conference on Artificial Intelligence
- 1 August 2016
Inspired by the findings from the Cmabrigde Uinervtisy effect, a word recognition model based on a semi-character level recurrent neural network (scRNN) is proposed that has significantly more robust performance in word spelling correction compared to existing spelling checkers and character-based convolutional neural network.
Reassessing the Goals of Grammatical Error Correction: Fluency Instead of Grammaticality
- Keisuke Sakaguchi, Courtney Napoles, Matt Post, J. Tetreault
- Computer ScienceInternational Conference on Topology, Algebra and…
- 2 June 2016
It is shown that automatic evaluation with the authors' new annotation scheme has very strong correlation with expert rankings, and it is advocated for a fundamental and necessary shift in the goal of GEC, from correcting small, labeled error types, to producing text that has native fluency.
Grammatical Error Correction with Neural Reinforcement Learning
- Keisuke Sakaguchi, Matt Post, Benjamin Van Durme
- Computer ScienceInternational Joint Conference on Natural…
- 2 July 2017
It is demonstrated that NRL outperforms MLE both in human and automated evaluation metrics, achieving the state-of-the-art on a fluency-oriented GEC corpus.
...
...