WINOGRANDE: An Adversarial Winograd Schema Challenge at Scale

@inproceedings{Sakaguchi2020WINOGRANDEAA,
  title={WINOGRANDE: An Adversarial Winograd Schema Challenge at Scale},
  author={Keisuke Sakaguchi and Ronan Le Bras and Chandra Bhagavatula and Yejin Choi},
  booktitle={AAAI},
  year={2020}
}
The Winograd Schema Challenge (WSC), proposed by Levesque et al. (2011) as an alternative to the Turing Test, was originally designed as a pronoun resolution problem that cannot be solved based on statistical patterns in large text corpora. [...] Key Method Key to our approach is a novel adversarial filtering algorithm AFLITE for systematic bias reduction, combined with a careful crowdsourcing design. Despite the significant increase in training data, the performance of existing state-of-the-art methods remains…Expand
99 Citations
Precise Task Formalization Matters in Winograd Schema Evaluations
  • Highly Influenced
  • PDF
WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challenge
  • 5
  • PDF
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations
  • 1
  • Highly Influenced
  • PDF
A Review of Winograd Schema Challenge Datasets and Approaches
  • 3
  • PDF
An Analysis of Dataset Overlap on Winograd-Style Tasks
  • 1
  • Highly Influenced
  • PDF
Adversarial Filters of Dataset Biases
  • 39
  • PDF
UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark
  • 1
  • PDF
XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning
  • 19
  • PDF
G-DAUG: Generative Data Augmentation for Commonsense Reasoning
  • 11
  • PDF
Generative Data Augmentation for Commonsense Reasoning
  • 4
  • PDF
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 50 REFERENCES
SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference
  • 295
  • PDF
A Simple Method for Commonsense Reasoning
  • 141
  • PDF
Establishing a Human Baseline for the Winograd Schema Challenge
  • 19
  • PDF
Annotation Artifacts in Natural Language Inference Data
  • 398
  • Highly Influential
  • PDF
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
  • 190
  • PDF
Easy Victories and Uphill Battles in Coreference Resolution
  • 180
  • PDF
Probing Neural Network Comprehension of Natural Language Arguments
  • 152
  • PDF
On the Evaluation of Common-Sense Reasoning in Natural Language Understanding
  • 17
  • Highly Influential
  • PDF
...
1
2
3
4
5
...