• DOI:10.18653/v1/2020.findings-emnlp.311
Corpus ID: 222141056

UnQovering Stereotyping Biases via Underspecified Questions

@inproceedings{Li2020UnQoveringSB,
  title={UnQovering Stereotyping Biases via Underspecified Questions},
  author={Tao Li and Daniel Khashabi and Tushar Khot and Ashish Sabharwal and Vivek Srikumar},
  booktitle={EMNLP},
  year={2020}
}

• Tao Li, Daniel Khashabi, +2 authors Vivek Srikumar
• Published in EMNLP 2020
• Computer Science

While language embeddings have been shown to have stereotyping biases, how these biases affect downstream question answering (QA) models remains unexplored. We present UNQOVER, a general framework to probe and quantify biases through underspecified questions. We show that a naive use of model scores can lead to incorrect bias estimates due to two forms of reasoning errors: positional dependence and question independence. We design a formalism that isolates the aforementioned errors. As case… Expand

View on ACL

arxiv.org

Save to Library

Create Alert

Cite

Launch Research Feed