Evaluating Gender Bias in Machine Translation

  title={Evaluating Gender Bias in Machine Translation},
  author={Gabriel Stanovsky and Noah A. Smith and Luke Zettlemoyer},
  • Gabriel Stanovsky, Noah A. Smith, Luke Zettlemoyer
  • Published 2019
  • Computer Science
  • ArXiv
  • We present the first challenge set and evaluation protocol for the analysis of gender bias in machine translation (MT. [...] Key Method We devise an automatic gender bias evaluation method for eight target languages with grammatical gender, based on morphological analysis (e.g., the use of female inflection for the word "doctor"). Our analyses show that four popular industrial MT systems and two recent state-of-the-art academic MT models are significantly prone to gender-biased translation errors for all…Expand Abstract

    Figures, Tables, and Topics from this paper.

    Explore Further: Topics Discussed in This Paper

    Gender Coreference and Bias Evaluation at WMT 2020
    Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpus
    Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem
    • 5
    • Highly Influenced
    • Open Access
    Automatically Identifying Gender Issues in Machine Translation using Perturbations
    • 2
    • Open Access


    Publications referenced by this paper.
    Equalizing Gender Biases in Neural Machine Translation with Word Embeddings Techniques
    • 37
    • Open Access
    Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods
    • 136
    • Highly Influential
    • Open Access
    Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns
    • 74
    • Open Access
    Evaluating Discourse Phenomena in Neural Machine Translation
    • 105
    • Open Access
    Understanding Back-Translation at Scale
    • 274
    • Open Access
    Gender Bias in Coreference Resolution
    • 108
    • Highly Influential
    • Open Access
    A Challenge Set Approach to Evaluating Machine Translation
    • 84
    • Open Access
    Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them
    • 123
    • Open Access
    Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
    • 260
    • Highly Influential
    • Open Access