Evaluating Gender Bias in Machine Translation

@article{Stanovsky2019EvaluatingGB,
  title={Evaluating Gender Bias in Machine Translation},
  author={Gabriel Stanovsky and Noah A. Smith and Luke Zettlemoyer},
  journal={ArXiv},
  year={2019},
  volume={abs/1906.00591}
}
We present the first challenge set and evaluation protocol for the analysis of gender bias in machine translation (MT. [...] Key Method We devise an automatic gender bias evaluation method for eight target languages with grammatical gender, based on morphological analysis (e.g., the use of female inflection for the word "doctor"). Our analyses show that four popular industrial MT systems and two recent state-of-the-art academic MT models are significantly prone to gender-biased translation errors for all…Expand

Figures, Tables, and Topics from this paper

Gender Coreference and Bias Evaluation at WMT 2020
Evaluating Gender Bias in Hindi-English Machine Translation
Mitigating Gender Bias in Machine Translation with Target Gender Annotations
First the worst: Finding better gender translations during beam search
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 28 REFERENCES
Equalizing Gender Biases in Neural Machine Translation with Word Embeddings Techniques
Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods
Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns
Evaluating Discourse Phenomena in Neural Machine Translation
Understanding Back-Translation at Scale
Gender Bias in Coreference Resolution
A Challenge Set Approach to Evaluating Machine Translation
...
1
2
3
...