• Corpus ID: 17848183

Perception vs. reality: measuring machine translation post-editing productivity

@inproceedings{Gaspari2014PerceptionVR,
  title={Perception vs. reality: measuring machine translation post-editing productivity},
  author={Federico Gaspari and Antonio Toral and Sudip Kumar Naskar and Declan Groves and Andy Way},
  booktitle={Conference of the Association for Machine Translation in the Americas},
  year={2014}
}

• F. Gaspari, Antonio Toral, +2 authors Andy Way
• Published in
  Conference of the Association…
  2014
• Business

This paper presents a study of user-perceived vs real machine translation (MT) post-editing effort and productivity gains, focusing on two bidirectional language pairs: English—German and English—Dutch. Twenty experienced media professionals post-edited statistical MT output and also manually translated comparative texts within a production environment. The paper compares the actual post-editing time against the users’ perception of the effort and time required to post-edit the MT output to achieve publishable quality, thus measuring real (vs perceived) productivity gains. Although for all the language pairs users perceived MT post-editing to be slower, in fact it proved to be a faster option than manual translation for two translation directions out of four, i.e. for Dutch to English, and (marginally) for English to German. For further objective scrutiny, the paper also checks the correlation of three state-of-the-art automatic MT evaluation metrics (BLEU, METEOR and TER) with the actual post-editing time.