• Publications
  • Influence
Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation
We reassess a recent study (Hassan et al., 2018) that claimed that machine translation (MT) has reached human parity for the translation of news from Chinese into English, using pairwise ranking and
PET: a Tool for Post-editing and Assessing Machine Translation
This work describes a standalone tool that has two main purposes: facilitate the post-editing of translations from any MT system so that they reach publishable quality and collect sentence-level information from the post -editing process, e.g.: post-Editing time and detailed keystroke statistics.
Is Neural Machine Translation the New State of the Art?
Comparing the quality of NMT systems with statistical MT is compared by describing three studies using automatic and human evaluation methods by reporting increases in fluency but inconsistent results for adequacy and post-editing effort.
Translators’ perceptions of literary post-editing using statistical and neural machine translation
In the context of recent improvements in the quality of machine translation (MT) output and new use cases being found for that output, this article reports on an experiment using statistical and
A Set of Recommendations for Assessing Human-Machine Parity in Language Translation
It is shown that the professional human translations contained significantly fewer errors, and that perceived quality in human evaluation depends on the choice of raters, the availability of linguistic context, and the creation of reference translations.
Human factors in machine translation and post-editing among institutional translators
In September 2015, the ADAPT Centre for Digital Content Technology carried out a focus group study of 70 translators at the European Commission’s Directorate-General for Translation (DGT). The aim
Approaches to Human and Machine Translation Quality Assessment
This chapter provides a critical overview of the established and developing approaches to the definition and measurement of translation quality in human and machine translation workflows across a range of research, educational, and industry scenarios.
A Comparative Quality Evaluation of PBSMT and NMT using Professional Translators
Results are mixed for perceived adequacy and for errors of omission, addition, and mistranslation, but show a preference for NMT in side-by-side ranking for all language pairs, texts, and segment lengths.
Evaluating MT for massive open online courses
A multifaceted comparison between statistical and neural machine translation systems that were developed for translation of data from massive open online courses (MOOCs) shows that neural MT is preferred in side-by-side ranking, and is found to contain fewer overall errors.
Reading comprehension of machine translation output: what makes for a better read?
This paper reports on a pilot experiment that compares two different machine translation (MT) paradigms in reading comprehension tests. To explore a suitable methodology, we set up a pilot experiment