Corpus ID: 237485574

Neural Machine Translation Quality and Post-Editing Performance

@inproceedings{Zouhar2021NeuralMT,
  title={Neural Machine Translation Quality and Post-Editing Performance},
  author={Vil'em Zouhar and Alevs Tamchyna and Martin Popel and Ondvrej Bojar},
  booktitle={EMNLP},
  year={2021}
}
We test the natural expectation that using MT in professional translation saves human processing time. The last such study was carried out by Sanchez-Torron and Koehn (2016) with phrase-based MT, artificially reducing the translation quality. In contrast, we focus on neural MT (NMT) of high quality, which has become the state-of-the-art approach since then and also got adopted by most translation companies. Through an experimental study involving over 30 professional translators for English… 
WeTS: A Benchmark for Translation Suggestion
TLDR
A benchmark data set for TS, called WeTS, is created, which contains golden corpus annotated by expert translators on four translation directions, and the Transformer-based model for TS is introduced, and experimental results show that the model achieves State-Of-The-Art (SOTA) results on all fourtranslation directions.
Measuring Uncertainty in Translation Quality Evaluation (TQE)
TLDR
This work carries out such a motivated research to correctly estimate the confidence intervals depending on the sample size of translated text, e.g. the amount of words or sentences, that needs to be processed on TQE workflow step for confident and reliable evaluation of overall translation quality.

References

SHOWING 1-10 OF 24 REFERENCES
Post-editing Productivity with Neural Machine Translation: An Empirical Assessment of Speed and Quality in the Banking and Finance Domain
TLDR
It is found that even with language pairs that have received little attention in research settings and small amounts of in-domain data for system adaptation, NMT post-editing allows for substantial time savings and leads to equal or slightly better quality.
Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals
TLDR
A deep-learning system, CUBBITT, which approaches the quality of human translation and even surpasses it in adequacy in certain circumstances, suggesting that deep learning may have the potential to replace humans in applications where conservation of meaning is the primary aim.
Interactive translation prediction versus conventional post-editing in practice: a study with the CasMaCat  workbench
TLDR
Interactive translation prediction (ITP) can be as productive as conventional PE in terms of the total time required to produce the final translation and translation editors working with ITP require fewer key strokes to arrive at the final version of their translation.
Machine Translation Quality and Post-Editor Productivity
TLDR
The MT system with the lowest BLEU score produced the output that was post-edited to the lowest quality and with the highest PE effort, measured both in HTER and actual PE operations.
A product and process analysis of post-editor corrections on neural, statistical and rule-based machine translation output
TLDR
The results suggest that although different types of edits were needed to outputs from NMT, RBMT and SMT systems, the difference is not necessarily reflected in process-based effort indicators.
Community-based post-editing of machine-translated content: monolingual vs. bilingual
TLDR
It is found that monolingual post-editing can lead to improved fluency and comprehensibility scores similar to those achieved through bilingual post-edsiting, while fidelity improved considerably more for the bilingual set-up.
Is Machine Translation Post-editing Worth the Effort?: A Survey of Research into Post-editing and Effort
TLDR
An overview of the use of post-editing of machine translation as an increasingly central practice in the translation field is presented and a view of current knowledge concerning post-Editing productivity and effort is presented.
WMT20 Document-Level Markable Error Exploration
TLDR
This paper inspects which specific markables are problematic for MT systems and concludes with an analysis of the effect of markable error types on the MT performance measured by humans and automatic evaluation tools.
A Productivity Test of Statistical Machine Translation Post-Editing in a Typical Localisation Context
TLDR
A Productivity Test of Statistical Machine Translation Post-Editing in a Typical Localisation Context and results show a productivity increase for each participant, with significant variance across inviduals.
The FISKMÖ Project: Resources and Tools for Finnish-Swedish Machine Translation and Cross-Linguistic Research
TLDR
FISKMÖ, a project that focuses on the development of resources and tools for cross-linguistic research and machine translation between Finnish and Swedish, has released new data sets, a benchmark test set for MT development, pre-trained neural MT models with high coverage and competitive performance and a self-contained MT plugin for a popular CAT tool.
...
1
2
3
...