From Speech-to-Speech Translation to Automatic Dubbing

@article{Federico2020FromST,
  title={From Speech-to-Speech Translation to Automatic Dubbing},
  author={Marcello Federico and Robert Enyedi and R. Barra-Chicote and Ritwik Giri and Umut Isik and A. Krishnaswamy},
  journal={ArXiv},
  year={2020},
  volume={abs/2001.06785}
}
We present enhancements to a speech-to-speech translation pipeline in order to perform automatic dubbing. Our architecture features neural machine translation generating output of preferred length, prosodic alignment of the translation with the original speech segments, neural text-to-speech with fine tuning of the duration of each utterance, and, finally, audio rendering to enriches text-to-speech output with background noise and reverberation extracted from the original audio. We report and… Expand
Improvements to Prosodic Alignment for Automatic Dubbing
Evaluating and Optimizing Prosodic Alignment for Automatic Dubbing
Machine Translation Verbosity Control for Automatic Dubbing
Speech Translation and the End-to-End Promise: Taking Stock of Where We Are
The Two Shades of Dubbing in Neural Machine Translation

References

SHOWING 1-10 OF 47 REFERENCES
Finite-state speech-to-speech translation
  • E. Vidal
  • Computer Science
  • 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing
  • 1997
Prosodic Phrase Alignment for Machine Dubbing
The ATR Multilingual Speech-to-Speech Translation System
The NESPOLE! speech-to-speech translation system
Automatic post-synchronization of speech utterances
Automatic dubbing of TV programmes for the hearing impaired
MuST-C: a Multilingual Speech Translation Corpus
Interactive Translation of Conversational Speech
...
1
2
3
4
5
...