Lightweight Adapter Tuning for Multilingual Speech Translation

  title={Lightweight Adapter Tuning for Multilingual Speech Translation},
  author={Hang Le and Juan Miguel Pino and Changhan Wang and Jiatao Gu and Didier Schwab and Laurent Besacier},
Adapter modules were recently introduced as an efficient alternative to fine-tuning in NLP. Adapter tuning consists in freezing pretrained parameters of a model and injecting lightweight modules between layers, resulting in the addition of only a small number of taskspecific trainable parameters. While adapter tuning was investigated for multilingual neural machine translation, this paper proposes a comprehensive analysis of adapters for multilingual speech translation (ST). Starting from… Expand

Figures and Tables from this paper


Simple, Scalable Adaptation for Neural Machine Translation
The proposed approach consists of injecting tiny task specific adapter layers into a pre-trained model, which adapt the model to multiple individual tasks simultaneously, paving the way towards universal machine translation. Expand
Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation
The benefits and drawbacks of freezing parameters, and adding new ones, when fine-tuning a pre-trained model on Machine Translation (MT), are investigated. Expand
Parameter-Efficient Transfer Learning for NLP
To demonstrate adapter's effectiveness, the recently proposed BERT Transformer model is transferred to 26 diverse text classification tasks, including the GLUE benchmark, and adapter attain near state-of-the-art performance, whilst adding only a few parameters per task. Expand
Multilingual Translation with Extensible Multilingual Pretraining and Finetuning
This work shows that multilingual translation models can be created through multilingual finetuning, and demonstrates that pretrained models can been extended to incorporate additional languages without loss of performance. Expand
Multilingual Denoising Pre-training for Neural Machine Translation
Abstract This paper demonstrates that multilingual denoising pre-training produces significant performance gains across a wide variety of machine translation (MT) tasks. We present mBART—aExpand
Multilingual Speech Recognition with Self-Attention Structured Parameterization
This paper investigates a streamable end-to-end multilingual system based on the Transformer Transducer and proposes several techniques for adapting the self-attention architecturebased on the language id, and analyses the trade-offs of each method with regards to quality gains and number of additional parameters introduced. Expand
Language Adapters for Zero Shot Neural Machine Translation
We propose a novel adapter layer formalism for adapting multilingual models. They are more parameter-efficient than existing adapter layers while obtaining as good or better performance. The layersExpand
Adapt-and-Adjust: Overcoming the Long-Tail Problem of Multilingual Speech Recognition
The A2 framework overcomes the long-tail problem via three techniques: exploiting a pretrained multilingual language model (mBERT) to improve the performance of low-resource languages; proposing dual adapters consisting of both language-specific and language-agnostic adaptation with minimal additional parameters; and overcoming the class imbalance. Expand
Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Model
This work presents an E2E multilingual system which is equipped to operate in low-latency interactive applications, as well as handle a key challenge of real world data: the imbalance in training data across languages. Expand
MuST-C: a Multilingual Speech Translation Corpus
MuST-C is created, a multilingual speech translation corpus whose size and quality will facilitate the training of end-to-end systems for SLT from English into 8 languages and an empirical verification of its quality and SLT results computed with a state-of-the-art approach on each language direction. Expand