• Corpus ID: 26392513

Neural machine translation for low-resource languages

@article{stling2017NeuralMT,
  title={Neural machine translation for low-resource languages},
  author={Robert {\"O}stling and J{\"o}rg Tiedemann},
  journal={ArXiv},
  year={2017},
  volume={abs/1708.05729}
}
Neural machine translation (NMT) approaches have improved the state of the art in many machine translation settings over the last couple of years, but they require large amounts of training data to produce sensible output. [] Key Result We find that while SMT remains the best option for low-resource settings, our method can produce acceptable translations with only 70000 tokens of training data, a level where the baseline NMT system fails completely.

Tables from this paper

Revisiting Low-Resource Neural Machine Translation: A Case Study
TLDR
It is shown that, without the use of any auxiliary monolingual or multilingual data, an optimized NMT system can outperform PBSMT with far less data than previously claimed.
Revisiting Low-Resource Neural Machine Translation: A Case Study
TLDR
It is shown that, without the use of any auxiliary monolingual or multilingual data, an optimized NMT system can outperform PBSMT with far less data than previously claimed.
Transformers for Low-resource Neural Machine Translation
TLDR
This research modified the Transformer-based neural machine translation architectures for low-resource polysynthetic languages and outperformed the strong baseline in the automatic evaluation of the experiments on the public benchmark datasets.
Comparing Statistical and Neural Machine Translation Performance on Hindi-To-Tamil and English-To-Tamil
TLDR
This work investigates the performance of PB-SMT and NMT on two rarely tested under-resourced language pairs, English-To-Tamil and Hindi-To -Tamil, taking a specialised data domain into consideration.
Neural Machine Translation: A Review and Survey
TLDR
This work traces back the origins of modern NMT architectures to word and sentence embeddings and earlier examples of the encoder-decoder network family and concludes with a survey of recent trends in the field.
Neural Machine Translation: A Review
TLDR
This work traces back the origins of modern NMT architectures to word and sentence embeddings and earlier examples of the encoder-decoder network family and concludes with a survey of recent trends in the field.
Dict-NMT: Bilingual Dictionary based NMT for Extremely Low Resource Languages
TLDR
This paper explores methods using bilingual dictionaries with an NMT model to improve translations for extremely low resource languages, and extends this work to multilingual systems, exhibiting zero-shot properties.
Feature decay algorithms for neural machine translation
TLDR
It is revealed that it is possible to find a subset of sentence pairs, that outperforms by 1.11 BLEU points the full training corpus, when used for training a German-English NMT system.
An Error-based Investigation of Statistical and Neural Machine Translation Performance on Hindi-to-Tamil and English-to-Tamil
TLDR
This work investigates the performance of phrase-based SMT (PB-SMT) and NMT on two rarely-tested low-resource language-pairs, English-to-Tamil and Hindi-to theTamil, taking a specialised data domain (software localisation) into consideration.
Low-Resource Machine Transliteration Using Recurrent Neural Networks
TLDR
Evaluation and experiments involving French and Vietnamese showed that with only a small bilingual pronunciation dictionary available for training the transliteration models, promising results were obtained with a large increase in BLEU scores and a reduction in Translation Error Rate and Phoneme Error Rate.
...
...

References

SHOWING 1-10 OF 18 REFERENCES
Transfer Learning for Low-Resource Neural Machine Translation
TLDR
A transfer learning method is presented that significantly improves Bleu scores across a range of low-resource languages by first train a high-resource language pair, then transfer some of the learned parameters to the low- resource pair to initialize and constrain training.
Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models
TLDR
A novel word-character solution to achieving open vocabulary NMT that can successfully learn to not only generate well-formed words for Czech, a highly-inflected language with a very complex vocabulary, but also build correct representations for English source words.
Neural Machine Translation by Jointly Learning to Align and Translate
TLDR
It is conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and it is proposed to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly.
Multilingual Projection for Parsing Truly Low-Resource Languages
TLDR
This work proposes a novel approach to cross-lingual part-of-speech tagging and dependency parsing for truly low-resource languages that consistently provides top-level accuracies, close to established upper bounds, and outperforms several competitive baselines.
Statistical Phrase-Based Translation
TLDR
The empirical results suggest that the highest levels of performance can be obtained through relatively simple means: heuristic learning of phrase translations from word-based alignments and lexical weighting of phrase translation.
Sequence to Sequence Learning with Neural Networks
TLDR
This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.
If all you have is a bit of the Bible: Learning POS taggers for truly low-resource languages
TLDR
This work learns POS taggers for 100 languages, using the languages to bootstrap each other and is often competitive with weakly supervised approaches that assume high-quality parallel corpora, representative monolingual corpora with perfect tokenization, and/or tag dictionaries.
Minimum Error Rate Training in Statistical Machine Translation
TLDR
It is shown that significantly better results can often be obtained if the final evaluation criterion is taken directly into account as part of the training procedure.
Moses: Open Source Toolkit for Statistical Machine Translation
We describe an open-source toolkit for statistical machine translation whose novel contributions are (a) support for linguistically motivated factors, (b) confusion network decoding, and (c)
LSTM Neural Networks for Language Modeling
TLDR
This work analyzes the Long Short-Term Memory neural network architecture on an English and a large French language modeling task and gains considerable improvements in WER on top of a state-of-the-art speech recognition system.
...
...