Fast and Robust Neural Network Joint Models for Statistical Machine Translation

@inproceedings{Devlin2014FastAR,
  title={Fast and Robust Neural Network Joint Models for Statistical Machine Translation},
  author={Jacob Devlin and Rabih Zbib and Zhongqiang Huang and Thomas Lamar and Richard M. Schwartz and John Makhoul},
  booktitle={ACL},
  year={2014}
}
Recent work has shown success in using neural network language models (NNLMs) as features in MT systems. [] Key Method Our model is purely lexicalized and can be integrated into any MT decoder. We also present several variations of the NNJM which provide significant additive improvements.

Figures and Tables from this paper

A Binarized Neural Network Joint Model for Machine Translation
TLDR
This paper proposes an alternative to NCE, the binarized NNJM (BNNJM), which learns a binary classifier that takes both the context and target words as input, and can be efficiently trained using MLE.
Neural Reordering Feature for Statistical Machine Translation
TLDR
A novel neural reordering model that directly models word pairs and their alignment is presented that achieves significant improvements over various baseline systems by utilizing LSTM recurrent neural networks.
Neural network joint modeling via context-dependent projection
  • Yik-Cheung Tam, Yun Lei
  • Computer Science
    2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2015
TLDR
This work proposes a context-dependent linear projection approach which considers neighboring words in neural network joint modeling and shows that this approach improves NNJM by 0.5 BLEU for English-Iraqi Arabic translation in N-best rescoring.
Neural Network Joint Language Model : An Investigation and An Extension With Global Source Context
TLDR
This project first investigates a state-of-theart joint language model by studying architectural and parametric factors through experiments and visualizations and proposes an extension to this model that incorporates global source context information.
Fast and Accurate Preordering for SMT using Neural Networks
TLDR
This work proposes the use of neural networks to model source-side preordering for faster and better statistical machine translation, and shows that this yields the best reordering performance against other state-of-the-art techniques.
A Bidirectional Recurrent Neural Language Model for Machine Translation
TLDR
Results show that the neural model trained with the selected data outperforms the results obtained by an n-gram language model.
LSTM Neural Reordering Feature for Statistical Machine Translation
TLDR
A novel neural reordering model that directly models word pairs and alignment is presented that achieves significant improvements over various baseline systems by utilizing LSTM recurrent neural networks.
Hybrid Neural Network Alignment and Lexicon Model in Direct HMM for Statistical Machine Translation
TLDR
This work proposes a direct HMM with neural network-based lexicon and alignment models, which are trained jointly using the Baum-Welch algorithm to rerank the n-best list created by a state-of-the-art phrase-based translation system.
Bilingual Recurrent Neural Networks for improved statistical machine translation
TLDR
This work investigates using bilingual word-aligned sentences to train a bilingual recurrent neural network model, and employs a bag-of-word representation of a source sentence as additional input features in model training.
An Investigation on Statistical Machine Translation with Neural Language Models
TLDR
The authors' experiments on large-scale datasets show that decoding with conventional n-gram LMs plus un-normalized feedforward NPLMs extended with word clusters could significantly improve the translation performance, but results also show that current NPL Ms, including feedforward and RNN still cannot simply replace n- gram LMs for SMT.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 32 REFERENCES
Decoding with Large-Scale Neural Language Models Improves Translation
TLDR
This work develops a new model that combines the neural probabilistic language model of Bengio et al., rectified linear units, and noise-contrastive estimation, and incorporates it into a machine translation system both by reranking k-best lists and by direct integration into the decoder.
Joint Language and Translation Modeling with Recurrent Neural Networks
TLDR
This work presents a joint language and translation model based on a recurrent neural network which predicts target words based on an unbounded history of both source and target words which shows competitive accuracy compared to the traditional channel model features.
11,001 New Features for Statistical Machine Translation
TLDR
On a large-scale Chinese-English translation task, the Margin Infused Relaxed Algorithm is used to add a large number of new features to two machine translation systems: the Hiero hierarchical phrase-based translation system and the syntax-basedtranslation system.
Continuous Space Translation Models with Neural Networks
TLDR
Several continuous space translation models are explored, where translation probabilities are estimated using a continuous representation of translation units in lieu of standard discrete representations, jointly computed using a multi-layer neural network with a SOUL architecture.
Continuous Space Translation Models for Phrase-Based Statistical Machine Translation
TLDR
Experimental evidence is provided that the approach seems to be able to infer meaningful translation probabilities for phrase pairs not seen in the training data, or even predict a list of the most likely translations given a source phrase.
String-to-Dependency Statistical Machine Translation
We propose a novel string-to-dependency algorithm for statistical machine translation. This algorithm employs a target dependency language model during decoding to exploit long distance word
Continuous Space Language Models for Statistical Machine Translation
TLDR
This work proposes to use a new statistical language model that is based on a continuous representation of the words in the vocabulary, which achieves consistent improvements in the BLEU score on the development and test data.
Recurrent neural network based language model
TLDR
Results indicate that it is possible to obtain around 50% reduction of perplexity by using mixture of several RNN LMs, compared to a state of the art backoff language model.
Factored bilingual n-gram language models for statistical machine translation
TLDR
This work extends the n-gram-based approach to SMT by tightly integrating more general word representations, such as lemmas and morphological classes, and uses the flexible framework of FLMs to apply a number of different back-off techniques.
Recurrent Continuous Translation Models
We introduce a class of probabilistic continuous translation models called Recurrent Continuous Translation Models that are purely based on continuous representations for words, phrases and sentences
...
1
2
3
4
...