Multimodal Attention for Neural Machine Translation

@article{Caglayan2016MultimodalAF,
  title={Multimodal Attention for Neural Machine Translation},
  author={Ozan Caglayan and Lo{\"i}c Barrault and Fethi Bougares},
  journal={CoRR},
  year={2016},
  volume={abs/1609.03976}
}
The attention mechanism is an important part of the neural machine translation (NMT) where it was reported to produce richer source representation compared to fixed-length encoding sequence-to-sequence models. Recently, the effectiveness of attention has also been explored in the context of image captioning. In this work, we assess the feasibility of a multimodal attention mechanism that simultaneously focus over an image and its natural language description for generating a description in… CONTINUE READING
Highly Cited
This paper has 18 citations. REVIEW CITATIONS
Related Discussions
This paper has been referenced on Twitter 12 times. VIEW TWEETS

From This Paper

Figures, tables, results, and topics from this paper.

Key Quantitative Results

  • We show that a dedicated attention for each modality achieves up to 1.6 points in BLEU and METEOR compared to a textual NMT baseline.

Citations

Publications citing this paper.
Showing 1-10 of 13 extracted citations

References

Publications referenced by this paper.
Showing 1-10 of 30 references

CIDEr: Consensus-based image description evaluation

2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) • 2015
View 4 Excerpts
Highly Influenced

Deep Residual Learning for Image Recognition

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) • 2016
View 4 Excerpts
Highly Influenced

Similar Papers

Loading similar papers…