Bidirectional Joint Representation Learning with Symmetrical Deep Neural Networks for Multimodal and Crossmodal Applications

@inproceedings{Vukotic2016BidirectionalJR,
  title={Bidirectional Joint Representation Learning with Symmetrical Deep Neural Networks for Multimodal and Crossmodal Applications},
  author={Vedran Vukotic and Christian Raymond and Guillaume Gravier},
  booktitle={ICMR},
  year={2016}
}
Common approaches to problems involving multiple modalities (classification, retrieval, hyperlinking, etc.) are early fusion of the initial modalities and crossmodal translation from one modality to the other. Recently, deep neural networks, especially deep autoencoders, have proven promising both for crossmodal translation and for early fusion via multimodal embedding. In this work, we propose a flexible crossmodal deep neural network architecture for multimodal and crossmodal representation… CONTINUE READING
Highly Cited
This paper has 28 citations. REVIEW CITATIONS

From This Paper

Figures, tables, results, and topics from this paper.

Key Quantitative Results

  • Our method demonstrates improved crossmodal translation capabilities and produces a multimodal embedding that significantly outperforms multimodal embeddings obtained by deep autoencoders, resulting in an absolute increase of 14.14 in precision at 10 on a video hyperlinking task.

Citations

Publications citing this paper.
Showing 1-10 of 14 extracted citations

A Study on Multimodal Video Hyperlinking with Visual Aggregation

2018 IEEE International Conference on Multimedia and Expo (ICME) • 2018
View 6 Excerpts

References

Publications referenced by this paper.

Multimodal Deep Learning

ICML • 2011
View 4 Excerpts
Highly Influenced

Similar Papers

Loading similar papers…