Modulating early visual processing by language

@inproceedings{Vries2017ModulatingEV,
  title={Modulating early visual processing by language},
  author={Harm de Vries and Florian Strub and J{\'e}r{\'e}mie Mary and Hugo Larochelle and Olivier Pietquin and Aaron C. Courville},
  booktitle={NIPS},
  year={2017}
}
It is commonly assumed that language refers to high-level visual concepts while leaving low-level visual processing unaffected. This view dominates the current literature in computational models for language-vision tasks, where visual and linguistic inputs are mostly processed independently before being fused into a single representation. In this paper, we deviate from this classic pipeline and propose to modulate the entire visual processing by a linguistic input. Specifically, we introduce… CONTINUE READING
Highly Cited
This paper has 48 citations. REVIEW CITATIONS

Citations

Publications citing this paper.
Showing 1-10 of 36 extracted citations

References

Publications referenced by this paper.
Showing 1-10 of 30 references

MUTAN: Multimodal Tucker Fusion for Visual Question Answering

2017 IEEE International Conference on Computer Vision (ICCV) • 2017
View 4 Excerpts
Highly Influenced

Ask Your Neurons: A Neural-Based Approach to Answering Questions about Images

2015 IEEE International Conference on Computer Vision (ICCV) • 2015
View 4 Excerpts
Highly Influenced

VQA: Visual Question Answering

International Journal of Computer Vision • 2015
View 8 Excerpts
Highly Influenced

Hadamard product for low-rank bilinear pooling

J. Kim, K. On, J. Ha, B. Zhang
In Proc. of ICLR, • 2017
View 6 Excerpts
Highly Influenced

Deep Residual Learning for Image Recognition

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) • 2016
View 3 Excerpts
Highly Influenced

Multimodal Residual Learning for Visual QA

NIPS • 2016
View 6 Excerpts
Highly Influenced

GuessWhat?! Visual Object Discovery through Multi-modal Dialogue

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) • 2017
View 5 Excerpts

Similar Papers

Loading similar papers…