From captions to visual concepts and back

  title={From captions to visual concepts and back},
  author={Hao Fang and Saurabh Gupta and Forrest N. Iandola and Rupesh Kumar Srivastava and Li Deng and Piotr Doll{\'a}r and Jianfeng Gao and Xiaodong He and Margaret Mitchell and John C. Platt and C. Lawrence Zitnick and Geoffrey Zweig},
  journal={2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
This paper presents a novel approach for automatically generating image descriptions: visual detectors, language models, and multimodal similarity models learnt directly from a dataset of image captions. We use multiple instance learning to train visual detectors for words that commonly occur in captions, including many different parts of speech such as nouns, verbs, and adjectives. The word detector outputs serve as conditional inputs to a maximum-entropy language model. The language model… CONTINUE READING
Highly Influential
This paper has highly influenced 53 other papers. REVIEW HIGHLY INFLUENTIAL CITATIONS
Highly Cited
This paper has 630 citations. REVIEW CITATIONS

From This Paper

Figures, tables, and topics from this paper.


Publications citing this paper.
Showing 1-10 of 366 extracted citations

Image Captioning With Visual-Semantic Double Attention

TOMM • 2019
View 16 Excerpts
Method Support
Highly Influenced

Computer Vision – ECCV 2018

Lecture Notes in Computer Science • 2018
View 11 Excerpts
Highly Influenced

From Eliza to XiaoIce: challenges and opportunities with social chatbots

Frontiers of Information Technology & Electronic Engineering • 2018
View 4 Excerpts
Highly Influenced

From image to language and back again

Natural Language Engineering • 2018
View 10 Excerpts
Highly Influenced

Image Captioning and Visual Question Answering Based on Attributes and External Knowledge

IEEE Transactions on Pattern Analysis and Machine Intelligence • 2018
View 5 Excerpts
Highly Influenced

Multi Instance Learning For Unbalanced Data

ArXiv • 2018
View 8 Excerpts
Method Support
Highly Influenced

631 Citations

Citations per Year
Semantic Scholar estimates that this publication has 631 citations based on the available data.

See our FAQ for additional information.