Corpus ID: 173990332

Learning to Generate Grounded Image Captions without Localization Supervision

@article{Ma2019LearningTG,
  title={Learning to Generate Grounded Image Captions without Localization Supervision},
  author={Chih-Yao Ma and Yannis Kalantidis and G. Al-Regib and P{\'e}ter Vajda and Marcus Rohrbach and Z. Kira},
  journal={ArXiv},
  year={2019},
  volume={abs/1906.00283}
}

Chih-Yao Ma, Yannis Kalantidis, +3 authors Z. Kira

Published 2019

Computer Science

ArXiv

When generating a sentence description for an image, it frequently remains unclear how well the generated caption is grounded in the image or if the model hallucinates based on priors in the dataset and/or the language model. [...] Key Method In this work, we propose a novel cyclical training regimen that forces the model to localize each word in the image after the sentence decoder generates it and then reconstruct the sentence from the localized image region(s) to match the ground-truth.Expand Abstract

View PDF on arXiv

Save to Library

Create Alert

Cite

Launch Research Feed