Skip to search formSkip to main contentSkip to account menu

Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

This work balances the popular VQA dataset by collecting complementary images such that every question in the authors' balanced dataset is associated with not just a single image, but rather a pair of similar images that result in two different answers to the question.
View on IEEE (opens in a new tab)

Yin and Yang: Balancing and Answering Binary Visual Questions

This paper addresses binary Visual Question Answering on abstract scenes as visual verification of concepts inquired in the questions by converting the question to a tuple that concisely summarizes the visual concept to be detected in the image.
View on IEEE (opens in a new tab)

Counterfactual Visual Explanations

It is found that users trained to distinguish bird species fare better when given access to counterfactual explanations in addition to training examples, and the effectiveness of these explanations in teaching humans is explored.
View PDF on arXiv (opens in a new tab)

Explaining Classifiers with Causal Concept Effect (CaCE)

This work defines the Causal Concept Effect (CaCE) as the causal effect of a human-interpretable concept on a deep neural net's predictions, and shows that the CaCE measure can avoid errors stemming from confounding.
View PDF on arXiv (opens in a new tab)

Towards Transparent AI Systems: Interpreting Visual Question Answering Models

The problem of interpreting Visual Question Answering (VQA) models is addressed, and it is found that even without explicit attention mechanisms, VQA models may sometimes be implicitly attending to relevant regions in the image, and often to appropriate words in the question.
View PDF on arXiv (opens in a new tab)

Question-Conditioned Counterfactual Image Generation for VQA

This ongoing work proposes learning to generate counterfactual images for a VQA model - i.e. given a question-image pair, the model is asked to generate a new image such that i) the V QA model outputs a different answer, ii) the new image is minimally different from the original, and iii) thenew image is realistic.
View PDF on arXiv (opens in a new tab)

CloudCV: Large-Scale Distributed Computer Vision as a Cloud Service

The goal is to democratize computer vision; one should not have to be a computer vision, big data and distributed computing expert to have access to state-of-the-art distributed computer vision algorithms.
View PDF on arXiv (opens in a new tab)

Image Retrieval from Contextual Descriptions

A new multimodal challenge, Image Retrieval from Contextual Descriptions (ImageCoDe), where models are tasked with retrieving the correct image from a set of 10 minimally contrastive candidates based on a contextual description, revealing that these models dramatically lag behind human performance.
View on ACL (opens in a new tab)

Resolving Language and Vision Ambiguities Together: Joint Segmentation & Prepositional Attachment Resolution in Captioned Scenes

This work presents an approach to simultaneously perform semantic segmentation and prepositional phrase attachment resolution for captioned images and shows that joint reasoning produces more accurate results than any module operating in isolation.
View on ACL (opens in a new tab)

Interpreting Visual Question Answering Models

This paper uses two visualization techniques -- guided backpropagation and occlusion -- to find important words in the question and important regions in the image and presents qualitative and quantitative analyses of these importance maps.
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy (opens in a new tab), Terms of Service (opens in a new tab), and Dataset License (opens in a new tab)