ZeroC: A Neuro-Symbolic Model for Zero-shot Concept Recognition and Acquisition at Inference Time

@article{Wu2022ZeroCAN,
  title={ZeroC: A Neuro-Symbolic Model for Zero-shot Concept Recognition and Acquisition at Inference Time},
  author={Tailin Wu and Megan Tjandrasuwita and Zhengxuan Wu and Xuelin Yang and Kevin Liu and Rok Sosivc and Jure Leskovec},
  journal={ArXiv},
  year={2022},
  volume={abs/2206.15049}
}
Humans have the remarkable ability to recognize and acquire novel visual concepts in a zero-shot manner. Given a high-level, symbolic description of a novel concept in terms of previously learned visual concepts and their relations, humans can recognize novel concepts without seeing any examples. Moreover, they can acquire new concepts by parsing and communicating symbolic structures using learned visual concepts and relations. Endowing these capabilities in machines is pivotal in improving… 

Figures and Tables from this paper

ViRel: Unsupervised Visual Relations Discovery with Graph-level Analogy

TLDR
ViRel is introduced, a method for unsupervised discovery and learning of Vi sual Rel ations with graph-level analogy that achieves above 95% accuracy in relation classification, discovers the relation graph structure for most tasks, and further generalizes to unseen tasks with more complicated relational structures.

CS 231N Project Report Unsupervised learning of Visual Object Relations with Graph-Level Analogy

TLDR
This work introduces a method for unsupervised discovery and learning of Visual Relations with graph-level analogy, and shows that this method achieves above 95% accuracy in relation classification, discovers the relation graph structure for most tasks, and further generalizes to unseen tasks with more complicated relational structures.

References

SHOWING 1-10 OF 25 REFERENCES

The Neuro-Symbolic Concept Learner: Interpreting Scenes Words and Sentences from Natural Supervision

We propose the Neuro-Symbolic Concept Learner (NS-CL), a model that learns visual concepts, words, and semantic parsing of sentences without explicit supervision on any of them; instead, our model

SCAN: Learning Hierarchical Compositional Visual Concepts

TLDR
SCAN (Symbol-Concept Association Network), a new framework for learning such abstractions in the visual domain that allows for traversal and manipulation of the implicit hierarchy of visual concepts through symbolic instructions and learnt logical recombination operations, is described.

Concept Learners for Few-Shot Learning

Developing algorithms that are able to generalize to a novel task given only a few labeled examples represents a fundamental challenge in closing the gap between machine- and human-level performance.

Contrastive Learning of Structured World Models

TLDR
These experiments demonstrate that C-SWMs can overcome limitations of models based on pixel reconstruction and outperform typical representatives of this model class in highly structured environments, while learning interpretable object-based representations.

Human-level concept learning through probabilistic program induction

TLDR
A computational model is described that learns in a similar fashion and does so better than current deep learning algorithms and can generate new letters of the alphabet that look “right” as judged by Turing-like tests of the model's output in comparison to what real humans produce.

Generating Visual Representations for Zero-Shot Classification

TLDR
This paper suggests to address ZSC and GZSC by i) learning a conditional generator using seen classes ii) generate artificial training examples for the categories without exemplars, which is then turned into a standard supervised learning problem.

An embarrassingly simple approach to zero-shot learning

TLDR
This paper describes a zero-shot learning approach that can be implemented in just one line of code, yet it is able to outperform state of the art approaches on standard datasets.

Unsupervised Learning of Compositional Energy Concepts

TLDR
This work proposes COMET, which discovers and represents concepts as separate energy functions, enabling us to represent both global concepts as well as objects under a unified framework.

Compositional Visual Generation and Inference with Energy Based Models

TLDR
This paper shows that energy-based models can exhibit compositional generation abilities of their model by directly combining probability distributions, and demonstrates other unique advantages of the model, such as the ability to continually learn and incorporate new concepts, or infer compositions of concept properties underlying an image.

Zero-Shot Learning: An Energy Based Approach

TLDR
This paper proposes an Energy-Based Zero-shot Learning model (EBZL), which adapt tradition deep Boltzmann machine to a supervised setting without changing its property as an undirected probabilistic graphic model, which helps to preserve semantic integrity and circumvents semantic loss problem.