Label-Embedding for Image Classification

@article{Akata2016LabelEmbeddingFI,
  title={Label-Embedding for Image Classification},
  author={Zeynep Akata and Florent Perronnin and Za{\"i}d Harchaoui and Cordelia Schmid},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2016},
  volume={38},
  pages={1425-1438}
}
Attributes act as intermediate representations that enable parameter sharing between classes, a must when training data is scarce. We propose to view attribute-based image classification as a label-embedding problem: each class is embedded in the space of attribute vectors. We introduce a function that measures the compatibility between an image and a label embedding. The parameters of this function are learned on a training set of labeled samples to ensure that, given an image, the correct… 

Figures and Tables from this paper

Semantic Diversity Learning for Zero-Shot Multi-label Classification

TLDR
This study introduces an end-to-end model training for multi-label zero-shot learning that supports the semantic diversity of the images and labels and proposes to use an embedding matrix having principal embedding vectors trained using a tailored loss function.

Attribute Prototype Network for Zero-Shot Learning

TLDR
A novel zero-shot representation learning framework that jointly learns discriminative global and local features using only class-level attributes and points to the visual evidence of the attributes in an image, confirming the improved attribute localization ability of the image representation.

Attribute Prediction in the Zero-Shot Setting as Multiple Instance Learning

  • Computer Science
  • 2022
TLDR
This work applies the Multiple Instance Learning (MIL) paradigm to attribute learning (AMIL) while only using class-level labels and proposes MIL-DAP, an attribute-based zero-shot classification method, based on Direct Attribute Prediction (DAP), to evaluate attribute prediction methods when no image-level data is available for evaluation.

Zero-Shot Image Classification Using Coupled Dictionary Embedding

A Simple Approach for Zero-Shot Learning based on Triplet Distribution Embeddings

TLDR
This work addresses the issue of expressivity in terms of modeling the intra-class variability for each class in Zero-Shot Learning by leveraging the use of distribution embeddings, which are modeled as Gaussian distributions.

A Joint Label Space for Generalized Zero-Shot Classification

TLDR
This paper proposes a novel pathway that uses the label space to jointly reconcile visual and semantic spaces directly, which is named Attributing Label Space (ALS).

Improving Semantic Embedding Consistency by Metric Learning for Zero-Shot Classiffication

TLDR
The proposed approach is to control the semantic embedding of images -- one of the main ingredients of zero-shot learning -- by formulating it as a metric learning problem, and gives state-of-the-art results on four challenging datasets used for zero- shot recognition evaluation.

Evaluation of output embeddings for fine-grained image classification

TLDR
This project shows that compelling classification performance can be achieved on fine-grained categories even without labeled training data, and establishes a substantially improved state-of-the-art on the Animals with Attributes and Caltech-UCSD Birds datasets.

Cross-Modal Mapping for Generalized Zero-Shot Learning by Soft-Labeling

TLDR
A novel architecture of casting ZSL as a standard neuralnetwork with cross-entropy loss to embed visual space to semantic space to link visual and semantic spaces in a cross-modal transfer/embedding space is proposed.

Dual-View Ranking with Hardness Assessment for Zero-Shot Learning

TLDR
A novel DuAl-view RanKing (DARK) loss for zeroshot learning simultaneously ranking labels for an image by point-to-point metric and ranking images for a label by pointto-set metric is proposed, which is capable of better modeling the relationship between images and classes.
...

References

SHOWING 1-10 OF 82 REFERENCES

Label-Embedding for Attribute-Based Classification

TLDR
This work proposes to view attribute-based image classification as a label-embedding problem: each class is embedded in the space of attribute vectors, and introduces a function which measures the compatibility between an image and a label embedding.

Evaluation of output embeddings for fine-grained image classification

TLDR
This project shows that compelling classification performance can be achieved on fine-grained categories even without labeled training data, and establishes a substantially improved state-of-the-art on the Animals with Attributes and Caltech-UCSD Birds datasets.

Attribute-Based Transfer Learning for Object Categorization with Zero/One Training Example

TLDR
This paper builds a generative attribute model to learn the probabilistic distributions of image features for each attribute, which can be used to classify unseen images of target categories or facilitate learning classifiers for target categories when there is only one training examples per target category (one-shot learning).

Zero-Shot Learning Through Cross-Modal Transfer

TLDR
This work introduces a model that can recognize objects in images even if no training data is available for the object class, and uses novelty detection methods to differentiate unseen classes from seen classes.

DeViSE: A Deep Visual-Semantic Embedding Model

TLDR
This paper presents a new deep visual-semantic embedding model trained to identify visual objects using both labeled image data as well as semantic information gleaned from unannotated text and shows that the semantic information can be exploited to make predictions about tens of thousands of image labels not observed during training.

Learning to detect unseen object classes by between-class attribute transfer

TLDR
The experiments show that by using an attribute layer it is indeed possible to build a learning object detection system that does not require any training images of the target classes, and assembled a new large-scale dataset, “Animals with Attributes”, of over 30,000 animal images that match the 50 classes in Osherson's classic table of how strongly humans associate 85 semantic attributes with animal classes.

Label Embedding Trees for Large Multi-Class Tasks

TLDR
An algorithm for learning a tree-structure of classifiers which, by optimizing the overall tree loss, provides superior accuracy to existing tree labeling methods and a method that learns to embed labels in a low dimensional space that is faster than non-embedding approaches and has superior accuracyto existing embedding approaches are proposed.

Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost

TLDR
The goal is to devise classifiers which can incorporate images and classes on-the-fly at (near) zero cost and to explore k-nearest neighbor (k-NN) and nearest class mean (NCM) classifiers.

Designing Category-Level Attributes for Discriminative Visual Recognition

TLDR
A novel formulation to automatically design discriminative "category-level attributes", which can be efficiently encoded by a compact category-attribute matrix, which allows to achieve intuitive and critical design criteria (category-separability, learn ability) in a principled way.

Augmented Attribute Representations

TLDR
A new learning method is proposed to infer a mid-level feature representation that combines the advantage of semantic attribute representations with the higher expressive power of non-semantic features with better results in terms of object categorization accuracy than the semantic representation alone.
...