Label-Embedding for Image Classification

@article{Akata2016LabelEmbeddingFI,
  title={Label-Embedding for Image Classification},
  author={Zeynep Akata and Florent Perronnin and Za{\"i}d Harchaoui and Cordelia Schmid},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2016},
  volume={38},
  pages={1425-1438}
}
Attributes act as intermediate representations that enable parameter sharing between classes, a must when training data is scarce. We propose to view attribute-based image classification as a label-embedding problem: each class is embedded in the space of attribute vectors. We introduce a function that measures the compatibility between an image and a label embedding. The parameters of this function are learned on a training set of labeled samples to ensure that, given an image, the correct… 

Figures and Tables from this paper

Semantic Diversity Learning for Zero-Shot Multi-label Classification
TLDR
This study introduces an end-to-end model training for multi-label zero-shot learning that supports the semantic diversity of the images and labels and proposes to use an embedding matrix having principal embedding vectors trained using a tailored loss function.
Structurally Constrained Correlation Transfer for Zero-shot Learning
TLDR
A novel zero-shot learning model is proposed that forms a neighborhood-preserving structure in the semantic embedding space and utilize it to predict classifiers for unseen classes and generates effective semantic representations and out-performs state-of-the-art methods.
Attribute Prototype Network for Zero-Shot Learning
TLDR
A novel zero-shot representation learning framework that jointly learns discriminative global and local features using only class-level attributes and points to the visual evidence of the attributes in an image, confirming the improved attribute localization ability of the image representation.
Attribute Prediction in the Zero-Shot Setting as Multiple Instance Learning
  • Computer Science
  • 2022
TLDR
This work applies the Multiple Instance Learning (MIL) paradigm to attribute learning (AMIL) while only using class-level labels and proposes MIL-DAP, an attribute-based zero-shot classification method, based on Direct Attribute Prediction (DAP), to evaluate attribute prediction methods when no image-level data is available for evaluation.
Zero-Shot Image Classification Using Coupled Dictionary Embedding
A Simple Approach for Zero-Shot Learning based on Triplet Distribution Embeddings
TLDR
This work addresses the issue of expressivity in terms of modeling the intra-class variability for each class in Zero-Shot Learning by leveraging the use of distribution embeddings, which are modeled as Gaussian distributions.
A Joint Label Space for Generalized Zero-Shot Classification
TLDR
This paper proposes a novel pathway that uses the label space to jointly reconcile visual and semantic spaces directly, which is named Attributing Label Space (ALS).
Improving Semantic Embedding Consistency by Metric Learning for Zero-Shot Classiffication
TLDR
The proposed approach is to control the semantic embedding of images -- one of the main ingredients of zero-shot learning -- by formulating it as a metric learning problem, and gives state-of-the-art results on four challenging datasets used for zero- shot recognition evaluation.
Evaluation of output embeddings for fine-grained image classification
TLDR
This project shows that compelling classification performance can be achieved on fine-grained categories even without labeled training data, and establishes a substantially improved state-of-the-art on the Animals with Attributes and Caltech-UCSD Birds datasets.
Cross-Modal Mapping for Generalized Zero-Shot Learning by Soft-Labeling
TLDR
A novel architecture of casting ZSL as a standard neuralnetwork with cross-entropy loss to embed visual space to semantic space to link visual and semantic spaces in a cross-modal transfer/embedding space is proposed.
...
...

References

SHOWING 1-10 OF 79 REFERENCES
Label-Embedding for Attribute-Based Classification
TLDR
This work proposes to view attribute-based image classification as a label-embedding problem: each class is embedded in the space of attribute vectors, and introduces a function which measures the compatibility between an image and a label embedding.
Evaluation of output embeddings for fine-grained image classification
TLDR
This project shows that compelling classification performance can be achieved on fine-grained categories even without labeled training data, and establishes a substantially improved state-of-the-art on the Animals with Attributes and Caltech-UCSD Birds datasets.
A joint learning framework for attribute models and object descriptions
TLDR
By incorporating class information into the attribute classifier learning, this work gets an attribute-level representation that generalizes well to both unseen examples of known classes and unseen classes.
Attribute-Based Transfer Learning for Object Categorization with Zero/One Training Example
TLDR
This paper builds a generative attribute model to learn the probabilistic distributions of image features for each attribute, which can be used to classify unseen images of target categories or facilitate learning classifiers for target categories when there is only one training examples per target category (one-shot learning).
Zero-Shot Learning Through Cross-Modal Transfer
TLDR
This work introduces a model that can recognize objects in images even if no training data is available for the object class, and uses novelty detection methods to differentiate unseen classes from seen classes.
DeViSE: A Deep Visual-Semantic Embedding Model
TLDR
This paper presents a new deep visual-semantic embedding model trained to identify visual objects using both labeled image data as well as semantic information gleaned from unannotated text and shows that the semantic information can be exploited to make predictions about tens of thousands of image labels not observed during training.
Learning to detect unseen object classes by between-class attribute transfer
TLDR
The experiments show that by using an attribute layer it is indeed possible to build a learning object detection system that does not require any training images of the target classes, and assembled a new large-scale dataset, “Animals with Attributes”, of over 30,000 animal images that match the 50 classes in Osherson's classic table of how strongly humans associate 85 semantic attributes with animal classes.
Label Embedding Trees for Large Multi-Class Tasks
TLDR
An algorithm for learning a tree-structure of classifiers which, by optimizing the overall tree loss, provides superior accuracy to existing tree labeling methods and a method that learns to embed labels in a low dimensional space that is faster than non-embedding approaches and has superior accuracyto existing embedding approaches are proposed.
Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost
TLDR
The goal is to devise classifiers which can incorporate images and classes on-the-fly at (near) zero cost and to explore k-nearest neighbor (k-NN) and nearest class mean (NCM) classifiers.
Designing Category-Level Attributes for Discriminative Visual Recognition
TLDR
A novel formulation to automatically design discriminative "category-level attributes", which can be efficiently encoded by a compact category-attribute matrix, which allows to achieve intuitive and critical design criteria (category-separability, learn ability) in a principled way.
...
...