• Corpus ID: 44061218

TADAM: Task dependent adaptive metric for improved few-shot learning

@inproceedings{Oreshkin2018TADAMTD,
  title={TADAM: Task dependent adaptive metric for improved few-shot learning},
  author={Boris N. Oreshkin and Pau Rodr{\'i}guez L{\'o}pez and Alexandre Lacoste},
  booktitle={NeurIPS},
  year={2018}
}
Few-shot learning has become essential for producing models that generalize from few examples. In this work, we identify that metric scaling and metric task conditioning are important to improve the performance of few-shot algorithms. Our analysis reveals that simple metric scaling completely changes the nature of few-shot algorithm parameter updates. Metric scaling provides improvements up to 14% in accuracy for certain metrics on the mini-Imagenet 5-way 5-shot classification task. We further… 

Figures and Tables from this paper

Learning Instance and Task-Aware Dynamic Kernels for Few Shot Learning
TLDR
This paper proposes to learn the dynamic kernels of a convolution network as a function of the task at hand, enabling faster generalization and empirically shows that this model improves performance on few-shot classification and detection tasks, achieving a tangible improvement over several baseline models.
Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning
TLDR
A Transductive Episodic-wise Adaptive Metric (TEAM) framework for few-shot learning is proposed, by integrating the meta-learning paradigm with both deep metric learning and transductive inference and leverages an attention-based bi-directional similarity strategy for extracting the more robust relationship between queries and prototypes.
ReMP: Rectified Metric Propagation for Few-Shot Learning
TLDR
The resulting approach, called rectified metric propagation (ReMP), further optimizes an attentive prototype propagation network, and applies a repulsive force to make confident predictions.
TAAN: Task-Aware Attention Network for Few-shot Classification
  • Zhe Wang, Li Liu, Fanzhang Li
  • Computer Science
    2020 25th International Conference on Pattern Recognition (ICPR)
  • 2021
TLDR
This work proposes a novel algorithm called Task-Aware Attention Network (TAAN), which generates channel attentions for each sample by aggregating the context of the entire support set and identifies the most relevant features for similarity comparison in metric-based few-shot learners.
Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning
TLDR
A simple process: meta-learning over a whole-classification pre-trained model on its evaluation metric achieves competitive performance to state-of-the-art methods on standard bench-marks and sheds some light on understanding the trade-offs between the meta- learning objective and the whole- classification objective in few-shot learning.
Task-Adaptive Clustering for Semi-Supervised Few-Shot Classification
TLDR
This work proposes a few-shot learner that can work well under the semi-supervised setting where a large portion of training data is unlabeled, and introduces a concept of controlling the degree of task-conditioning for meta-learning.
Trainable Class Prototypes for Few-Shot Learning
TLDR
This paper proposes the trainable prototypes for distance measure instead of the artificial ones within the meta-training and task-training framework and adopts non-episodic meta- training based on self-supervised learning.
A New Meta-Baseline for Few-Shot Learning
TLDR
This work presents a Meta-Baseline method, by pre-training a classifier on all base classes and meta-learning on a nearest-centroid based few-shot classification algorithm, which outperforms recent state-of-the-art methods by a large margin.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 38 REFERENCES
Learning to Compare: Relation Network for Few-Shot Learning
TLDR
A conceptually simple, flexible, and general framework for few-shot learning, where a classifier must learn to recognise new classes given only few examples from each, which is easily extended to zero- shot learning.
Optimization as a Model for Few-Shot Learning
Meta-Learning for Semi-Supervised Few-Shot Classification
TLDR
This work proposes novel extensions of Prototypical Networks that are augmented with the ability to use unlabeled examples when producing prototypes, and confirms that these models can learn to improve their predictions due to unlabeling examples, much like a semi-supervised algorithm would.
Matching Networks for One Shot Learning
TLDR
This work employs ideas from metric learning based on deep neural features and from recent advances that augment neural networks with external memories to learn a network that maps a small labelled support set and an unlabelled example to its label, obviating the need for fine-tuning to adapt to new class types.
Prototypical Networks for Few-shot Learning
TLDR
This work proposes Prototypical Networks for few-shot classification, and provides an analysis showing that some simple design decisions can yield substantial improvements over recent approaches involving complicated architectural choices and meta-learning.
Discriminative k-shot learning using probabilistic models
TLDR
It is shown that even a simple probabilistic model achieves state-of-the-art on a standard k-shot learning dataset by a large margin and is able to accurately model uncertainty, leading to well calibrated classifiers, and is easily extensible and flexible, unlike many recent approaches to k- shot learning.
Siamese Neural Networks for One-Shot Image Recognition
TLDR
A method for learning siamese neural networks which employ a unique structure to naturally rank similarity between inputs and is able to achieve strong results which exceed those of other deep learning models with near state-of-the-art performance on one-shot classification tasks.
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning
Low-Shot Learning from Imaginary Data
TLDR
This work builds on recent progress in meta-learning by combining a meta-learner with a "hallucinator" that produces additional training examples, and optimizing both models jointly, yielding state-of-the-art performance on the challenging ImageNet low-shot classification benchmark.
Learning Visual Reasoning Without Strong Priors
TLDR
This work shows that a general-purpose, Conditional Batch Normalization approach achieves state-of-the-art results on the CLEVR Visual Reasoning benchmark with a 2.4% error rate, and probes the model to shed light on how it reasons, showing it has learned a question-dependent, multi-step process.
...
1
2
3
4
...