Corpus ID: 53509466

LATENT EMBEDDING OPTIMIZATION

@inproceedings{Rao2018LATENTEO,
  title={LATENT EMBEDDING OPTIMIZATION},
  author={Dushyant Rao and Jakub Sygnowski and Oriol Vinyals and Razvan Pascanu and Simon Osindero},
  year={2018}
}
Gradient-based meta-learning techniques are both widely applicable and proficient at solving challenging few-shot learning and fast adaptation problems. However, they have practical difficulties when operating on high-dimensional parameter spaces in extreme low-data regimes. We show that it is possible to bypass these limitations by learning a data-dependent latent generative representation of model parameters, and performing gradient-based meta-learning in this lowdimensional latent space. The… Expand

Figures and Tables from this paper

References

SHOWING 1-10 OF 51 REFERENCES
Optimization as a Model for Few-Shot Learning
beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework
Learning an interpretable factorised representation of the independent data generative factors of the world without supervision is an important precursor for the development of artificialExpand
Meta-learning autoencoders for few-shot prediction
TLDR
It is demonstrated that for previously unseen tasks, without additional training, this Meta-Learning Autoencoder (MeLA) framework can build models that closely match the true underlying models, with loss significantly lower than given by fine-tuned baseline networks and performance that compares favorably with state-of-the-art meta-learning algorithms. Expand
Transductive Propagation Network for Few-shot Learning
TLDR
This paper proposes Transductive Propagation Network (TPN), a transductive method that classifies the entire test set at once to alleviate the low-data problem and explicitly learns an underlying manifold space that is appropriate to propagate labels from few-shot examples. Expand
Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace
TLDR
This work demonstrates that the dimension of this learned subspace reflects the complexity of the task-specific learner's adaptation task, and also that the model is less sensitive to the choice of initial learning rates than previous gradient-based meta-learning methods. Expand
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learningExpand
Probabilistic Model-Agnostic Meta-Learning
TLDR
This paper proposes a probabilistic meta-learning algorithm that can sample models for a new task from a model distribution that is trained via a variational lower bound, and shows how reasoning about ambiguity can also be used for downstream active learning problems. Expand
TADAM: Task dependent adaptive metric for improved few-shot learning
TLDR
This work identifies that metric scaling and metric task conditioning are important to improve the performance of few-shot algorithms and proposes and empirically test a practical end-to-end optimization procedure based on auxiliary task co-training to learn a task-dependent metric space. Expand
Uncertainty in Multitask Transfer Learning
TLDR
Using variational Bayes neural networks, this work develops an algorithm capable of accumulating knowledge into a prior capable of few-shot learning on new tasks and provides experiments showing that other existing methods can fail to perform well in different benchmarks. Expand
Deep Meta-Learning: Learning to Learn in the Concept Space
TLDR
By learning to learn in the concept space rather than in the complicated instance space, deep meta- learning can substantially improve vanilla meta-learning, which is demonstrated on various few-shot image recognition problems. Expand
...
1
2
3
4
5
...