Corpus ID: 29153681

Meta-learning with differentiable closed-form solvers

@article{Bertinetto2019MetalearningWD,
  title={Meta-learning with differentiable closed-form solvers},
  author={Luca Bertinetto and Jo{\~a}o F. Henriques and Philip H. S. Torr and A. Vedaldi},
  journal={ArXiv},
  year={2019},
  volume={abs/1805.08136}
}
Adapting deep networks to new concepts from a few examples is challenging, due to the high computational requirements of standard fine-tuning procedures. Most work on few-shot learning has thus focused on simple learning techniques for adaptation, such as nearest neighbours or gradient descent. Nonetheless, the machine learning literature contains a wealth of methods that learn non-deep models very efficiently. In this paper, we propose to use these fast convergent methods as the main… Expand
[Re] Meta learning with differentiable closed-form solvers
TLDR
Meta-learning, also known as learning-to-learn, is a paradigm that exploits cross-task information and training experience to perform well on a new unseen task. Expand
MetaKernel: Learning Variational Random Features with Limited Labels
TLDR
This paper proposes meta-learning kernels with random Fourier features for few-shot learning, called MetaKernel, which consistently delivers at least comparable and often better performance than state-of-the-art alternatives. Expand
B-Small: A Bayesian Neural Network Approach to Sparse Model-Agnostic Meta-Learning
  • Anish Madan, Ranjitha Prasad
  • Computer Science
  • ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2021
TLDR
The proposed framework incorporates a sparse variational loss term alongside the loss function of MAML, which uses a sparsifying approximated KL divergence as a regularizer, and demonstrates applicability of the approach in distributed sensor networks, where sparsity and meta-learning can be beneficial. Expand
Learning Features with Differentiable Closed-Form Solver for Tracking
TLDR
A novel and easy-to-implement training framework for visual tracking which can generalize well to the trackers based on online discriminatively trained ridge regression model and modify the origin shrinkage loss and then employ it as the loss function for efficient and effective training. Expand
Meta-Learning With Differentiable Convex Optimization
TLDR
The objective is to learn feature embeddings that generalize well under a linear classification rule for novel categories and this work exploits two properties of linear classifiers: implicit differentiation of the optimality conditions of the convex problem and the dual formulation of the optimization problem. Expand
Meta-Learning of Structured Representation by Proximal Mapping
Underpinning the success of deep learning are the effective regularization techniques that allow a broad range of structures in data to be compactly modeled in a deep architecture. Examples includeExpand
Meta-Transfer Learning through Hard Tasks
TLDR
This work proposes a novel approach called meta-transfer learning (MTL), which learns to transfer the weights of a deep NN for few-shot learning tasks, and introduces the hard task (HT) meta-batch scheme as an effective learning curriculum of few- shot classification tasks. Expand
Deep Kernel Transfer in Gaussian Processes for Few-shot Learning
TLDR
This work proposes a simple, yet effective variant of deep kernel learning in which the kernel is transferred across tasks, which it is called deep kernel transfer, and demonstrates that the proposed method outperforms several state-of-the-art algorithms in few-shot regression, classification, and cross-domain adaptation. Expand
Learning to Forget for Meta-Learning
TLDR
This work proposes task-and-layer-wise attenuation on the compromised initialization of model-agnostic meta-learning to reduce its influence and names the method as L2F (Learn to Forget). Expand
Decoupling Adaptation from Modeling with Meta-Optimizers for Meta Learning
TLDR
This work begins with an experimental analysis of MAML, finding that deep models are crucial for its success, even given sets of simple tasks where a linear model would suffice on any individual task. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 72 REFERENCES
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learningExpand
Optimization as a Model for Few-Shot Learning
Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
TLDR
This paper finds that deep representation combined with standard gradient descent have sufficient capacity to approximate any learning algorithm, and finds that gradient-based meta-learning consistently leads to learning strategies that generalize more widely compared to those represented by recurrent models. Expand
A Simple Neural Attentive Meta-Learner
TLDR
This work proposes a class of simple and generic meta-learner architectures that use a novel combination of temporal convolutions and soft attention; the former to aggregate information from past experience and the latter to pinpoint specific pieces of information. Expand
Learning feed-forward one-shot learners
TLDR
This paper constructs the learner as a second deep network, called a learnet, which predicts the parameters of a pupil network from a single exemplar, and obtains an efficient feed-forward one-shot learner, trained end-to-end by minimizing a one- shot classification objective in a learning to learn formulation. Expand
Memory-based Parameter Adaptation
TLDR
Memory-based Parameter Adaptation alleviates several shortcomings of neural networks, such as catastrophic forgetting, fast, stable acquisition of new knowledge, learning with an imbalanced class labels, and fast learning during evaluation. Expand
Matching Networks for One Shot Learning
TLDR
This work employs ideas from metric learning based on deep neural features and from recent advances that augment neural networks with external memories to learn a network that maps a small labelled support set and an unlabelled example to its label, obviating the need for fine-tuning to adapt to new class types. Expand
Siamese Neural Networks for One-Shot Image Recognition
TLDR
A method for learning siamese neural networks which employ a unique structure to naturally rank similarity between inputs and is able to achieve strong results which exceed those of other deep learning models with near state-of-the-art performance on one-shot classification tasks. Expand
Learning to Compare: Relation Network for Few-Shot Learning
TLDR
A conceptually simple, flexible, and general framework for few-shot learning, where a classifier must learn to recognise new classes given only few examples from each, which is easily extended to zero- shot learning. Expand
On First-Order Meta-Learning Algorithms
TLDR
A family of algorithms for learning a parameter initialization that can be fine-tuned quickly on a new task, using only first-order derivatives for the meta-learning updates, including Reptile, which works by repeatedly sampling a task, training on it, and moving the initialization towards the trained weights on that task. Expand
...
1
2
3
4
5
...