Corpus ID: 221341010

learn2learn: A Library for Meta-Learning Research

@article{Arnold2020learn2learnAL,
  title={learn2learn: A Library for Meta-Learning Research},
  author={S{\'e}bastien M. R. Arnold and Praateek Mahajan and Debajyoti Datta and Ian Bunner and Konstantinos Saitas Zarkias},
  journal={ArXiv},
  year={2020},
  volume={abs/2008.12284}
}
Meta-learning researchers face two fundamental issues in their empirical work: prototyping and reproducibility. Researchers are prone to make mistakes when prototyping new algorithms and tasks because modern meta-learning methods rely on unconventional functionalities of machine learning frameworks. In turn, reproducing existing results becomes a tedious endeavour -- a situation exacerbated by the lack of standardized implementations and benchmarks. As a result, researchers spend inordinate… Expand
A Channel Coding Benchmark for Meta-Learning
TLDR
This work proposes the channel coding problem as a benchmark for meta- learning and uses this benchmark to study several aspects of meta-learning, including the impact of task distribution breadth and shift, which can be controlled in the coding problem. Expand
A Channel Coding Benchmark for Meta-Learning
  • Rui Li
  • 2021
Meta-learning provides a popular and effective family of methods for data-efficient learning of new tasks. However, several important issues in meta-learning have proven hard to study thus far. ForExpand
Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation
TLDR
This work proves that for over-parameterized neural networks with sufficient depth, the learned predictive functions of MTL and GBML are close, and corroborates the theoretical findings by showing that, with proper implementation, MTL is competitive against state-of-the-art GBML algorithms on a set of few-shot image classification benchmarks. Expand
protANIL: a Fast and Simple Meta-Learning Algorithm
A broad recognition of important practical benefits inherent to meta-learning paradigm has recently elevated the few-shot learning problem into the spotlight of machine learning research. While theExpand
Embedding Adaptation is Still Needed for Few-Shot Learning
TLDR
This work proposes ATG, a principled clustering method to defining train and test tasksets without additional human knowledge, and empirically demonstrates the effectiveness of ATG in generating tasksets that are easier, in-between, or harder than existing benchmarks, including those that rely on semantic information. Expand
Minimax and Neyman–Pearson Meta-Learning for Outlier Languages
TLDR
Two variants of MAML are created based on alternative criteria that reduce the maximum risk across languages, while Neyman–Pearson MAMl constrains the risk in each language to a maximum threshold, which constitute fully differentiable two-player games. Expand
On sensitivity of meta-learning to support data
TLDR
It is demonstrated the existence of (unaltered, in-distribution, natural) images that, when used for adaptation, yield accuracy as low as 4% or as high as 95% on standard few-shot image classification benchmarks, suggesting that robust and safe meta-learning requires larger margins than supervised learning. Expand
Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stability
TLDR
A probably approximately correct (PAC) bound is derived for gradient-based metalearning using two different generalization frameworks in order to deal with the qualitatively different challenges of generalization at the “base” and “meta” levels. Expand
Uniform Sampling over Episode Difficulty
TLDR
This paper proposes a method to approximate episode sampling distributions based on their difficulty and finds that sampling uniformly over episode difficulty outperforms other sampling schemes, including curriculum and easy-/hard-mining. Expand
Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGD
  • Chen Fan, P. Ram, Sijia Liu
  • Computer Science
  • ArXiv
  • 2021
TLDR
Sign-MAML is theoretically-grounded as it does not impose any assumption on the absence of second-order derivatives during meta training, and compared to MAML, it achieves a much more graceful tradeoff between classification accuracy and computation efficiency. Expand
...
1
2
...

References

SHOWING 1-10 OF 36 REFERENCES
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
TLDR
The ANIL (Almost No Inner Loop) algorithm is proposed, a simplification of MAML where the inner loop is removed for all but the (task-specific) head of a MAMl-trained network, and performance on the test tasks is entirely determined by the quality of the learned features, and one can remove even the head of the network (the NIL algorithm). Expand
Meta-SGD: Learning to Learn Quickly for Few Shot Learning
TLDR
Meta-SGD, an SGD-like, easily trainable meta-learner that can initialize and adapt any differentiable learner in just one step, shows highly competitive performance for few-shot learning on regression, classification, and reinforcement learning. Expand
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learningExpand
Torchmeta: A Meta-Learning library for PyTorch
The constant introduction of standardized benchmarks in the literature has helped accelerating the recent advances in meta-learning research. They offer a way to get a fair comparison betweenExpand
Meta-learning with differentiable closed-form solvers
TLDR
The main idea is to teach a deep network to use standard machine learning tools, such as ridge regression, as part of its own internal model, enabling it to quickly adapt to novel data. Expand
Meta-Learning With Differentiable Convex Optimization
TLDR
The objective is to learn feature embeddings that generalize well under a linear classification rule for novel categories and this work exploits two properties of linear classifiers: implicit differentiation of the optimality conditions of the convex problem and the dual formulation of the optimization problem. Expand
Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning
TLDR
This work uses meta-learning to train a dynamics model prior such that, when combined with recent data, this prior can be rapidly adapted to the local context and demonstrates the importance of incorporating online adaptation into autonomous agents that operate in the real world. Expand
Benchmarking Deep Reinforcement Learning for Continuous Control
TLDR
This work presents a benchmark suite of continuous control tasks, including classic tasks like cart-pole swing-up, tasks with very high state and action dimensionality such as 3D humanoid locomotion, task with partial observations, and tasks with hierarchical structure. Expand
Matching Networks for One Shot Learning
TLDR
This work employs ideas from metric learning based on deep neural features and from recent advances that augment neural networks with external memories to learn a network that maps a small labelled support set and an unlabelled example to its label, obviating the need for fine-tuning to adapt to new class types. Expand
ProMP: Proximal Meta-Policy Search
TLDR
A novel meta-learning algorithm is developed that overcomes both the issue of poor credit assignment and previous difficulties in estimating meta-policy gradients and leads to superior pre-adaptation policy behavior and consistently outperforms previous Meta-RL algorithms in sample-efficiency, wall-clock time, and asymptotic performance. Expand
...
1
2
3
4
...