Corpus ID: 202712906

Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML

@article{Raghu2020RapidLO,
  title={Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML},
  author={Aniruddh Raghu and M. Raghu and S. Bengio and Oriol Vinyals},
  journal={ArXiv},
  year={2020},
  volume={abs/1909.09157}
}
  • Aniruddh Raghu, M. Raghu, +1 author Oriol Vinyals
  • Published 2020
  • Mathematics, Computer Science
  • ArXiv
  • An important research direction in machine learning has centered around developing meta-learning algorithms to tackle few-shot learning. An especially successful algorithm has been Model Agnostic Meta-Learning (MAML), a method that consists of two optimization loops, with the outer loop finding a meta-initialization, from which the inner loop can efficiently learn new tasks. Despite MAML's popularity, a fundamental open question remains -- is the effectiveness of MAML due to the meta… CONTINUE READING
    Meta-Learning in Neural Networks: A Survey
    • 38
    • PDF
    Decoupling Adaptation from Modeling with Meta-Optimizers for Meta Learning
    • 2
    Few-Shot Learning via Learning the Representation, Provably
    • 14
    • PDF
    Model-Agnostic Meta-Learning using Runge-Kutta Methods
    • 2
    • Highly Influenced
    • PDF
    Modular Meta-Learning with Shrinkage
    • 5
    • PDF
    Does MAML really want feature reuse only?
    Meta-learnt priors slow down catastrophic forgetting in neural networks
    • 3
    • PDF
    Concept Learners for Generalizable Few-Shot Learning
    • 1
    • PDF
    Side-Tuning: Network Adaptation via Additive Side Networks
    • 5
    • PDF

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 49 REFERENCES
    Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
    • 2,268
    • Highly Influential
    • PDF
    Optimization as a Model for Few-Shot Learning
    • 1,184
    Matching Networks for One Shot Learning
    • 1,857
    • PDF
    On First-Order Meta-Learning Algorithms
    • 434
    • PDF
    Learning to Compare: Relation Network for Few-Shot Learning
    • 783
    • Highly Influential
    • PDF
    Prototypical Networks for Few-shot Learning
    • 1,495
    • PDF
    Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm
    • 110
    • PDF
    How to train your MAML
    • 124
    • PDF