A Survey of Deep Meta-Learning

@article{Huisman2021ASO,
  title={A Survey of Deep Meta-Learning},
  author={Mike Huisman and Jan N. van Rijn and Aske Plaat},
  journal={Artif. Intell. Rev.},
  year={2021},
  volume={54},
  pages={4483-4541}
}
Deep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Meta-learning is one approach to address this issue, by enabling the network to learn how to learn. The field of Deep Meta-Learning advances at great speed, but lacks a unified, in-depth overview of current techniques. With this work, we aim to bridge this gap. After providing the reader with a theoretical… 

A Survey on Deep Semi-supervised Learning

A taxonomy for deep semi-supervised learning is presented that categorizes existing methods, including deep generative methods, consistency regularization methods, graph-based methods, pseudo-labeling methods, and hybrid methods and offers a detailed comparison of these methods in terms of the type of losses, contributions, and architecture differences.

Multi-Objective Meta Learning

This paper proposes a unified gradient-based Multi-Objective Meta Learning (MOML) framework and devise the first gradient-Based optimization algorithm to solve the MOBLP by alternatively solving the lower-level and upper-level subproblems via the gradient descent method and the gradient- based multi-objective optimization method, respectively.

Deep Meta-learning in Recommendation Systems: A Survey

A taxonomy is proposed to discuss existing methods according to recommendation scenarios, meta- learning techniques, and meta-knowledge representations, which could provide the design space for meta-learning based recommendation methods.

Principal Component Analysis for Gaussian Process Posteriors

This study reduces the infiniteness of GP to the finite-dimensional case under the information geometrical framework by considering a space of GP posteriors that have the same prior.

Reinforcement learning method for machining deformation control based on meta-invariant feature space

Precise control of machining deformation is crucial for improving the manufacturing quality of structural aerospace components. In the machining process, different batches of blanks have different

A new benchmark for group distribution shifts in hand grasp regression for object manipulation. Can meta-learning raise the bar?

A novel benchmark for object group distribution shifts in hand and object pose regression for object grasping is proposed and the hypothesis that meta-learning a baseline pose regression neural network can adapt to these shifts and generalize better to unknown objects is tested.

SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition

A seminal span-based prototypical network (SpanProto) that tackles few-shot NER via a two-stage approach, including span extraction and mention classification, which outperforms strong baselines by a large margin.

Multi-Task Meta Learning: learn how to adapt to unseen tasks

This work proposes Multi-task Meta Learning, integrating two learning paradigms Multi-Task Learning ( MTL ) and meta learning, to bring together the best of both worlds and achieves state-of-the-art results for most of the tasks.

Neural network based image classifier resilient to destructive perturbation influences – architecture and training method

A model and method of training a classifier that exhibits characteristics of resilience to adversarial attacks, fault injection, and concept drift is developed and the ability to adapt to the appearance of new classes or the concept drift has been confirmed 8 times faster than learning from scratch.
...

References

SHOWING 1-10 OF 99 REFERENCES

Meta-Learning in Neural Networks: A Survey

A new taxonomy is proposed that provides a more comprehensive breakdown of the space of meta-learning methods today and surveys promising applications and successes ofMeta-learning such as few-shot learning and reinforcement learning.

A Simple Neural Attentive Meta-Learner

This work proposes a class of simple and generic meta-learner architectures that use a novel combination of temporal convolutions and soft attention; the former to aggregate information from past experience and the latter to pinpoint specific pieces of information.

Meta-learning with differentiable closed-form solvers

The main idea is to teach a deep network to use standard machine learning tools, such as ridge regression, as part of its own internal model, enabling it to quickly adapt to novel data.

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning

Optimization as a Model for Few-Shot Learning

Meta-Learning: A Survey

This chapter provides an overview of the state of the art in meta-learning, the science of systematically observing how different machine learning approaches perform on a wide range of learning tasks and then learning from this experience, or meta-data, to learn new tasks much faster than otherwise possible.

How to train your MAML

This paper proposes various modifications to MAML that not only stabilize the system, but also substantially improve the generalization performance, convergence speed and computational overhead of MAMl, which it is called M AML++.

Meta Networks

A novel meta learning method, Meta Networks (MetaNet), is introduced that learns a meta-level knowledge across tasks and shifts its inductive biases via fast parameterization for rapid generalization.

Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm

This paper finds that deep representation combined with standard gradient descent have sufficient capacity to approximate any learning algorithm, and finds that gradient-based meta-learning consistently leads to learning strategies that generalize more widely compared to those represented by recurrent models.

Meta-Learning with Memory-Augmented Neural Networks

The ability of a memory-augmented neural network to rapidly assimilate new data, and leverage this data to make accurate predictions after only a few samples is demonstrated.
...