# MetaFun: Meta-Learning with Iterative Functional Updates

@inproceedings{Xu2020MetaFunMW, title={MetaFun: Meta-Learning with Iterative Functional Updates}, author={Jin Xu and Jean-Francois Ton and Hyunjik Kim and Adam R. Kosiorek and Yee Whye Teh}, booktitle={ICML}, year={2020} }

We develop a functional encoder-decoder approach to supervised meta-learning, where labeled data is encoded into an infinite-dimensional functional representation rather than a finite-dimensional one. Furthermore, rather than directly producing the representation, we learn a neural update rule resembling functional gradient descent which iteratively improves the representation. The final representation is used to condition the decoder to make predictions on unlabeled data. Our approach is the…

## Figures, Tables, and Topics from this paper

## 22 Citations

Meta Learning for Causal Direction

- Mathematics, Computer ScienceAAAI
- 2021

This paper introduces a novel generative model that allows distinguishing cause and effect in the bivariate setting under limited observational data and proposes an end-to-end algorithm that makes use of similar training datasets at test time.

Amortized Bayesian Prototype Meta-learning: A New Probabilistic Meta-learning Approach to Few-shot Image Classification

- Computer ScienceAISTATS
- 2021

This paper proposes a novel probabilistic metalearning method called amortized Bayesian prototype meta-learning, which learns to learn the posterior distributions of these latent prototypes in anAmortized inference way with no need for an extra amortization network, such that they can easily approximate their posteriors conditional on few labeled samples at meta-training or meta-testing stage.

Few-shot Learning for Topic Modeling

- Computer Science, MathematicsArXiv
- 2021

A neural network-based few-shot learning method that can learn a topic model from just a few documents using a set of multiple text corpora with an episodic training framework is proposed.

Function Contrastive Learning of Transferable Meta-Representations

- Computer Science, MathematicsICML
- 2021

A decoupled encoder-decoder approach to supervised meta-learning, where the encoder is trained with a contrastive objective to find a good representation of the underlying function and the representations obtained outperform strong baselines in terms of downstream performance and noise robustness.

Gaussian Process Meta Few-shot Classifier Learning via Linear Discriminant Laplace Approximation

- Computer ScienceArXiv
- 2021

This work considers the Bayesian Gaussian process (GP) approach, in which the meta-learns the GP prior, and the adaptation to a new task is carried out by the GP predictive model from the posterior inference.

Generative vs Discriminative: Rethinking The Meta-Continual Learning

- 2021

Deep neural networks have achieved human-level capabilities in various learning tasks. However, they generally lose performance in more realistic scenarios like learning in a continual manner. In…

Group Equivariant Conditional Neural Processes

- Computer Science, MathematicsICLR
- 2021

A decomposition theorem for permutation-invariant and group-equivariant maps is given, which leads to construct EquivCNPs with an infinite-dimensional latent space to handle group symmetries and shows that EquivCNP with translation equivariance achieves comparable performance to conventional CNPs in a 1D regression task.

Hierarchical Few-Shot Generative Models

- Computer Science, MathematicsArXiv
- 2021

This work generalizes deep latent variable approaches to few-shot learning, taking a step towards large-scale few- shot generation with a formulation that readily can work with current state-of-the-art deep generative models.

Learning to Rectify for Robust Learning with Noisy Labels

- Computer SciencePattern Recognition
- 2021

W warped probabilistic inference (WarPI) is proposed to achieve adaptively rectifying the training procedure for the classification network within the meta-learning scenario, demonstrating a significant improvement of the generalization ability.

Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module

- Computer ScienceArXiv
- 2021

A meta reinforcement learning ( meta-RL) framework is proposed to tackle the problem of effective communication mechanism among agents in reinforcement learning and employs a meta-learned Communication Pattern Recognition module to identify communication behavior and extract information that facilitates the training process.

## References

SHOWING 1-10 OF 47 REFERENCES

Meta-Learning with Latent Embedding Optimization

- Mathematics, Computer ScienceICLR
- 2019

This work shows that latent embedding optimization can achieve state-of-the-art performance on the competitive miniImageNet and tieredImageNet few-shot classification tasks, and indicates LEO is able to capture uncertainty in the data, and can perform adaptation more effectively by optimizing in latent space.

Meta-Learning for Semi-Supervised Few-Shot Classification

- Computer Science, MathematicsICLR
- 2018

This work proposes novel extensions of Prototypical Networks that are augmented with the ability to use unlabeled examples when producing prototypes, and confirms that these models can learn to improve their predictions due to unlabeling examples, much like a semi-supervised algorithm would.

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

- Computer ScienceICML
- 2017

We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning…

Matching Networks for One Shot Learning

- Computer Science, MathematicsNIPS
- 2016

This work employs ideas from metric learning based on deep neural features and from recent advances that augment neural networks with external memories to learn a network that maps a small labelled support set and an unlabelled example to its label, obviating the need for fine-tuning to adapt to new class types.

Meta-Learning With Differentiable Convex Optimization

- Computer Science2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2019

The objective is to learn feature embeddings that generalize well under a linear classification rule for novel categories and this work exploits two properties of linear classifiers: implicit differentiation of the optimality conditions of the convex problem and the dual formulation of the optimization problem.

Optimization as a model for fewshot learning

- In International Conference on Learning Representations,
- 2016

Attentive Neural Processes

- Computer Science, MathematicsICLR
- 2019

Attention is incorporated into NPs, allowing each input location to attend to the relevant context points for the prediction, which greatly improves the accuracy of predictions, results in noticeably faster training, and expands the range of functions that can be modelled.

Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks

- Computer ScienceICML
- 2019

This work presents an attention-based neural network module, the Set Transformer, specifically designed to model interactions among elements in the input set, and reduces the computation time of self-attention from quadratic to linear in the number of Elements in the set.

Convolutional Conditional Neural Processes

- Mathematics, Computer ScienceICLR
- 2020

This work introduces the Convolutional Conditional Conditional Neural Process (ConvCNP), a new member of the Neural Process family that models translation equivariance in the data, and demonstrates that any translation-equivariant embedding can be represented using a convolutional deep set.