• Corpus ID: 220935574

A Foliated View of Transfer Learning

@article{Petangoda2020AFV,
  title={A Foliated View of Transfer Learning},
  author={Janith C. Petangoda and Nick A. M. Monk and Marc Peter Deisenroth},
  journal={ArXiv},
  year={2020},
  volume={abs/2008.00546}
}
Transfer learning considers a learning process where a new task is solved by transferring relevant knowledge from known solutions to related tasks. While this has been studied experimentally, there lacks a foundational description of the transfer learning problem that exposes what related tasks are, and how they can be exploited. In this work, we present a definition for relatedness between tasks and identify foliations as a mathematical framework to represent such relationships. 

Figures from this paper

Using ROC and Unlabeled Data for Increasing Low-Shot Transfer Learning Classification Accuracy

TLDR
A low-shot classifier is proposed which can serve as the top layer to any existing CNN that the feature extractor was already trained, and is able to increase identification accuracy by up to 30% for the images that do not belong to any specific classes, while retaining the ability to identify images that belong to the specific classes of interest.

GENNI: Visualising the Geometry of Equivalences for Neural Network Identifiability

TLDR
The proposed method, GENNI, allows us to efficiently identify parameters that are functionally equivalent and then visualise the subspace of the resulting equivalence class, to better explore questions surrounding identifiability.

References

SHOWING 1-10 OF 39 REFERENCES

Exploiting Task Relatedness for Mulitple Task Learning

TLDR
This work offers an alternative approach to multiple task learning, defining relatedness of tasks on the basis of similarity between the example generating distributions that underline these task.

Is Learning The n-th Thing Any Easier Than Learning The First?

  • S. Thrun
  • Computer Science, Education
    NIPS
  • 1995
TLDR
It is shown that across the board, lifelong learning approaches generalize consistently more accurately from less training data, by their ability to transfer knowledge across learning tasks.

A notion of task relatedness yielding provable multiple-task learning guarantees

TLDR
This work provides a formal framework for this notion of task relatedness, which captures a sub-domain of the wide scope of issues in which one may apply a multiple task learning approach and allows the formal derivation of generalization bounds that are strictly stronger than the previously known bounds.

Meta-Learning in Neural Networks: A Survey

TLDR
A new taxonomy is proposed that provides a more comprehensive breakdown of the space of meta-learning methods today and surveys promising applications and successes ofMeta-learning such as few-shot learning and reinforcement learning.

Representation Learning: A Review and New Perspectives

TLDR
Recent work in the area of unsupervised feature learning and deep learning is reviewed, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks.

Multitask Learning

TLDR
Prior work on MTL is reviewed, new evidence that MTL in backprop nets discovers task relatedness without the need of supervisory signals is presented, and new results for MTL with k-nearest neighbor and kernel regression are presented.

Matching Networks for One Shot Learning

TLDR
This work employs ideas from metric learning based on deep neural features and from recent advances that augment neural networks with external memories to learn a network that maps a small labelled support set and an unlabelled example to its label, obviating the need for fine-tuning to adapt to new class types.

A Survey on Transfer Learning

TLDR
The relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift are discussed.

Meta Reinforcement Learning with Latent Variable Gaussian Processes

TLDR
This paper frame meta learning as a hierarchical latent variable model and infer the relationship between tasks automatically from data and shows that this results in up to a 60% reduction in the average interaction time needed to solve tasks compared to strong baselines.

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning