Corpus ID: 236087922

Non-binary deep transfer learning for imageclassification

@article{Plested2021NonbinaryDT,
  title={Non-binary deep transfer learning for imageclassification},
  author={Jo Plested and Xuyang Shen and Tom Gedeon},
  journal={ArXiv},
  year={2021},
  volume={abs/2107.08585}
}
The current standard for a variety of computer vision tasks using smaller numbers of labelled training examples is to fine-tune from weights pre-trained on a large image classification dataset such as ImageNet. The application of transfer learning and transfer learning methods tends to be rigidly binary. A model is either pre-trained or not pre-trained. Pre-training a model either increases performance or decreases it, the latter being defined as negative transfer. Application of L2-SP… Expand

References

SHOWING 1-10 OF 46 REFERENCES
The Earth Mover's Distance as a Metric for Image Retrieval
TLDR
This paper investigates the properties of a metric between two distributions, the Earth Mover's Distance (EMD), for content-based image retrieval, and compares the retrieval performance of the EMD with that of other distances. Expand
Rethinking the Hyperparameters for Fine-tuning
TLDR
It is found that picking the right value for momentum is critical for fine-tuning performance and is connected with previous theoretical findings. Expand
DELTA: DEep Learning Transfer using Feature Map with Attention for Convolutional Networks
TLDR
A novel regularized transfer learning framework DELTA, namely DEep Learning Transfer using Feature Map with Attention that intends to align the outer layer outputs of two networks, through constraining a subset of feature maps that are precisely selected by attention that has been learned in an supervised learning manner. Expand
Do Better ImageNet Models Transfer Better?
TLDR
It is found that, when networks are used as fixed feature extractors or fine-tuned, there is a strong correlation between ImageNet accuracy and transfer accuracy, and ImageNet features are less general than previously suggested. Expand
Towards Making Deep Transfer Learning Never Hurt
TLDR
The empirical results show that the proposed descent direction estimation strategy DTNH can always improve the performance of deep transfer learning tasks based on all above regularizers, even when transferring pre-trained weights from inappropriate networks. Expand
How transferable are features in deep neural networks?
TLDR
This paper quantifies the generality versus specificity of neurons in each layer of a deep convolutional neural network and reports a few surprising results, including that initializing a network with transferred features from almost any number of layers can produce a boost to generalization that lingers even after fine-tuning to the target dataset. Expand
Deep Face Recognition: A Survey
TLDR
A comprehensive review of the recent developments on deep FR, covering broad topics on algorithm designs, databases, protocols, and application scenes, and the technical challenges and several promising directions are highlighted. Expand
TResNet: High Performance GPU-Dedicated Architecture
TLDR
A series of architecture modifications are introduced that aim to boost neural networks’ accuracy, while retaining their GPU training and inference efficiency, and a new family of GPU-dedicated models, called TResNet, which achieves better accuracy and efficiency than previous ConvNets1. Expand
A Simple Framework for Contrastive Learning of Visual Representations
TLDR
It is shown that composition of data augmentations plays a critical role in defining effective predictive tasks, and introducing a learnable nonlinear transformation between the representation and the contrastive loss substantially improves the quality of the learned representations, and contrastive learning benefits from larger batch sizes and more training steps compared to supervised learning. Expand
Big Transfer (BiT): General Visual Representation Learning
TLDR
By combining a few carefully selected components, and transferring using a simple heuristic, Big Transfer achieves strong performance on over 20 datasets and performs well across a surprisingly wide range of data regimes -- from 1 example per class to 1M total examples. Expand
...
1
2
3
4
5
...