Corpus ID: 210700853

Self-supervised visual feature learning with curriculum

@article{Keshav2020SelfsupervisedVF,
  title={Self-supervised visual feature learning with curriculum},
  author={Vishal Keshav and Fabien Delattre},
  journal={ArXiv},
  year={2020},
  volume={abs/2001.05634}
}
Self-supervised learning techniques have shown their abilities to learn meaningful feature representation. This is made possible by training a model on pretext tasks that only requires to find correlations between inputs or parts of inputs. However, such pretext tasks need to be carefully hand selected to avoid low level signals that could make those pretext tasks trivial. Moreover, removing those shortcuts often leads to the loss of some semantically valuable information. We show that it… Expand
2 Citations
A Multi-view Perspective of Self-supervised Learning
TLDR
A multi-view perspective is borrowed to decouple a class of popular pretext tasks into a combination of view data augmentation (VDA) and view label classification (VLC), where it is found that it is VDA rather than generally considered VLC that dominates the performance of such SSL. Expand
Self-Supervised Representation Learning for Detection of ACL Tear Injury in Knee MRI
TLDR
Experiments on the pretext task show that this proposed approach enables the model to learn spatial context invariant features which helps in reliable and explainable performance in downstream tasks like classification of ACL tear injury from knee MRI. Expand

References

SHOWING 1-10 OF 21 REFERENCES
Unsupervised Representation Learning by Predicting Image Rotations
TLDR
This work proposes to learn image features by training ConvNets to recognize the 2d rotation that is applied to the image that it gets as input, and demonstrates both qualitatively and quantitatively that this apparently simple task actually provides a very powerful supervisory signal for semantic feature learning. Expand
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
TLDR
A novel unsupervised learning approach to build features suitable for object detection and classification and to facilitate the transfer of features to other tasks, the context-free network (CFN), a siamese-ennead convolutional neural network is introduced. Expand
Unsupervised Visual Representation Learning by Context Prediction
TLDR
It is demonstrated that the feature representation learned using this within-image context indeed captures visual similarity across images and allows us to perform unsupervised visual discovery of objects like cats, people, and even birds from the Pascal VOC 2011 detection dataset. Expand
Curriculum learning
TLDR
It is hypothesized that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions). Expand
CASSL: Curriculum Accelerated Self-Supervised Learning
TLDR
This work presents a novel approach - Curriculum Accelerated Self-Supervised Learning (CASSL) - to train policies that map visual information to high-level, higher-dimensional action spaces and applies it to learning how to grasp using an adaptive, underactuated multi-fingered gripper, a challenging system to control. Expand
Automated Curriculum Learning for Neural Networks
TLDR
Experimental results for LSTM networks on three curricula demonstrate that the approach can significantly accelerate learning, in some cases halving the time required to attain a satisfactory performance level. Expand
Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks
TLDR
While features learned with this approach cannot compete with class specific features from supervised training on a classification task, it is shown that they are advantageous on geometric matching problems, where they also outperform the SIFT descriptor. Expand
Learning Multiple Layers of Features from Tiny Images
TLDR
It is shown how to train a multi-layer generative model that learns to extract meaningful features which resemble those found in the human visual cortex, using a novel parallelization algorithm to distribute the work among multiple machines connected on a network. Expand
Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
TLDR
An extensive review of deep learning-based self-supervised general visual feature learning methods from images or videos as a subset of unsupervised learning methods to learn general image and video features from large-scale unlabeled data without using any human-annotated labels is provided. Expand
An Analysis of Single-Layer Networks in Unsupervised Feature Learning
TLDR
The results show that large numbers of hidden nodes and dense feature extraction are critical to achieving high performance—so critical, in fact, that when these parameters are pushed to their limits, they achieve state-of-the-art performance on both CIFAR-10 and NORB using only a single layer of features. Expand
...
1
2
3
...