Corpus ID: 221761345

'Less Than One'-Shot Learning: Learning N Classes From M

@inproceedings{Sucholutsky2021LessTO,
  title={'Less Than One'-Shot Learning: Learning N Classes From M},
  author={Ilia Sucholutsky and Matthias Schonlau},
  booktitle={AAAI},
  year={2021}
}
Deep neural networks require large training sets but suffer from high computational cost and long training times. Training on much smaller training sets while maintaining nearly the same accuracy would be very beneficial. In the few-shot learning setting, a model must learn a new class given only a small number of samples from that class. One-shot learning is an extreme form of few-shot learning where the model must learn a new class from a single example. We propose the `less than one'-shot… 

Figures from this paper

One Line To Rule Them All: Generating LO-Shot Soft-Label Prototypes
TLDR
This work proposes a novel, modular method for generating soft-label prototypical lines that still maintains representational accuracy even when there are fewer prototypes than the number of classes in the data, and proposes the Hierarchical Soft-Label Prototype k-Nearest Neighbor classification algorithm based on these prototypeical lines.
Applying Deutsch’s concept of good explanations to artificial intelligence and neuroscience – An initial exploration
  • D. Elton
  • Computer Science
    Cognitive Systems Research
  • 2021
TLDR
This work investigates Deutsch's hard-to-vary principle and how it relates to more formalized principles in deep learning such as the bias-variance trade-off and Occam's razor and makes contact with the framework of Popperian epistemology which rejects induction and asserts that knowledge generation is an evolutionary process which proceeds through conjecture and refutation.
Bridge Networks: Relating Inputs through Vector-Symbolic Manipulations
TLDR
This work proposes and implements a novel information processing architecture, the ‘Bridge network,’ which provides unique advantages which can address the problem of global losses and catastrophic forgetting and provides a further basis for increasing energy efficiency of execution and the ability to reason symbolically.
Implicit Regularization in Overparameterized Bilevel Optimization
Bilevel problems involve inner and outer parameters, each optimized for its own objective. Most prior work makes the simplifying assumption that the inner and outer objectives have unique solutions,
Interviewer-Candidate Role Play: Towards Developing Real-World NLP Systems
TLDR
This work presents a multi-stage task that simulates a typical human-human questioner-responder interaction such as an interview and finds that theMulti-stage formulation of the task leads to OOD generalization performance improvement up to 2.29% in Stage 1, 1.91% in stages 2, and 72.02% in stage 4 over the standard unguided prediction.
Synthesising Facial Macro- and Micro-Expressions Using Reference Guided Style Transfer
TLDR
A new approach to generate synthetic long videos and recommend assessment methods to inspect dataset quality is introduced and a dataset is published to enable future research and to increase the data pool of micro-expressions research, especially in the spotting task.
Shockingly Simple:"KEYS" for Better AI for SE
As 2020 drew to a close, I was thinking about what lessons we have learned about software engineering (SE) for artificial intelligence (AI)-things that we can believe now but, in the last century,
SecDD: Efficient and Secure Method for Remotely Training Neural Networks
TLDR
This work leverages what are typically considered the worst qualities of deep learning algorithms to create a method for the secure and efficient training of remotely deployed neural networks over unsecured channels.
The Ouroboros Model, Proposal for Self-Organizing General Cognition Substantiated
TLDR
The Ouroboros Model has been proposed as a biologically-inspired comprehensive cognitive architecture for general intelligence, comprising natural and artificial manifestations, and how the postulated structures have met with supportive evidence over recent years is described.
Clinician-driven artificial intelligence in ophthalmology: resources enabling democratization
TLDR
Clinicians are the use-case experts who are best suited to drive AI projects tackling patient-relevant outcome measures, and open datasets, efficient labeling techniques, code-free AutoML and cloud platforms break the barriers for clinician-driven AI.
...
1
2
...

References

SHOWING 1-10 OF 32 REFERENCES
Matching Networks for One Shot Learning
TLDR
This work employs ideas from metric learning based on deep neural features and from recent advances that augment neural networks with external memories to learn a network that maps a small labelled support set and an unlabelled example to its label, obviating the need for fine-tuning to adapt to new class types.
Generalizing from a Few Examples: A Survey on Few-Shot Learning
TLDR
A thorough survey to fully understand Few-Shot Learning (FSL), and categorizes FSL methods from three perspectives: data, which uses prior knowledge to augment the supervised experience; model, which used to reduce the size of the hypothesis space; and algorithm, which using prior knowledgeto alter the search for the best hypothesis in the given hypothesis space.
Prototypical Networks for Few-shot Learning
TLDR
This work proposes Prototypical Networks for few-shot classification, and provides an analysis showing that some simple design decisions can yield substantial improvements over recent approaches involving complicated architectural choices and meta-learning.
Active Learning for Convolutional Neural Networks: A Core-Set Approach
TLDR
This work defines the problem of active learning as core-set selection as choosing set of points such that a model learned over the selected subset is competitive for the remaining data points, and presents a theoretical result characterizing the performance of any selected subset using the geometry of the datapoints.
One-shot learning of object categories
TLDR
It is found that on a database of more than 100 categories, the Bayesian approach produces informative models when the number of training examples is too small for other methods to operate successfully.
Pruning training sets for learning of object categories
TLDR
This work proposes a fully automatic mechanism for noise cleaning, called 'data pruning', and demonstrates its success on learning of human faces and shows that data pruning can improve on generalization performance for algorithms with various robustness to noise.
Human-level concept learning through probabilistic program induction
TLDR
A computational model is described that learns in a similar fashion and does so better than current deep learning algorithms and can generate new letters of the alphabet that look “right” as judged by Turing-like tests of the model's output in comparison to what real humans produce.
Soft-Label Dataset Distillation and Text Dataset Distillation
TLDR
This work proposes to simultaneously distill both images and their labels, thus assigning each synthetic sample a `soft' label (a distribution of labels) and demonstrates that text distillation outperforms other methods across multiple datasets.
Core Vector Machines: Fast SVM Training on Very Large Data Sets
TLDR
This paper shows that many kernel methods can be equivalently formulated as minimum enclosing ball (MEB) problems in computational geometry and obtains provably approximately optimal solutions with the idea of core sets, and proposes the proposed Core Vector Machine (CVM) algorithm, which can be used with nonlinear kernels and has a time complexity that is linear in m.
Dataset Distillation
TLDR
This paper keeps the model fixed and instead attempts to distill the knowledge from a large training dataset into a small one, to synthesize a small number of data points that do not need to come from the correct data distribution but will approximate the model trained on the original data.
...
1
2
3
4
...