Corpus ID: 12122362

Learning to Remember Rare Events

@article{Kaiser2017LearningTR,
  title={Learning to Remember Rare Events},
  author={Lukasz Kaiser and Ofir Nachum and Aurko Roy and Samy Bengio},
  journal={ArXiv},
  year={2017},
  volume={abs/1703.03129}
}
Despite recent advances, memory-augmented deep neural networks are still limited when it comes to life-long and one-shot learning, especially in remembering rare events. [...] Key MethodOur memory module can be easily added to any part of a supervised neural network. To show its versatility we add it to a number of networks, from simple convolutional ones tested on image classification to deep sequence-to-sequence and recurrent-convolutional models. In all cases, the enhanced network gains the ability to…Expand

Paper Mentions

Memory Matching Networks for One-Shot Image Recognition
TLDR
Memory Matching Networks (MM-Net) is presented - a novel deep architecture that explores the training procedure, following the philosophy that training and test conditions must match, and improves one-shot accuracy on Omniglot and miniImageNet. Expand
Concept Learning through Deep Reinforcement Learning with Memory-Augmented Neural Networks
TLDR
This paper presents a memory-augmented neural network which is motivated by the process of human concept learning, and proposes a sequential process, during which the network should decide how to remember each sample at every step. Expand
Adaptive Posterior Learning: few-shot learning with a surprise-based memory module
TLDR
This paper introduces APL, an algorithm that approximates probability distributions by remembering the most surprising observations it has encountered and is able to generalize with fewer than one example per class via deductive reasoning. Expand
Labeled Memory Networks for Online Model Adaptation
TLDR
It is shown that LMNs are better than other MANNs designed for meta-learning and more accurate and faster than state-of-the-art methods of retuning model parameters for adapting to domain-specific labeled data. Expand
Inflated Episodic Memory With Region Self-Attention for Long-Tailed Visual Recognition
  • Linchao Zhu, Y. Yang
  • Computer Science
  • 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2020
TLDR
An Inflated Episodic Memory (IEM) is introduced for long-tailed visual recognition to deal with the class imbalance problem and a novel region self-attention mechanism for multi-scale spatial feature map encoding is introduced. Expand
Few-Shot Ensemble Learning for Video Classification with SlowFast Memory Networks
TLDR
This paper introduces a family of few-shot learners based on SlowFast networks which are used to extract informative features at multiple rates, and incorporates a memory unit into each network to enable encoding and retrieving crucial information instantly. Expand
Memory-based Parameter Adaptation
TLDR
Memory-based Parameter Adaptation alleviates several shortcomings of neural networks, such as catastrophic forgetting, fast, stable acquisition of new knowledge, learning with an imbalanced class labels, and fast learning during evaluation. Expand
Label Organized Memory Augmented Neural Network
TLDR
It is shown that LMNs are better than other MANNs designed for meta-learning and more accurate and faster than state-of-the-art methods of retuning model parameters for adapting to domain-specific labeled data. Expand
Robust Compare Network for Few-Shot Learning
TLDR
This work argues that it is desirable to learn a robust encoder that can draw inferences about other cases from one example, and improves the accuracy of few-shot learning by mining the internal mechanism of deep networks, which can leverage label information more effectively. Expand
Self-Attention Relation Network for Few-Shot Learning
TLDR
A self-attention relation network for few shot learning that can discover non-local information and allow long-range dependency, and outperforms the state-of-the-art algorithms in terms of both few shot and zero shot tasks. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 36 REFERENCES
Sequence to Sequence Learning with Neural Networks
TLDR
This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier. Expand
Siamese Neural Networks for One-Shot Image Recognition
TLDR
A method for learning siamese neural networks which employ a unique structure to naturally rank similarity between inputs and is able to achieve strong results which exceed those of other deep learning models with near state-of-the-art performance on one-shot classification tasks. Expand
Long Short-Term Memory
TLDR
A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Expand
Matching Networks for One Shot Learning
TLDR
This work employs ideas from metric learning based on deep neural features and from recent advances that augment neural networks with external memories to learn a network that maps a small labelled support set and an unlabelled example to its label, obviating the need for fine-tuning to adapt to new class types. Expand
Can Active Memory Replace Attention?
TLDR
An extended model of active memory is proposed that matches existing attention models on neural machine translation and generalizes better to longer sentences and discusses when active memory brings most benefits and where attention can be a better choice. Expand
Hierarchical Memory Networks
TLDR
A form of hierarchical memory network is explored, which can be considered as a hybrid between hard and soft attention memory networks, and is organized in a hierarchical structure such that reading from it is done with less computation than soft attention over a flat memory, while also being easier to train than hard attention overA flat memory. Expand
Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes
TLDR
This work presents an end-to-end differentiable memory access scheme, which they call Sparse Access Memory (SAM), that retains the representational power of the original approaches whilst training efficiently with very large memories, and achieves asymptotic lower bounds in space and time complexity. Expand
ImageNet classification with deep convolutional neural networks
TLDR
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective. Expand
Order Matters: Sequence to sequence for sets
TLDR
An extension of the seq2seq framework that goes beyond sequences and handles input sets in a principled way is discussed and a loss is proposed which, by searching over possible orders during training, deals with the lack of structure of output sets. Expand
Weakly Supervised Memory Networks
TLDR
This paper introduces a variant of Memory Networks that needs significantly less supervision to perform question and answering tasks and applies it to the synthetic bAbI tasks, showing that the approach is competitive with the supervised approach, particularly when trained on a sufficiently large amount of data. Expand
...
1
2
3
4
...