Overcoming catastrophic forgetting in neural networks
- J. Kirkpatrick, Razvan Pascanu, R. Hadsell
- Computer ScienceProceedings of the National Academy of Sciences
- 2 December 2016
It is shown that it is possible to overcome the limitation of connectionist models and train networks that can maintain expertise on tasks that they have not experienced for a long time and selectively slowing down learning on the weights important for previous tasks.
On the difficulty of training recurrent neural networks
- Razvan Pascanu, Tomas Mikolov, Yoshua Bengio
- Computer ScienceInternational Conference on Machine Learning
- 21 November 2012
This paper proposes a gradient norm clipping strategy to deal with exploding gradients and a soft constraint for the vanishing gradients problem and validates empirically the hypothesis and proposed solutions.
Relational inductive biases, deep learning, and graph networks
- P. Battaglia, Jessica B. Hamrick, Razvan Pascanu
- Computer ScienceArXiv
- 4 June 2018
It is argued that combinatorial generalization must be a top priority for AI to achieve human-like abilities, and that structured representations and computations are key to realizing this objective.
A simple neural network module for relational reasoning
- Adam Santoro, David Raposo, T. Lillicrap
- Computer ScienceNIPS
- 5 June 2017
This work shows how a deep learning architecture equipped with an RN module can implicitly discover and learn to reason about entities and their relations.
Theano: A Python framework for fast computation of mathematical expressions
- Rami Al-Rfou, Guillaume Alain, Ying Zhang
- Computer ScienceArXiv
- 9 May 2016
The performance of Theano is compared against Torch7 and TensorFlow on several machine learning models and recently-introduced functionalities and improvements are discussed.
Progressive Neural Networks
- Andrei A. Rusu, Neil C. Rabinowitz, R. Hadsell
- Computer ScienceArXiv
- 15 June 2016
This work evaluates the progressive networks architecture extensively on a wide variety of reinforcement learning tasks, and shows that it outperforms common baselines based on pretraining and netuning and demonstrates that transfer occurs at both low-level sensory and high-level control layers of the learned policy.
Meta-Learning with Latent Embedding Optimization
- Andrei A. Rusu, Dushyant Rao, R. Hadsell
- Computer ScienceInternational Conference on Learning…
- 16 July 2018
This work shows that latent embedding optimization can achieve state-of-the-art performance on the competitive miniImageNet and tieredImageNet few-shot classification tasks, and indicates LEO is able to capture uncertainty in the data, and can perform adaptation more effectively by optimizing in latent space.
Interaction Networks for Learning about Objects, Relations and Physics
- P. Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, K. Kavukcuoglu
- Computer Science, PhysicsNIPS
- 1 December 2016
The interaction network is introduced, a model which can reason about how objects in complex systems interact, supporting dynamical predictions, as well as inferences about the abstract properties of the system, and is implemented using deep neural networks.
Theano: new features and speed improvements
- Frédéric Bastien, Pascal Lamblin, Yoshua Bengio
- Computer ScienceArXiv
- 23 November 2012
New features and efficiency improvements to Theano are presented, and benchmarks demonstrating Theano's performance relative to Torch7, a recently introduced machine learning library, and to RNNLM, a C++ library targeted at recurrent neural networks.
Progress & Compress: A scalable framework for continual learning
- Jonathan Schwarz, Wojciech M. Czarnecki, R. Hadsell
- Computer ScienceInternational Conference on Machine Learning
- 16 May 2018
The progress & compress approach is demonstrated on sequential classification of handwritten alphabets as well as two reinforcement learning domains: Atari games and 3D maze navigation.
...
...