# Neural Turing Machines

@article{Graves2014NeuralTM, title={Neural Turing Machines}, author={Alex Graves and Greg Wayne and Ivo Danihelka}, journal={ArXiv}, year={2014}, volume={abs/1410.5401} }

We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes. [... ] Key Result Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples. Expand

## Figures and Tables from this paper

## 1,660 Citations

Neural networks with external dynamic memory to solve algorithmic problems

- Computer Science
- 2019

Preliminary results show that Differentiable Neural Computers can infer simple algorithms like Neural Turing Machines but in a more efficient and efficacy way, and also has the capability of generalizing results without retraining the model.

Evolving Neural Turing Machines!

- Computer Science, Psychology
- 2015

Preliminary results suggest that this setup can greatly simplify the neural model, generalizes better, and does not require accessing the entire memory content at each time-step.

Learning Operations on a Stack with Neural Turing Machines

- Computer ScienceArXiv
- 2016

It is shown that not only does the NTM emulate a stack with its heads and learn an algorithm to recognize such words, but it is also capable of strongly generalizing to much longer sequences.

A review on Neural Turing Machine

- Computer ScienceArXiv
- 2019

The attempt is made here to run a systematic review on Neural Turing Machine in terms of concepts, structure, variety of versions, implemented tasks, comparisons, etc.

Neural Turing Machines: Convergence of Copy Tasks

- Computer Science
- 2016

The architecture of neural Turing machines is differentiable end to end and is trainable with gradient descent methods. Due to their large unfolded depth Neural Turing Machines are hard to train and…

Turing Computation with Recurrent Artificial Neural Networks

- Computer ScienceCoCo@NIPS
- 2015

This work provides a novel and parsimonious constructive mapping between Turing Machines and Recurrent Artificial Neural Networks, based on recent developments of Nonlinear Dynamical Automata, and provides a framework to directly program the R-ANNs from Turing Machine descriptions, in absence of network training.

Implementing Neural Turing Machines

- Computer ScienceICANN
- 2018

This paper finds that the choice of memory contents initialization scheme is crucial in successfully implementing a NTM, and finds networks with memory contents initialized to small constant values converge on average 2 times faster than the next best memory contents initialize scheme.

Neural Structured Turing Machine

- Computer Science, Psychology
- 2017

This work takes structured information into consideration and proposes a Neural Structured Turing Machine (NSTM), leading to a novel perspective for improving differential external memory based mechanism, e.g., NTM.

Continual Learning through Evolvable Neural Turing Machines

- Computer Science
- 2016

This paper shows that the recently proposed Evolving Neural Turing Machine (ENTM) approach is able to perform one-shot learning in a reinforcement learning task without catastrophic forgetting of previously stored associations.

Neural Stored-program Memory

- Computer ScienceICLR
- 2020

A new memory to store weights for the controller, analogous to the stored-program memory in modern computer architectures is introduced, creating differentiable machines that can switch programs through time, adapt to variable contexts and thus resemble the Universal Turing Machine.

## References

SHOWING 1-10 OF 44 REFERENCES

On the Computational Power of Neural Nets

- Computer ScienceJ. Comput. Syst. Sci.
- 1995

It is proved that one may simulate all Turing Machines by rational nets in linear time, and there is a net made up of about 1,000 processors which computes a universal partial-recursive function.

Neural networks and physical systems with emergent collective computational abilities.

- Computer ScienceProceedings of the National Academy of Sciences of the United States of America
- 1982

A model of a system having a large number of simple equivalent components, based on aspects of neurobiology but readily adapted to integrated circuits, produces a content-addressable memory which correctly yields an entire memory from any subpart of sufficient size.

Learning Context-free Grammars: Capabilities and Limitations of a Recurrent Neural Network with an External Stack Memory

- Computer Science
- 1992

An analog stack is developed which reverts to a discrete stack by quantization of all activations, after the network has learned the transition rules and stack actions, and an enhancement of the network's learning capabilities by providing hints.

Simple Substrates for Complex Cognition

- Computer ScienceFront. Neurosci.
- 2008

A recent suggestion for a uniform architecture for habitual and rule-based execution is reviewed, some of the habitual mechanisms that underpin the use of rules are discussed, and a statistical relationship between rules and habits is considered.

The Algebraic Mind: Integrating Connectionism and Cognitive Science

- Psychology, Biology
- 2001

Gary Marcus outlines a variety of ways in which neural systems could be organized so as to manipulate symbols, and he shows why such systems are more likely to provide an adequate substrate for language and cognition than neural systems that are inconsistent with the manipulation of symbols.

Generating Sequences With Recurrent Neural Networks

- Computer ScienceArXiv
- 2013

This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time. The approach…

BoltzCONS: Dynamic Symbol Structures in a Connectionist Network

- Computer ScienceArtif. Intell.
- 1990

A general framework for adaptive processing of data structures

- Computer ScienceIEEE Trans. Neural Networks
- 1998

The framework described in this paper is an attempt to unify adaptive models like artificial neural nets and belief nets for the problem of processing structured information, where relations between data variables are expressed by directed acyclic graphs, where both numerical and categorical values coexist.