# Neural Turing Machines

@article{Graves2014NeuralTM, title={Neural Turing Machines}, author={Alex Graves and Greg Wayne and Ivo Danihelka}, journal={ArXiv}, year={2014}, volume={abs/1410.5401} }

We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes. [... ] Key Result Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples. Expand

## Figures and Tables from this paper

## 1,712 Citations

### Neural networks with external dynamic memory to solve algorithmic problems

- Computer Science
- 2019

Preliminary results show that Differentiable Neural Computers can infer simple algorithms like Neural Turing Machines but in a more efficient and efficacy way, and also has the capability of generalizing results without retraining the model.

### Evolving Neural Turing Machines!

- Computer Science, Psychology
- 2015

Preliminary results suggest that this setup can greatly simplify the neural model, generalizes better, and does not require accessing the entire memory content at each time-step.

### Learning Operations on a Stack with Neural Turing Machines

- Computer ScienceArXiv
- 2016

It is shown that not only does the NTM emulate a stack with its heads and learn an algorithm to recognize such words, but it is also capable of strongly generalizing to much longer sequences.

### A review on Neural Turing Machine

- Computer ScienceArXiv
- 2019

The attempt is made here to run a systematic review on Neural Turing Machine in terms of concepts, structure, variety of versions, implemented tasks, comparisons, etc.

### Neural Turing Machines: Convergence of Copy Tasks

- Computer Science
- 2016

The architecture of neural Turing machines is differentiable end to end and is trainable with gradient descent methods. Due to their large unfolded depth Neural Turing Machines are hard to train and…

### Implementing Neural Turing Machines

- Computer ScienceICANN
- 2018

This paper finds that the choice of memory contents initialization scheme is crucial in successfully implementing a NTM, and finds networks with memory contents initialized to small constant values converge on average 2 times faster than the next best memory contents initialize scheme.

### Neural Structured Turing Machine

- Computer Science, Psychology
- 2017

This work takes structured information into consideration and proposes a Neural Structured Turing Machine (NSTM), leading to a novel perspective for improving differential external memory based mechanism, e.g., NTM.

### Continual Learning through Evolvable Neural Turing Machines

- Computer Science
- 2016

This paper shows that the recently proposed Evolving Neural Turing Machine (ENTM) approach is able to perform one-shot learning in a reinforcement learning task without catastrophic forgetting of previously stored associations.

### Structured Memory for Neural Turing Machines

- Computer ScienceArXiv
- 2015

This paper proves in experiments that two of the proposed structured-memory NTMs could lead to better convergence, in term of speed and prediction accuracy on copy task and associative recall task as in (Graves et al. 2014).

### Neural Stored-program Memory

- Computer ScienceICLR
- 2020

A new memory to store weights for the controller, analogous to the stored-program memory in modern computer architectures is introduced, creating differentiable machines that can switch programs through time, adapt to variable contexts and thus resemble the Universal Turing Machine.

## References

SHOWING 1-10 OF 44 REFERENCES

### On the Computational Power of Neural Nets

- Computer ScienceJ. Comput. Syst. Sci.
- 1995

It is proved that one may simulate all Turing Machines by rational nets in linear time, and there is a net made up of about 1,000 processors which computes a universal partial-recursive function.

### Neural networks and physical systems with emergent collective computational abilities.

- Computer ScienceProceedings of the National Academy of Sciences of the United States of America
- 1982

A model of a system having a large number of simple equivalent components, based on aspects of neurobiology but readily adapted to integrated circuits, produces a content-addressable memory which correctly yields an entire memory from any subpart of sufficient size.

### Simple Substrates for Complex Cognition

- Computer ScienceFront. Neurosci.
- 2008

A recent suggestion for a uniform architecture for habitual and rule-based execution is reviewed, some of the habitual mechanisms that underpin the use of rules are discussed, and a statistical relationship between rules and habits is considered.

### The Algebraic Mind: Integrating Connectionism and Cognitive Science

- Psychology, Biology
- 2001

Gary Marcus outlines a variety of ways in which neural systems could be organized so as to manipulate symbols, and he shows why such systems are more likely to provide an adequate substrate for language and cognition than neural systems that are inconsistent with the manipulation of symbols.

### BoltzCONS: Dynamic Symbol Structures in a Connectionist Network

- Computer ScienceArtif. Intell.
- 1990

### Generating Text with Recurrent Neural Networks

- Computer ScienceICML
- 2011

The power of RNNs trained with the new Hessian-Free optimizer by applying them to character-level language modeling tasks is demonstrated, and a new RNN variant that uses multiplicative connections which allow the current input character to determine the transition matrix from one hidden state vector to the next is introduced.