Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning

  title={Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning},
  author={Peter Ford Dominey},
  journal={Biological Cybernetics},
A novel neural network model is presented that learns by trial-and-error to reproduce complex sensory-motor sequences. One subnetwork, corresponding to the prefrontal cortex (PFC), is responsible for generating unique patterns of activity that represent the continuous state of sequence execution. A second subnetwork, corresponding to the striatum, associates these state-encoding patterns with the correct response at each point in the sequence execution. From a neuroscience perspective, the… 

A Neural Model for Context-dependent Sequence Learning

Experimental results demonstrate that the use of context has three important benefits: it prevents catastrophic interference during learning of multiple overlapping sequences, it enables the completion of sequences from missing or noisy patterns, and it provides a mechanism to selectively explore the space of learned sequences during free recall.

From Sensorimotor Sequence to Grammatical Construction: Evidence from Simulation and Neurophysiology

A functional trajectory from sensorimotor sequence learning to the learning of grammatical constructions in language is described, which will provide insight into the relation between the coding structure in human languages, and constraints derived from the underlying neurophysiological computational mechanisms.

A Neural Network Architecture for the Categorization of Temporal Information

A neural network architecture that is capable of continuous learning multiple, possibly overlapping, arbitrary input sequences relatively quickly, autonomously and online and gives the network the ability to not only adjust incorrectly recalled parts of a sequence but also to reinforce and stabilize the recall of previously acquired sequences.

Reservoir Computing Properties of Neural Dynamics in Prefrontal Cortex

This second version of the model demonstrates how a hybrid dynamical regime combining spatio-temporal processing of reservoirs, and input driven attracting dynamics generated by the feedback neuron, can be used to solve a complex cognitive task.

Feature and structural learning of memory sequences with recurrent and gated spiking neural networks using free-energy: application to speech perception and production II

A genuine coding strategy using the gain-modulation mechanism to represent abstract sequences based on the rank and location of items within them only and it is shown that this mechanism can construct a repertoire of neurons sensitive to the temporal structure in sequences from which they can represent any novel sequences.

A three-layered cortical network encodes identity and abstract categorical structure of behavioral sequences as in the primate lateral prefrontal cortex

It is argued that a richly interconnected cortical circuit is capable of internally generating a neural representation of category membership, thus significantly extending the scope of recurrent network computation.

Adaptive networks for robotics and the emergence of reward anticipatory circuits

This thesis proposes a formal model of functional integration, encompassing locomotion and sequence learning, and presents some ideas on how the co-evolution of GasNet and reward adaptive circuits might lead to significant improvements in the synthesis of agents capable of exhibiting complex adaptive behaviour.

Selection of cortical dynamics for motor behaviour by the basal ganglia

A computational model in which a thalamo-cortical dynamical neural reservoir is modulated by disinhibitory selection of the basal ganglia guided by top-down information, so that it responds with different dynamics to the same bottom-up input is proposed.

Efficient Reward-Based Learning through Body Representation in a Spiking Neural Network

A spiking neural network model for the reward-based learning of canonical babbling found that some neurons in the SNN were more highly activated during vocalization of a consonant than during other sounds, which facilitated brain-body interaction and accelerated the acquisition of babbling behavior.



Complex temporal sequence learning based on short-term memory

It is shown that a quantity called the input potential increases monotonically with sequence presentation, and that the neuron can only be fired when its input signals are arranged in a specific sequence.

A Model of Corticostriatal Plasticity for Learning Oculomotor Associations and Sequences

Abstract We present models that learn context-dependent oculomotor behavior in (1) conditional visual discrimination and (2) sequence reproduction tasks, based on the following three principles: (1)

Neural networks that learn temporal sequences by selection.

A network architecture composed of three layers of neuronal clusters is shown to exhibit active recognition and learning of time sequences by selection: the network spontaneously produces prerepresentations that are selected according to their resonance with the input percepts.

On the Modularity of Sequence Representation

Abstract A modular theory of motor control posits that the representation of an action sequence is independent of the effector (motor) system that implements the sequence. Three experiments tested

Models of spatial accuracy and conditional behavior in oculomotor control

A new model for the spatiotemporal transformation required for the production of reflexive, high-velocity eye movements (saccades) to peripheral targets is presented, and a dual-mechanism theory for maintenance of spatial accuracy is introduced to provide a coherent explanation for a controversial body of experimental data.

Hebbian learning reconsidered: Representation of static and dynamic objects in associative neural nets

It turns out that pure Hebbian learning is by selection: the network produces synaptic representations that are selected according to their resonance with the input percepts.

A cortico-subcortical model for generation of spatially accurate sequential saccades.

A major thesis of this model is that a topography of saccade direction and amplitude is preserved through multiple projections between brain regions until they are finally transformed into a temporal pattern of activity that drives the eyes to the target.

Neural activity in the caudate nucleus of monkeys during spatial sequencing

In a majority of cells, activation did not depend only on the retinal position of the stimuli or on the spatial parameters of gaze and arm movements, but was contingent on the particular sequence in which the targets were illuminated or the movements were performed.

Responses of monkey midbrain dopamine neurons during delayed alternation performance

Prefrontal cortex and spatial sequencing in macaque monkey

Properties of VT and context cells revealed in the present experiment are consistent with the idea that the superior arcuate area is involved in temporally and spatially extended structures of behavior.