Hierarchical control of goal-directed action in the cortical–basal ganglia network

@article{Balleine2015HierarchicalCO,
  title={Hierarchical control of goal-directed action in the cortical–basal ganglia network},
  author={Bernard W. Balleine and Amir Dezfouli and Makato Ito and Kenji Doya},
  journal={Current Opinion in Behavioral Sciences},
  year={2015},
  volume={5},
  pages={1-7}
}
A generative spiking neural-network model of goal-directed behaviour and one-step planning
TLDR
The architecture represents the first instance of a model bridging probabilistic planning and spiking-processes that has a degree of autonomy analogous to the one of real organisms.
Striatal and hippocampal contributions to flexible navigation in rats and humans
TLDR
It is argued the dorsal striatum encodes the transition structure of the environment during flexible, goal-directed behaviour and should be investigated via a biophysically plausible framework explained by reinforcement learning models.
Computational evidence for hierarchically structured reinforcement learning in humans
TLDR
Evidence is presented that hierarchical RL, a biologically inspired and computationally simple algorithm, can capture human behavior in complex, hierarchical environments and opens the avenue for future research in this field.
A spiking neural-network model of goal-directed behaviour
TLDR
This work proposes a model of goal-directed behaviour that has a probabilistic interpretation and is centred on a recurrent spiking neural network representing the world model and presents three main novelties: it can plan even while it is still learning the consequences of actions by deciding if planning or exploring the environment based on how confident it is on its predictions.
Computational evidence for hierarchically-structured reinforcement learning in humans
TLDR
Evidence is presented that hierarchical RL, a biologically-inspired and computationally cheap algorithm, can reproduce human behavior in complex, hierarchical environments, and opens the avenue for future research in this field.
Explaining the neural activity distribution associated with discrete movement sequences: Evidence for parallel functional systems
TLDR
The results corroborate C-SMB’s assumption that at advanced skill levels the systems executing motor chunks and translating key-specific stimuli are racing to trigger individual responses and support recent behavioral indications that spatial sequence representations continue to be used.
The evolution of hierarchical structure building capacity for language and music: a bottom-up perspective.
  • R. Asano
  • Biology, Psychology
    Primates; journal of primatology
  • 2021
TLDR
It is argued that the evolution of hierarchical structure building capacity for language and music is tractable for comparative evolutionary study once the authors focus on the gradual elaboration of shared brain architecture: the cortico-basal ganglia-thalamocortical circuits for hierarchical control of goal-directed action and the dorsal pathways for hierarchical internal models.
Predictive representations can link model-based reinforcement learning to model-free mechanisms
TLDR
This work lays out a family of approaches by which model-based computation may be built upon a core of TD learning, and suggests that this framework represents a neurally plausible family of mechanisms for model- based evaluation.
Predictive representations can link model-based reinforcement learning to model-free mechanisms
TLDR
This work lays out a family of approaches by which model-based computation may be built upon a core of TD learning, and shows that the types of behaviors associated with either process could result from a common learning mechanism applied to different strategies for how populations of neurons could represent candidate actions.
Striatal Neuropeptides Enhance Selection and Rejection of Sequential Actions
TLDR
It is demonstrated that diffuse neuropeptide connectivity enhanced the selection of unordered action requests, and that for true action sequences, a patterning of the SP connectivity reflecting this ordering enhanced selection of actions presented in the correct sequential order and suppressed incorrect ordering.
...
1
2
3
4
...

References

SHOWING 1-10 OF 67 REFERENCES
The Basal Ganglia and Chunking of Action Repertoires
  • A. Graybiel
  • Biology, Psychology
    Neurobiology of Learning and Memory
  • 1998
TLDR
It is proposed that one aspect of basal ganglia-based learning is the recoding of cortically derived information within the striatum that can chunk the representations of motor and cognitive action sequences so that they can be implemented as performance units.
Actions, Action Sequences and Habits: Evidence That Goal-Directed and Habitual Action Control Are Hierarchically Organized
TLDR
It is found that human subjects combined single actions to build action sequences and that the formation of such action sequences was sufficient to explain habitual actions, and based on Bayesian model comparison, a family of hierarchical RL models, assuming a hierarchical interaction between habit and goal-directed processes, provided a better fit of the subjects' behavior than aFamily of flat models.
Basal Ganglia Subcircuits Distinctively Encode the Parsing and Concatenation of Action Sequences
TLDR
Direct and indirect pathways of basal ganglia were concomitantly active during sequence initiation, but behaved differently during sequence performance, revealing a more complex functional organization of these circuits than previously postulated.
Premotor cortex is critical for goal-directed actions
TLDR
It is indicated that M2 is critical for actions to be updated based on their consequences, and suggested that habitual action strategies may not require processing by M2 and the updating of motor plans.
Multiple Representations of Belief States and Action Values in Corticobasal Ganglia Loops
TLDR
A unified view is presented on how three subloops of the corticobasal ganglia network are involved in reward prediction and action selection using different types of information.
Internal models in the cerebellum
Habits as action sequences: hierarchical action control and changes in outcome value
TLDR
This hierarchical view suggests that the development of action sequences and the insensitivity of actions to changes in outcome value are essentially two sides of the same coin, explaining why these two aspects of automatic behaviour involve a shared neural structure.
Parallel Cortico-Basal Ganglia Mechanisms for Acquisition and Execution of Visuomotor SequencesA Computational Approach
TLDR
A computational model based on the hypothesis that the visual and motor loops achieve both the quick acquisition of novel sequences and the robust execution of well-learned sequences is examined and found that the dual mechanism with the coordinator was superior to the single (visual or motor) mechanism.
Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action
TLDR
The neural basis of the integration of learning and motivation in choice and decision-making is still controversial and some recent hypotheses relating to this issue are reviewed.
...
1
2
3
4
5
...