• Publications
  • Influence
Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
TLDR
This text introduces the basic mathematical and computational methods of theoretical neuroscience and presents applications in a variety of areas including vision, sensory-motor integration, development, learning, memory. Expand
  • 3,137
  • 301
  • PDF
Technical Note: Q-Learning
TLDR
We show that $$\mathcal{Q}$$ -learning converges to the optimum action-values with probability 1 so long as all actions are repeatedly sampled in all states and the action values are represented discretely. Expand
  • 2,747
  • 192
  • PDF
Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control
A broad range of neural and behavioral data suggests that the brain contains multiple systems for behavioral choice, including one associated with prefrontal cortex and another with dorsolateralExpand
  • 1,846
  • 167
  • PDF
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
Instrumental conditioning studies how animals and humans choose actions appropriate to the affective structure of an environment. According to recent reinforcement learning models, two distinctExpand
  • 1,846
  • 111
  • PDF
Model-based influences on humans’ choices and striatal prediction errors
Summary The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free predictionExpand
  • 1,061
  • 101
  • PDF
A framework for mesencephalic dopamine systems based on predictive Hebbian learning
We develop a theoretical framework that shows how mesencephalic dopamine systems could distribute to their targets a signal that represents information about future expectations. In particular, weExpand
  • 1,756
  • 94
  • PDF
Uncertainty, Neuromodulation, and Attention
Uncertainty in various forms plagues our interactions with the environment. In a Bayesian statistical framework, optimal inference and prediction, based on unreliable observations in changingExpand
  • 1,245
  • 84
  • PDF
The Effect of Correlated Variability on the Accuracy of a Population Code
TLDR
We study the impact of correlated neuronal firing rate variability on the accuracy with which an encoded quantity can be extracted from a population of neurons. Expand
  • 748
  • 63
  • PDF
Tonic dopamine: opportunity costs and the control of response vigor
RationaleDopamine neurotransmission has long been known to exert a powerful influence over the vigor, strength, or rate of responding. However, there exists no clear understanding of theExpand
  • 838
  • 53
  • PDF
The Helmholtz Machine
TLDR
We use a Helmholtz machine to learn a hierarchical generative model for maximum likelihood learning in a layered hierarchical connectionist network and show that it is tractable. Expand
  • 1,064
  • 53
  • PDF