Behavioral considerations suggest an average reward TD model of the dopamine system

  title={Behavioral considerations suggest an average reward TD model of the dopamine system},
  author={Nathaniel D. Daw and David S. Touretzky},
Recently there has been much interest in modeling the activity of primate midbrain dopamine neurons as signalling reward prediction error. But since the models are based on temporaldi!erence (TD) learning, they assume an exponential decline with time in the value of delayed reinforcers, an assumption long known to con#ict with animal behavior. We show that a variant of TD learning that tracks variations in the average reward per timestep rather than cumulative discounted reward preserves the… CONTINUE READING

From This Paper

Figures, tables, and topics from this paper.


Publications citing this paper.


Publications referenced by this paper.
Showing 1-10 of 12 references

An adjusting procedure for studying delayed reinforcement

J. E. Mazur
in: M.L. Commons, J.E. Mazur, J.A. Nevin, H. Rachlin (Eds.), Quantitative Analyses of Behavior, Vol. V, Erlbaum, Hillsdale • 1987
View 7 Excerpts
Highly Influenced


C. R. Gallistel, J. Gibbon
rate and conditioning • 2000
View 1 Excerpt

Predictive reward signal of dopamine neurons.

Journal of neurophysiology • 1998
View 2 Excerpts

A framework for mesencephalic dopamine systems based on predictive Hebbian learning.

The Journal of neuroscience : the official journal of the Society for Neuroscience • 1996
View 3 Excerpts

A model of how the basal ganglia generate and use neural signals that predict reinforcement

J. C. Houk, J. L. Adams, A. G. Barto
in: J.C. Houk, J.L. Davis, D.G. Beiser (Eds.), Models of Information Processing in the Basal Ganglia, MIT Press, Cambridge • 1995
View 2 Excerpts

Choice between delayed reinforcers in a discrete-trials schedule: the effect of deprivation level.

The Quarterly journal of experimental psychology. B, Comparative and physiological psychology • 1992
View 1 Excerpt

Similar Papers

Loading similar papers…