When is it adaptive to be patient? A general framework for evaluating delayed rewards

@article{Fawcett2012WhenII,
  title={When is it adaptive to be patient? A general framework for evaluating delayed rewards},
  author={Tim W. Fawcett and John M. McNamara and Alasdair Houston},
  journal={Behavioural Processes},
  year={2012},
  volume={89},
  pages={128-136}
}
The tendency of animals to seek instant gratification instead of waiting for greater long-term benefits has been described as impatient, impulsive or lacking in self-control. How can we explain the evolution of such seemingly irrational behaviour? Here we analyse optimal behaviour in a variety of simple choice situations involving delayed rewards. We show that preferences for more immediate rewards should depend on a variety of factors, including whether the choice is a one-off or is likely to… 
Prospection and the Present Moment: The Role of Episodic Foresight in Intertemporal Choices between Immediate and Delayed Rewards
Humans are capable of imagining future rewards and the contexts in which they may be obtained. Functionally, intertemporal choices between smaller but immediate and larger but delayed rewards may be
Rational temporal predictions can underlie apparent failures to delay gratification.
TLDR
It is concluded that delay-of-gratification failure, generally viewed as a manifestation of limited self-control capacity, can instead arise as an adaptive response to the perceived statistics of one's environment.
Postreward delays and systematic biases in measures of animal temporal discounting
TLDR
The data suggest that measured intertemporal preferences in animals may not reflect impulsivity, or even mental discounting of future options, and that standard human and animal intertem temporal choice tasks measure unrelated mental processes.
Decision makers calibrate behavioral persistence on the basis of time-interval experience
TLDR
The results show that human decision makers are responsive to statistical timing cues, modulating their level of persistence according to the distribution of delay durations they encounter, and conclude that temporal expectations act as a powerful and adaptive influence on people's tendency to sustain patient decisions.
When is it worth waiting for? Food quantity, but not food quality, affects delay tolerance in tufted capuchin monkeys
TLDR
Capuchins based their temporal preferences on the quantity of the smaller sooner option, rather than on that of the larger later option, probably because in the wild they virtually never have to choose between the above two options at the same time, but they more often encounter them consecutively.
Medial prefrontal cortical activity reflects dynamic re-evaluation during voluntary persistence
TLDR
An experiment in which human decision makers waited for rewards in two environments in which reward-timing statistics favored either a greater or lesser degree of behavioral persistence found that decision makers adaptively calibrated their level of persistence for each environment.
Delay discounting: Pigeon, rat, human--does it matter?
TLDR
A comparison of discounting models and the procedures used to assess delay discounting in nonhuman animals and an adaptationist perspective are discussed, suggesting it is a fundamental process underlying decision making.
Why has evolution not selected for perfect self-control?
  • B. Hayden
  • Psychology, Medicine
    Philosophical Transactions of the Royal Society B
  • 2018
TLDR
It is proposed that self-control failures result from cases in which well-learned (and thus routinized) decision-making strategies yield suboptimal choices, and that poor self- control, in this view, is ultimately a result of bounded optimality.
Waiting for better, not for more: corvids respond to quality in two delay maintenance tasks
TLDR
This study is the first to test crows and ravens with two different paradigms, the accumulation and the exchange of food, within the same experiment, allowing for fair comparisons between methods and species.
Why hasn’t evolution selected for perfect self-control?
Self-control refers to the ability to deliberately reject tempting options and instead select ones that produce greater long-term benefits. Although some apparent failures of self-control are, on
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 64 REFERENCES
Discrimination, discounting and impulsivity: a role for an informational constraint.
  • D. Stephens
  • Economics, Medicine
    Philosophical transactions of the Royal Society of London. Series B, Biological sciences
  • 2002
TLDR
This paper presents a new explanation of short-sighted decision-making called the discrimination advantage model that does not rely on discounting, and develops a new model that includes several possible causes of discounting.
Is a bird in the hand worth two in the future? The neuroeconomics of intertemporal decision-making
TLDR
The most recent literature on the behavioural and neural processes underlying intertemporal choices are reviewed, and to which extent these findings can be used to explain violations of DUT's assumptions are elucidated.
Exponential Versus Hyperbolic Discounting of Delayed Outcomes: Risk and Waiting Time
TLDR
The research demonstrates that, of the two, a hyperbola-like discounting model consistently explains more of the variance in temporal discounting data at the group level and, importantly, at the individual level as well.
The discounting-by-interruptions hypothesis: model and experiment
TLDR
A simple model of discounting by interruptions is developed and tested and the results do not support the discounting-by-interruptions hypothesis, one of several lines of evidence suggesting that investigators should seek alternative explanations of the animal impulsivity.
When foragers discount the future: constraint or adaptation?
Abstract Foraging currencies based on average rates of energy gain often fail to predict observed behaviour when an animal must choose between a smaller, more immediate reward and a larger, more
Uncertainty and Hyperbolic Discounting
We propose an evolutionary explanation for the pattern of intertemporal preference reversals often ascribed to "hyperbolic discounting." We take the view that preferences—manifested, for example, in
Loss of self-control in intertemporal choice may be attributable to logarithmic time-perception.
TLDR
It is hypothesized that empirically-observed dynamical inconsistency in intertemporal choice may result from errors in the perception of time-duration and proposed that perception of temporal duration following Weber's law might explain the dynamical consistency.
The adaptive value of preference for immediacy: when shortsighted rules have farsighted consequences
TLDR
This study tests the feeding preferences of captive blue jays in two economically equivalent situations, and explains how a rule of preference based on short-term rate comparisons can account for both situations.
Impulsiveness without discounting: the ecological rationality hypothesis
TLDR
It is found that impulsive rules are optimal in a simple foreground–background choice situation in the absence of discounting, and that comparable impulsiveness is not found in binary choice situations even when there is strong discounting.
Discounting of delayed rewards: Models of individual choice.
  • J. Myerson, L. Green
  • Psychology, Medicine
    Journal of the experimental analysis of behavior
  • 1995
TLDR
Two possible derivations of the temporal discounting function are proposed, a repeated choice model and an expected value model, which provide theoretical interpretations for amount-dependent discounting but amount-independent exponent parameters.
...
1
2
3
4
5
...