The Optimizer's Curse: Skepticism and Postdecision Surprise in Decision Analysis

@article{Smith2006TheOC,
  title={The Optimizer's Curse: Skepticism and Postdecision Surprise in Decision Analysis},
  author={James E. Smith and Robert L. Winkler},
  journal={Manag. Sci.},
  year={2006},
  volume={52},
  pages={311-322}
}
Decision analysis produces measures of value such as expected net present values or expected utilities and ranks alternatives by these value estimates. Other optimization-based processes operate in a similar manner. With uncertainty and limited resources, an analysis is never perfect, so these value estimates are subject to error. We show that if we take these value estimates at face value and select accordingly, we should expect the value of the chosen alternative to be less than its estimate… 

Figures and Tables from this paper

Bayesian Evaluation and Selection Strategies in Portfolio Decision Analysis
Practically all firms pursue goals by selecting a portfolio of courses of action that consume resources. Yet, due to uncertainties such as unforeseen market developments, it may not be possible to
From Data to Decisions: Distributionally Robust Optimization is Optimal
TLDR
This paper proposes a meta-optimization problem to find the least conservative predictors and prescriptors subject to constraints on their out-of-sample disappointment and proves that the best predictor-prescriptor-pair is obtained by solving a distributionally robust optimization problem over all distributions within a given relative entropy distance from the empirical distribution of the data.
Good Choice, Bad Judgment: How Choice Under Uncertainty Generates Overoptimism
TLDR
This work examines a fundamental feature of choice under uncertainty: Overestimating an alternative makes one more likely to choose it, and finds that students and managers exhibited behavior consistent with naïveté toward this relationship between estimation error and choice.
From Noise to Bias: Overconfidence in New Product Forecasting
We study decision behavior in the selection, forecasting, and production for a new product. In a stylized behavioral model and five experiments, we generate new insight into when and why this
From Noise to Bias: Overconfidence in New Product Forecasting
We study decision behavior in the selection, forecasting, and production for a new product. In a stylized behavioral model and five experiments across three populations, we generate new insight into
The Bounded Rationality Bias in Managerial Valuation of Real Options: Theory and Evidence from IT Projects
TLDR
This work shows how managers' valuation of real options are systematically biased by their bounded rationality, and contributes the first set of empirical measures for all key types of realOptions.
Friction and Decision Rules in Portfolio Decision Analysis
TLDR
Three frameworks are presented, each from a different field of study, that provide mathematical tools for studying friction, which suggests that further studies of friction may be worthwhile in portfolio decision analysis.
Estimation of Downside Risks in Project Portfolio Selection
In project portfolio selection, the aim is to choose projects which are expected to offer most value and satisfy relevant risk and other constraints. In this paper, we show that uncertainties about
...
...

References

SHOWING 1-10 OF 19 REFERENCES
Decision Making and Postdecision Surprises.
J. Richard Harrison and James G. March Most ideas of intelligent choice assume that decision making involves estimating the probable future values of currently available alternatives and choosing the
A Note on the Apparent Bias of Net Revenue Estimates for Capital Investment Projects
CONSIDER JOE. His job is evaluating potential investment projects for the XYZ Corporation. For each proposed project, Joe carefully estimates expected cost and revenue streams and calculates expected
Disappointment in Decision Making Under Uncertainty
TLDR
The implications of disappointment, a psychological reaction caused by comparing the actual outcome of a lottery to one's prior expectations, for decision making under uncertainty, are explored and explicit recognition that decision makers may be paying a premium to avoid potential disappointment is provided.
The Reconciliation of Decision Analyses
TLDR
It is shown that errors at decision nodes produce biases and the basic idea underlying this argument is the concept of true utilities and probabilities: this concept is included in a discussion of this concept.
A Perspective on Modeling in Decision Analysis
We use mathematical modeling in decision analysis to help us obtain a profit lottery that is “better” than the one we can assess directly. The “goodness” of the profit lottery is defined by the
The Use and Value of Models in Decision Analysis
TLDR
A normative approach to the use of models is presented in which modeling is viewed as a source of information and the expected value of modeling is defined as the increase in the expectedvalue of the outcome that results from theuse of the model-supplied information.
Decision Analysis Expert Use
TLDR
The present paper develops a structure in which the expert resolution problem may be logically formulated and conceptually solved and a framework is developed which enables a decision maker to encode his state of information concerning an expert.
The Value of Decision Analysis at Eastman Kodak Company, 1990-1999
Because of the one-time nature of typical decision-analysis projects, organizations often have difficulty identifying and documenting their value. Based on Eastman Kodak Company's records for 1990 to
The Economic Value of Analysis and Computation
  • J. Matheson
  • Economics, Business
    IEEE Trans. Syst. Sci. Cybern.
  • 1968
TLDR
This paper shows how the decision analysis approach can be used to determine the most economic method of carrying out computations or analyses, by combining the value structure of the primary decision problem with a model of that procedure.
Competitive Bidding with Dependent Value Estimates
TLDR
A bidding situation in which there is uncertainty about the value of the item of interest is modeled, and the effect of this dependence on the “winner's curse” is studied, and optimal bidding strategies are determined.
...
...