Intuitions About Combining Opinions: Misappreciation of the Averaging Principle

@article{Larrick2006IntuitionsAC,
  title={Intuitions About Combining Opinions: Misappreciation of the Averaging Principle},
  author={Richard P. Larrick and Jack B. Soll},
  journal={Manag. Sci.},
  year={2006},
  volume={52},
  pages={111-127}
}
Averaging estimates is an effective way to improve accuracy when combining expert judgments, integrating group members judgments, or using advice to modify personal judgments. If the estimates of two judges ever fall on different sides of the truth, which we term bracketing, averaging must outperform the average judge for convex loss functions, such as mean absolute deviation (MAD). We hypothesized that people often hold incorrect beliefs about averaging, falsely concluding that the average of… 

Figures and Tables from this paper

Extracting the collective wisdom in probabilistic judgments

  • Cem Peker
  • Computer Science
    SSRN Electronic Journal
  • 2022
This paper proposes an algorithm to aggregate probabilistic judgments under shared information and shows that if average prediction is a consistent estimator, the percentage of predictions and meta-predictions that exceed the average prediction should be the same.

Strategies for revising judgment: how (and how well) people use others' opinions.

The authors developed the probability, accuracy, redundancy (PAR) model and found that averaging was the more effective strategy across a wide range of commonly encountered environments and that despite this finding, people tend to favor the choosing strategy.

How the “wisdom of the inner crowd” can boost accuracy of confidence judgments.

Simulation and analytical results show that irrespective of the type of item, averaging consistently improves confidence judgments, but maximizing is risky: It outperformed averaging only once items were answered correctly 60% of the time or more—a result that has not been established in prior work.

Repeated judgment sampling: Boundaries

This paper investigates the boundaries of the recent result that eliciting more than one estimate from the same person and averaging these can lead to accuracy gains in judgment tasks. It first

Spurious consensus and opinion revision: why might people be more confident in their less accurate judgments?

The authors theorize that people tend to underestimate the informative value of independently drawn opinions, if these appear to conflict with one another, yet place some confidence even in the spurious consensus, which may arise when opinions are sampled interdependently.

Taking a Disagreeing Perspective Improves the Accuracy of People’s Quantitative Estimates

Many decisions rest on people’s ability to make estimates of unknown quantities. In these judgments, the aggregate estimate of a crowd of individuals is often more accurate than most individual

The Wisdom of Many in One Mind

The conditions under which dialectical bootstrapping fosters accuracy are derived and an empirical demonstration that its benefits go beyond reliability gains is provided.

The wisdom of select crowds.

The select-crowd strategy, which ranks judges based on a cue to ability and averages the opinions of the top judges, is introduced and is shown to be accurate, robust, and appealing as a mechanism for helping individuals tap collective wisdom.
...

References

SHOWING 1-10 OF 68 REFERENCES

The effects of averaging subjective probability estimates between and within judges.

Two studies test 3 predictions regarding averaging that follow from theorems based on a cognitive model of the judges and idealizations of the judgment situation, showing the extent to which they hold as the information conditions depart from the ideal and as J increases.

Averaging probability judgments: Monte Carlo analyses of asymptotic diagnostic value

Wallsten et al. (1997) developed a general framework for assessing the quality of aggregated probability judgments. Within this framework they presented a theorem regarding the effects of pooling

Intuitive Theories of Information: Beliefs about the Value of Redundancy

The present experiments show that the preference for redundancy depends on one's intuitive theory of information, and lends insight into how intuitive theories might develop and also has potential ramifications for how statistical concepts such as correlation might best be learned and internalized.

Representativeness revisited: Attribute substitution in intuitive judgment.

The program of research now known as the heuristics and biases approach began with a survey of 84 participants at the 1969 meetings of the Mathematical Psychology Society and the American

Individual differences in reasoning: Implications for the rationality debate?

In a series of experiments involving most of the classic tasks in the heuristics and biases literature, the implications of individual differences in performance for each of the four explanations of the normative/descriptive gap are examined.

Taking Advice: Accepting Help, Improving Judgment, and Sharing Responsibility☆☆☆

Abstract Why do people take advice? To find out, we provided a low, medium, or high level of training on a task in which judgments varied in importance. Then, in a test session, we eliminated

Evaluating and Combining Subjective Probability Estimates

This paper concerns the evaluation and combination of subjective probability estimates for categorical events. We argue that the appropriate criterion for evaluating individual and combined estimates

When Words Speak Louder Than Actions: Another's Evaluations Can Appear More Diagnostic Than Their Decisions

Abstract When making decisions we often infer the value of the alternatives from the behavior of others. In some domains, however, we fail to make such inferences. Stock trading is a case in
...