The Crowd Classification Problem: Social Dynamics of Binary-Choice Accuracy

@article{Becker2022TheCC,
  title={The Crowd Classification Problem: Social Dynamics of Binary-Choice Accuracy},
  author={Joshua Aaron Becker and Douglas Guilbeault and Edward Bishop Smith},
  journal={Manag. Sci.},
  year={2022},
  volume={68},
  pages={3949-3965}
}
Decades of research suggest that information exchange in groups and organizations can reliably improve judgment accuracy in tasks such as financial forecasting, market research, and medical decision making. However, we show that improving the accuracy of numeric estimates does not necessarily improve the accuracy of decisions. For binary-choice judgments, also known as classification tasks—for example, yes/no or build/buy decisions—social influence is most likely to grow the majority vote share… 
3 Citations

Figures and Tables from this paper

Probabilistic social learning improves the public’s judgments of news veracity
TLDR
Exchanging probabilistic estimates of news veracity substantially improved individual and group judgments, with the effect of eliminating polarization in news evaluation, and exchanging binary classifications reduced social learning and maintained polarization.
The reduction of race and gender bias in clinical treatment recommendations using clinician peer networks in an experimental setting
TLDR
Clinicians who exchanged real-time information in structured peer networks significantly improved their clinical accuracy and showed no bias in their final recommendations, indicating that clinician network interventions might be used in healthcare settings to reduce significant disparities in patient treatment.
Human-Algorithm Ensembles
TLDR
It is argued that human and AI-based algorithmic decision making can be ensembled even when neither has a clear advantage over the other at a decision task or its sub-components, and even if neither alone can attain satisfactory accuracy in absolute terms.

References

SHOWING 1-10 OF 71 REFERENCES
The Power of Social Influence on Estimation Accuracy
Research shows that crowds can provide more accurate estimates of uncertain quantities than individuals (Surowiecki, 2004). But little is known about how to organize crowd members to maximize
How social influence can undermine the wisdom of crowd effect
TLDR
This work demonstrates by experimental evidence that even mild social influence can undermine the wisdom of crowd effect in simple estimation tasks.
Aggregated knowledge from a small number of debates outperforms the wisdom of large crowds
The aggregation of many independent estimates can outperform the most accurate individual judgement1–3. This centenarian finding1,2, popularly known as the 'wisdom of crowds'3, has been applied to
Identifying Expertise to Extract the Wisdom of Crowds
TLDR
A new measure of contribution is proposed to assess the judges' performance relative to the group and positive contributors are used to build a weighting model for aggregating forecasts, showing that the model derives its power from identifying experts who consistently outperform the crowd.
The wisdom of select crowds.
TLDR
The select-crowd strategy, which ranks judges based on a cue to ability and averages the opinions of the top judges, is introduced and is shown to be accurate, robust, and appealing as a mechanism for helping individuals tap collective wisdom.
Network dynamics of social influence in the wisdom of crowds
TLDR
This work presents theoretical predictions and experimental results showing that, in decentralized communication networks, group estimates become reliably more accurate as a result of information exchange and shows that the dynamics of group accuracy change with network structure.
How social information can improve estimation accuracy in human groups
TLDR
It is found that social influence helps the group improve its properly defined collective accuracy and provide a moderate amount of incorrect information to individuals can counterbalance the human cognitive bias to systematically underestimate quantities and thereby improve collective performance.
Extracting the Wisdom of Crowds When Information Is Shared
TLDR
An aggregation method is developed, called pivoting, which separates individual forecasts into shared and private information and then recombines these results in the optimal manner and suggests that the pivoting method provides an effective forecast aggregation procedure that can significantly outperform the simple crowd average.
Are crowds on the internet wiser than experts? The case of a stock prediction community
Abstract According to the “Wisdom of Crowds” phenomenon, a large crowd can perform better than smaller groups or few individuals. This article investigates the performance of share recommendations,
Network Structures of Collective Intelligence: The Contingent Benefits of Group Discussion
TLDR
This paper shows how network theories of belief formation can resolve contradictions in previous research and offer practical recommendations for teams and organizations.
...
...