The Crowd Classification Problem: Social Dynamics of Binary-Choice Accuracy

@article{Becker2022TheCC,
  title={The Crowd Classification Problem: Social Dynamics of Binary-Choice Accuracy},
  author={Joshua Aaron Becker and Douglas Guilbeault and Edward Bishop Smith},
  journal={Manag. Sci.},
  year={2022},
  volume={68},
  pages={3949-3965}
}
Decades of research suggest that information exchange in groups and organizations can reliably improve judgment accuracy in tasks such as financial forecasting, market research, and medical decision making. However, we show that improving the accuracy of numeric estimates does not necessarily improve the accuracy of decisions. For binary-choice judgments, also known as classification tasks—for example, yes/no or build/buy decisions—social influence is most likely to grow the majority vote share… 
View PDF on arXiv
4 Citations

Figures and Tables from this paper

4 Citations

Probabilistic social learning improves the public’s judgments of news veracity

Exchanging probabilistic estimates of news veracity substantially improved individual and group judgments, with the effect of eliminating polarization in news evaluation, and exchanging binary classifications reduced social learning and maintained polarization.

Human-Algorithm Ensembles

It is argued that human and AI-based algorithmic decision making can be ensembled even when neither has a clear advantage over the other at a decision task or its sub-components, and even if neither alone can attain satisfactory accuracy in absolute terms.

The reduction of race and gender bias in clinical treatment recommendations using clinician peer networks in an experimental setting

Clinicians who exchanged real-time information in structured peer networks significantly improved their clinical accuracy and showed no bias in their final recommendations, indicating that clinician network interventions might be used in healthcare settings to reduce significant disparities in patient treatment.

The network science of collective intelligence

References

SHOWING 1-10 OF 69 REFERENCES

Social Influence Undermines the Wisdom of the Crowd in Sequential Decision Making

It is argued that this positive effect of social information turns negative if group members do not first contribute an independent vote but instead cast their votes sequentially such that early mistakes can cascade across strings of decision makers.

Counteracting estimation bias and social influence to improve the wisdom of crowds

This work measures individual biases and social influence rules in multiple experiments involving hundreds of individuals performing a classic numerosity estimation task to develop and validate three new aggregation measures that effectively counter sources of collective estimation error.

How social influence can undermine the wisdom of crowd effect

This work demonstrates by experimental evidence that even mild social influence can undermine the wisdom of crowd effect in simple estimation tasks.

Aggregated knowledge from a small number of debates outperforms the wisdom of large crowds

The aggregation of many independent estimates can outperform the most accurate individual judgement1–3. This centenarian finding1,2, popularly known as the 'wisdom of crowds'3, has been applied to…

Identifying Expertise to Extract the Wisdom of Crowds

A new measure of contribution is proposed to assess the judges' performance relative to the group and positive contributors are used to build a weighting model for aggregating forecasts, showing that the model derives its power from identifying experts who consistently outperform the crowd.

Harnessing the Wisdom of Crowds

By tracking user viewing activities, this work monitors the amount of information a user viewed before she makes an earnings forecast and finds the more public information viewed, the less weight she will put on her private information.

The wisdom of select crowds.

The select-crowd strategy, which ranks judges based on a cue to ability and averages the opinions of the top judges, is introduced and is shown to be accurate, robust, and appealing as a mechanism for helping individuals tap collective wisdom.

Network dynamics of social influence in the wisdom of crowds

This work presents theoretical predictions and experimental results showing that, in decentralized communication networks, group estimates become reliably more accurate as a result of information exchange and shows that the dynamics of group accuracy change with network structure.

How social information can improve estimation accuracy in human groups

It is found that social influence helps the group improve its properly defined collective accuracy and provide a moderate amount of incorrect information to individuals can counterbalance the human cognitive bias to systematically underestimate quantities and thereby improve collective performance.

Extracting the Wisdom of Crowds When Information Is Shared

An aggregation method is developed, called pivoting, which separates individual forecasts into shared and private information and then recombines these results in the optimal manner and suggests that the pivoting method provides an effective forecast aggregation procedure that can significantly outperform the simple crowd average.
...