Quantifying machine influence over human forecasters

@article{Abeliuk2020QuantifyingMI,
  title={Quantifying machine influence over human forecasters},
  author={Andr{\'e}s Abeliuk and Daniel Benjamin and Fred Morstatter and A. G. Galstyan},
  journal={Scientific Reports},
  year={2020},
  volume={10}
}
Crowdsourcing human forecasts and machine learning models each show promise in predicting future geopolitical outcomes. Crowdsourcing increases accuracy by pooling knowledge, which mitigates individual errors. On the other hand, advances in machine learning have led to machine models that increase accuracy due to their ability to parameterize and adapt to changing environments. To capitalize on the unique advantages of each method, recent efforts have shown improvements by “hybridizing… 
Humans rely more on algorithms than social influence as a task becomes more difficult
TLDR
It is found that people rely more on algorithmic advice relative to social influence as tasks become more difficult, and subjects tended to more strongly disregard inaccurate advice labeled as algorithmic compared to equally inaccurate advicelabeled as coming from a crowd of peers.
A Brief Taxonomy of Hybrid Intelligence
TLDR
A brief taxonomy of hybrid intelligence is offered, which describes possible relationships between human and machine intelligence for robust forecasting and the interpretation of recent studies in hybrid intelligence as precursors of a set of algorithms that are expected to be more prominent in the future.
A time-critical crowdsourced computational search for the origins of COVID-19
TLDR
A successful investigation on the origins of coronavirus disease 2019 will, I believe, benefit from implementing five principles: incentive structures, transparency, unbiased search, crowdsourcing and human–machine computational sense-making.
Human social sensing is an untapped resource for computational social science.
TLDR
It is argued that, through the lens of computational social science, this ability to 'sense' the social environment and thereby to understand the thoughts and actions of others can be used to advance research into human sociality.
Comparing Human and Machine Deepfake Detection with Affective and Holistic Processing
TLDR
The results suggest that considering emotional influences and harnessing specialized, holistic visual processing of ordinary people could be promising defenses against machine-manipulated media.
Deepfake detection by human crowds, machines, and machine-informed crowds
TLDR
It is found that manipulations designed to disrupt visual processing of faces hinder human participants’ performance while mostly not affecting the model’s performance, suggesting a role for specialized cognitive capacities in explaining human deepfake detection performance.
Models, Markets, and the Forecasting of Elections
TLDR
It is concluded that there is value in using hybrid forecasting methods, and a market design is proposed that incorporates model forecasts via a trading bot to generate synthetic predictions.

References

SHOWING 1-10 OF 104 REFERENCES
Dynamically Forming a Group of Human Forecasters and Machine Forecaster for Forecasting Economic Indicators
TLDR
An ensemble method for estimating the expected error of a machine forecast and dynamically determining the optimal number of humans included in the ensemble is proposed and attained the highest forecast accuracy for four datasets and the same accuracy as the highest one of traditional methods for two datasets.
Algorithm aversion: people erroneously avoid algorithms after seeing them err.
TLDR
It is shown that people are especially averse to algorithmic forecasters after seeing them perform, even when they see them outperform a human forecaster, and this phenomenon, which is called algorithm aversion, is costly, and it is important to understand its causes.
Identifying and Cultivating Superforecasters as a Method of Improving Probabilistic Predictions
  • B. Mellers, Eric Stone, P. Tetlock
  • Psychology
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2015
TLDR
The findings suggest that superforecasters are partly discovered and partly created—and that the high-performance incentives of tournaments highlight aspects of human judgment that would not come to light in laboratory paradigms focused on typical performance.
Human Decisions and Machine Predictions
TLDR
While machine learning can be valuable, realizing this value requires integrating these tools into an economic framework: being clear about the link between predictions and decisions; specifying the scope of payoff functions; and constructing unbiased decision counterfactuals.
Network dynamics of social influence in the wisdom of crowds
TLDR
This work presents theoretical predictions and experimental results showing that, in decentralized communication networks, group estimates become reliably more accurate as a result of information exchange and shows that the dynamics of group accuracy change with network structure.
How social influence can undermine the wisdom of crowd effect
TLDR
This work demonstrates by experimental evidence that even mild social influence can undermine the wisdom of crowd effect in simple estimation tasks.
SAGE: A Hybrid Geopolitical Event Forecasting System
TLDR
This work presents Synergistic Anticipation of Geopolitical Events (SAGE), a platform for human/computer interaction that facilitates human reasoning with machine models.
Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them
TLDR
Giving participants the freedom to modify an imperfect algorithm made them feel more satisfied with the forecasting process, more likely to believe that the algorithm was superior, and morelikely to choose to use an algorithm to make subsequent forecasts.
Effects of Social Influence on the Wisdom of Crowds
TLDR
It is shown that the question of whether social inuence has a positive or negative net eect on the wisdom of crowds is ill-dened, and it is the starting conguration of the population, in terms of its diversity and accuracy, that directly determines how benecial social inUence actually is.
The relative influence of advice from human experts and statistical methods on forecast adjustments
Decision makers and forecasters often receive advice from different sources including human experts and statistical methods. This research examines, in the context of stock price forecasting, how the
...
1
2
3
4
5
...