# Partial Information Framework: Aggregating Estimates from Diverse Information Sources

@article{Satop2015PartialIF, title={Partial Information Framework: Aggregating Estimates from Diverse Information Sources}, author={Ville A. Satop{\"a}{\"a} and Shane T. Jensen and Robin Pemantle and Lyle H. Ungar}, journal={arXiv: Methodology}, year={2015} }

Prediction polling is an increasingly popular form of crowdsourcing in which multiple participants estimate the probability or magnitude of some future event. These estimates are then aggregated into a single forecast. Historically, randomness in scientific estimation has been generally assumed to arise from unmeasured factors which are viewed as measurement noise. However, when combining subjective estimates, heterogeneity stemming from differences in the participants' information is often…

## 4 Citations

### Partial Information Framework: Basic Theory and Applications

- Computer Science
- 2016

This dissertation shows that measurement error is not appropriate for modeling forecast heterogeneity and then introduces information diversity as a more appropriate yet fundamentally different alternative and the Gaussian partial information model, a very close yet practical specification of the framework.

### Can Investors Profit from Information Diveristy? The Wisdom of Crowds in Security Analyst Recommendations

- Economics
- 2017

There is heterogeneity in individual forecasts of any variable — inflation, corporate earnings, etc. The standard consensus estimate takes a simple average of individual forecasts, implicitly…

### Combining and Extremizing Real-Valued Forecasts

- Economics, Computer Science
- 2015

This paper proposes a linear extremization technique for improving the weighted average of real-valued forecasts and the resulting more extreme version of the weightedAverage exhibits many properties of optimal aggregation.

## References

SHOWING 1-10 OF 25 REFERENCES

### Modeling Probability Forecasts via Information Diversity

- Computer Science
- 2014

A novel framework that uses partially overlapping information sources is proposed and applied to the task of aggregating the probabilities given by a group of forecasters who predict whether an event will occur or not and gives a more principled understanding of the historically ad hoc practice of extremizing average forecasts.

### Characterizing and aggregating agent estimates

- Computer Science, EconomicsAAMAS
- 2013

This work motivates the analysis in the context of a crowdsourced forecasting application, demonstrates the central insights theoretically, and shows how these insights manifest them-selves in actual data.

### Combining probability forecasts

- Environmental Science
- 2010

Summary. Linear pooling is by far the most popular method for combining probability forecasts. However, any non‐trivial weighted average of two or more distinct, calibrated probability forecasts is…

### Human Judgement is Heavy Tailed: Empirical Evidence and Implications for the Aggregation of Estimates and Forecasts

- Economics
- 2010

How frequent are large disagreements in human judgment? The substantial literature relating to expert assessments of real-valued quantities and their aggregation almost universally assumes that…

### Prediction without markets

- EconomicsEC '10
- 2010

It is found that the relative advantage of prediction markets is surprisingly small, as measured by squared error, calibration, and discrimination, and as policy makers consider adoption, costs should be weighed against potentially modest benefits.

### Probability Elicitation, Scoring Rules, and Competition Among Forecasters

- EconomicsManag. Sci.
- 2007

It is shown how a decision maker can revise probabilities of an event after receiving reported probabilities from competitive forecasters and note that the strategy of exaggerating probabilities can make well-calibrated forecasters appear to be overconfident.

### The Good Judgment Project: A Large Scale Test of Different Methods of Combining Expert Predictions

- BusinessAAAI Fall Symposium: Machine Aggregation of Human Judgment
- 2012

It is found that teams and prediction markets systematically outperformed averages of individual forecasters, that training forecasters helps, and that the exact form of how predictions are combined has a large effect on overall prediction accuracy.

### Interpreted and generated signals

- Economics, Computer ScienceJ. Econ. Theory
- 2009

### Optimal linear opinion pools

- Mathematics
- 1991

Consider a decision problem involving a group of m Bayesians in which each member reports his/her posterior distribution for some random variable \theta . The individuals all share a common prior…

### Psychological Strategies for Winning a Geopolitical Forecasting Tournament

- PsychologyPsychological science
- 2014

Support is found for three psychological drivers of accuracy: training, teaming, and tracking in a 2-year geopolitical forecasting tournament that produced the best forecasts 2 years in a row.