Aggregating Experts: Reducing Error and Assessing Reliability

  title={Aggregating Experts: Reducing Error and Assessing Reliability},
  author={Cherie D. Maestas},
This paper lays out a simple two-step approach for aggregating individual level data from expert evaluators to create improved quality measures of U.S. House incumbents that are reliable both within and across districts. The two-step procedure is designed to minimize both systematic biases and random error in evaluator opinions to increase the validity and reliability of the aggregated measure. The result is a measure that has higher intra-district agreement than simple mean aggregations of… CONTINUE READING