Disentangling Bias and Variance in Election Polls

  title={Disentangling Bias and Variance in Election Polls},
  author={Houshmand Shirani-mehr and David M. Rothschild and Sharad Goel and Andrew Gelman},
  journal={Journal of the American Statistical Association},
  pages={607 - 614}
ABSTRACT It is well known among researchers and practitioners that election polls suffer from a variety of sampling and nonsampling errors, often collectively referred to as total survey error. Reported margins of error typically only capture sampling variability, and in particular, generally ignore nonsampling errors in defining the target population (e.g., errors due to uncertainty in who will vote). Here, we empirically analyze 4221 polls for 608 state-level presidential, senatorial, and… 

Bias and Variance in Multiparty Election Polls

Prominent polling failures highlight that election polls are prone to biases that the “margin of error” customarily reported with polls does not capture. However, such systematic errors are difficult

Bias and Excess Variance in Election Polling: A Not-So-Hidden Markov Model

With historic misses in the 2016 and 2020 US Presidential elections, interest in measuring polling errors has increased. The most common method for measuring directional errors and non-sampling

The Twilight of the Polls? A Review of Trends in Polling Accuracy and the Causes of Polling Misses

Polls have had a number of high-profile misses in recent elections. We review the current polling environment, the performance of polls in a historical context, the mechanisms of polling error, and

Biased polls and the psychology of voter indecisiveness

It is shown that the allocation of undecided voters in the 2016 US presidential election biased polls and meta-polls in a manner consistent with psychological theories of decision making, implying that it will be important to take into account the underlying psychology of voting when making predictions about elections.

Can Biased Polls Distort Electoral Results? Evidence From The Lab

Biased exposure of voters to the outcome of polls constitutes a risk to the principle of balanced and impartial elections. We first show empirically how modern communication (through social media)

Polling bias and undecided voter allocations: US presidential elections, 2004–2016

Accounting for undecided and uncertain voters is a challenging issue for predicting election results from public opinion polls. Undecided voters typify the uncertainty of swing voters in polls but

Understanding the 2016 US Presidential Polls: The Importance of Hidden Trump Supporters

Abstract Following Donald Trump’s unexpected victory in the 2016 US presidential election, the American Association for Public Opinion Research announced that “the polls clearly got it wrong” and

Towards Principled Unskewing: Viewing 2020 Election Polls Through a Corrective Lens from 2016

Under hypothetical scenarios that the magnitude of the underlying nonresponse bias correlated with supporting Donald Trump is similar to that of the 2016 polls, the pollsters’ ability to correct systematic errors via weighting has not improved significantly, and turnout levels remain similar to 2016, Donald Trump’s estimated two-party voteshare estimate is shifted by a percentage point in his favor in the median battleground state.

Non-Representative Surveys: Modes, Dynamics, Party, and Likely Voter Space

State-level forecasts of the popular vote in the 2016 presidential election experienced a large, correlated error in a number of ultimately pivotal states. These forecasts were created by

Predicting elections: Experts, polls, and fundamentals

This study analyzes the relative accuracy of experts, polls, and the so-called ‘fundamentals’ in predicting the popular vote in the four U.S. presidential elections from 2004 to 2016. Although



The Mythical Swing Voter

It is concluded that vote swings in 2012 were mostly sample artifacts and that real swings were quite small, which better corresponds with the understanding of partisan polarization in modern American politics.

Pre-Election Survey Methodology: Details from Nine Polling Organizations, 1988 and 1992

Before every presidential election, journalists, pollsters, and politicians commission dozens of public opinion polls. Although the primary function of these surveys is to forecast the election

A method for estimating design-based sampling variances for surveys with weighting, poststratification, and

It is common practice to use weighting, poststratiication, and raking to correct for sampling and nonsampling biases and to improve eeciency of estimation in sample surveys. However, there is no

Effects of Question Order on Survey Responses

Two question orders were used in a Kentucky statewide random survey. In nonoverlapping sections, four questions which asked the respondents to express their general interests in politics and religion

Total Survey Error: Design, Implementation, and Evaluation

The total survey error (TSE) paradigm provides a theoretical framework for optimizing surveys by maximizing data quality within budgetary constraints. In this article, the TSE paradigm is viewed as


D. STEPHEN voss is a doctoral candidate in the Department of Government of Harvard University. ANDREW GELMAN is assistant professor in the Department of Statistics in the University of California,

Sampling: Design and Analysis

In his seminal book, Shewhart (1931) makes no demand on the distribution of the characteristic to be plotted on a control chart. How then can we explain the idea that normality is, if not required,

The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo

The No-U-Turn Sampler (NUTS), an extension to HMC that eliminates the need to set a number of steps L, and derives a method for adapting the step size parameter {\epsilon} on the fly based on primal-dual averaging.

Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper)

Various noninformative prior distributions have been suggested for scale parameters in hierarchical models. We construct a new folded-noncentral-t family of conditionally conjugate priors for

Conservative prior distributions for variance parameters in hierarchical models

Bayesian hierarchical models typically involve specifying prior distributions for one or more variance components. This is rather removed from the observed data, so specification based on expert