• Corpus ID: 231951374

Multilevel calibration weighting for survey data

@inproceedings{BenMichael2021MultilevelCW,
  title={Multilevel calibration weighting for survey data},
  author={Eli Ben-Michael and Avi Feller and Erin Hartman},
  year={2021}
}

• E. Ben-Michael, A. Feller, E. Hartman
• Published 17 February 2021
• Mathematics

In the November 2016 U.S. presidential election, many state level public opinion polls, particularly in the Upper Midwest, incorrectly predicted the winning candidate. One leading explanation for this polling miss is that the precipitous decline in traditional polling response rates led to greater reliance on statistical methods to adjust for the corresponding bias—and that these methods failed to adjust for important interactions between key variables like education, race, and geographic region. Finding calibration weights that account for important interactions remains challenging with traditional survey methods: raking typically balances the margins alone, while post-stratification, which exactly balances all interactions, is only feasible for a small number of variables. In this paper, we propose multilevel calibration weighting, which enforces tight balance constraints for marginal balance and looser constraints for higher-order interactions. This incorporates some of the benefits of post-stratification while retaining the guarantees of raking. We then correct for the bias due to the relaxed constraints via a flexible outcome model; we call this approach Double Regression with Post-stratification (DRP). We characterize the asymptotic properties of these estimators and show that the proposed calibration approach has a dual representation as a multilevel model for survey response. We then use these tools to to re-assess a large-scale survey of voter intention in the 2016 U.S. presidential election, finding meaningful gains from the proposed methods. The approach is available in the multical R package. We would like to thank Peng Ding, Yair Ghitza, Mark Handcock, Luke Keele, Shiro Kuriwaki, Drew Linzer, Luke Miratrix, Doug Rivers, Jonathan Robinson, Jas Sekhon, Betsy Sinclair, Brandon Stewart, and participants at BigSurv2020, AsianPolmeth VIII, and Polmeth 2021 for useful discussion and comments. This research was supported in part by the Hellman Family Fund at UC Berkeley and by the Institute of Education Sciences, U.S. Department of Education, through Grant R305D200010. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education. ar X iv :2 10 2. 09 05 2v 2 [ st at .M E ] 1 2 N ov 2 02 1