Political Predictors of Polling Errors

• Applicant: Professor Dr. Peter Selb
• Subject Area: Political Science ; Empirical Social Research
• Term: from 2019 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 426500462

• Final Report Year: 2024

Final Report Abstract

The aim of this research project has been to develop a contextual understanding of polling errors and their triggers. Unlike most previous studies, we have taken a cross-election comparative perspective and have put the theoretical focus on characteristics of the electoral contest which have been suspected to encourage polling errors. To this end, we have extended Shirani-Mehr et al.’s (2018) Bayesian approach to disentangling poll bias from variance to accommodate multiple parties. We also increased the model’s temporal flexibility to better distinguishing between poll bias and movements in public opinion over the course of the campaign. These developments are important for making realistic judgments on the accuracy of election polls. Finally, the inclusion of election-level covariates allowed us to test theoretical claims about contextual effects on poll accuracy. We collected theories of polling errors from the literature, and elaborated them within the total survey error framework. To operationalize election features, we relied on a wide variety of data sources and recent advances in the analysis of structured and unstructured data including official data on the voting-eligible population and turnout, campaign expenditure data, Wikipedia entries, predictions from name and facial recognition as well as ideal point models. Our analyses of more than 15,000 election polls from Germany and the US defies much common wisdom about polling failures. An inherent difficulty in determining the election-level sources of polling errors is that pollsters constantly adapt their methods in response to failures. Some observers in academia and industry are therefore skeptical and suspect that polling errors are essentially unpredictable. To facilitate future research, we have built an R package to collect poll results and additional information from the Wikipedia and other sources. As of now, we have been scraping over 74,000 pre-election polls, encompassing more than 2,400 elections across 60 countries from 1979 to present.

Publications

• Did “Trumpists” Perform Worse in US Senate Election Polls?. Significance, 19(3), 28-31.
  Chen, Sina & Körtner, John
  
• Bias and Variance in Multiparty Election Polls. Public Opinion Quarterly, 87(4), 1025-1037.
  Selb, Peter; Chen, Sina; Körtner, John & Bosch, Philipp
  
• Electoral predictors of polling errors. The Journal of Politics.
  Chen, Sina; Kortner, John; Wiederspohn, Jens & Selb, Peter