This is the ungated final version. Click here for an ungated PDF of “A New Multinomial Accuracy Measure for Polling Bias”.
To install our Stata add-on, type ssc install surveybias or click here.
For replication scripts and data, click on “Data”.
A New Multinomial Accuracy Measure for Polling Bias
1 Introduction
Work on pre-election polls forms a vital part of election analysis and forecasting in both academic publications and media coverage. Frederick Mosteller (Mosteller et al., 1949) introduced the notion of accuracy measures to assess polls against actual election results. Those indices are designed for two-party/-candidate races, and cannot easily be applied to multi-party/-candidate elections. An equivalent index has not so far been derived for multi-party elections, limiting the ability of researchers to measure overall polling accuracy in such cases.
Starting from the predictive accuracy measure proposed by Martin, Traugott and Kennedy (2005), we propose such an accuracy measure, B, for elections with more than two parties or candidates. First, we derive this index mathematically from an implementation of the multinomial logistic model, including the relevant tests of statistical significance. We then consider how this aggregate measure may be biased, given it is based upon compositional data, and use a simulation to examine the extent of this bias. Finally, we use the B measure as the dependent variable in an explanatory model of different sources of polling bias, as an illustration of how the measure may be applied empirically.
The rest of this paper is rather math-heavy and does not display well as HTML. Go here for the PDF version of the preprint.
The final version of this paper should appear later this year in Political Analysis. We have created a Stata command surveybias that implements the methods described in the paper. It is due to appear on SSC this summer.