## Survey Accuracy

**accuracy of pre-election surveys**is a matter of considerable debate. Obviously, any rigorous discussion of bias in opinion polls requires a

**scalar measure of survey accuracy**. Martin, Traugott, and Kennedy (2005) propose such a measure for the two-party case, and in our own work (Arzheimer/Evans 2014), Jocelyn Evans and I demonstrate how can be generalised to the multi-party case, giving rise to a new measure (seriously) and some friends and :

- Arzheimer, Kai and Jocelyn Evans. “A New Multinomial Accuracy Measure for Polling Bias.” Political Analysis 22.1 (2014): 31-44. doi:10.1093/pan/mpt012

[BibTeX] [Abstract] [Download PDF] [HTML] [DATA]

In this article, we propose a polling accuracy measure for multi-party elections based on a generalization of Martin, Traugott, and Kennedy s two-party predictive accuracy index. Treating polls as random samples of a voting population, we first estimate an intercept only multinomial logit model to provide proportionate odds measures of each party s share of the vote, and thereby both unweighted and weighted averages of these values as a summary index for poll accuracy. We then propose measures for significance testing, and run a series of simulations to assess possible bias from the resulting folded normal distribution across different sample sizes, finding that bias is small even for polls with small samples. We apply our measure to the 2012 French presidential election polls to demonstrate its applicability in tracking overall polling performance across time and polling organizations. Finally, we demonstrate the practical value of our measure by using it as a dependent variable in an explanatory model of polling accuracy, testing the different possible sources of bias in the French data.

```
@Article{arzheimer-evans-2013,
author = {Arzheimer, Kai and Evans, Jocelyn},
title = {A New Multinomial Accuracy Measure for Polling Bias },
journal = {Political Analysis},
year = 2014,
abstract = {In this article, we propose a polling accuracy measure for
multi-party elections based on a generalization of Martin,
Traugott, and Kennedy s two-party predictive accuracy index.
Treating polls as random samples of a voting population, we first
estimate an intercept only multinomial logit model to provide
proportionate odds measures of each party s share of the vote, and
thereby both unweighted and weighted averages of these values as a
summary index for poll accuracy. We then propose measures for
significance testing, and run a series of simulations to assess
possible bias from the resulting folded normal distribution across
different sample sizes, finding that bias is small even for polls
with small samples. We apply our measure to the 2012 French
presidential election polls to demonstrate its applicability in
tracking overall polling performance across time and polling
organizations. Finally, we demonstrate the practical value of our
measure by using it as a dependent variable in an explanatory model
of polling accuracy, testing the different possible sources of bias
in the French data.},
keywords = {meth-e},
volume = {22},
number = {1},
pages = {31--44},
url =
{http://pan.oxfordjournals.org/cgi/reprint/mpt012?ijkey=z9z740VU1fZp331&keytype=ref},
doi = {10.1093/pan/mpt012},
data = {http://hdl.handle.net/1902.1/21603},
html =
{http://www.kai-arzheimer.com/new-multinomial-accuracy-measure-for-polling-bias}
}
```

## The Surveybias Software 1.1

Calculating the accuracy measures is a matter of some algebra. Estimating standard errors is a bit trickier but could be done manually by making use of the relationship between and the multinomial logistic model on the one hand and Stata’s very powerful implementation of the Delta method on the other. But these calculations are **error-prone and become tedious** rather quickly. This is why we created a suite of user written programs (**surveybias**, **surveybiasi**, and **surveybiasseries**). They do all the necessary legwork and return the **estimates of accuracy, complete with standard errors and statistical tests**.

We have just updated our software. The new version 1.1 of surveybias features some **bug fixes**, a better mechanism for automagically dealing with convergence problems, **better documentation**, and a new example data set that compiles information on **152 German pre-election polls** conducted between January and September 2013.

## Examples, Please?

surveybias comes with **example data** from the **French presidential election 2012** and the **German parliamentary election 2013**. From within Stata, type `help surveybias`

, `help surveybiasi`

, and` help surveybiasseries`

to see how you can make use of our software. If I can find the time, I will illustrate the use of surveybias in a mini series of blogs over the next week.

## Updating Surveybias

The new version 1.1 ~~should appear~~ is now on SSC ~~within the next couple of days or so~~, but the truly impatient can get it **now**. In your internet-aware copy of Stata (version 11 or later), type

`net from http://www.kai-arzheimer.com/stata`

`net install surveybias, replace`

Or use SSC: `ssc install surveybias, replace`

Enjoy!