Mar 232017

Eight new German polls

Over the last fortnight, eight new polls have been published: two by Insa, two by Forsa, and one apiece by Dimap, Emnid, GMS, and FGW. For GMS, it is only the second poll conducted since the beginning of this year (a third one was published early in January, with fieldwork partly carried out in late December). As usual, there is quite a bit of variation in the data (hence the pooling), although there is no such shocker as the (unweighted) FGW poll done in mid-February which put the SPD at 42 per cent, twice of what they had been polling in early January.

The SPD and the Christian Democrats are in a dead heat, but …?

Probably no “but” so far. Back in late January, the SPD’s fortunes began to rise rapidly thanks to their new candidate Martin Schulz (aka St Martin), and by early February, support for the two major parties had become statistically indistinguishable. For the last ten days or so, the model has put the CDU about a percentage point ahead of the SPD, but that does not mean a thing: the credible intervals are still more or less identical. The SPD’s rise and rise in the polls has stopped in March, but the Schulz effect is still very much a thing.


The Left and the AfD

Since January, the AfD has lost a bit of steam (i.e. about 2 points) in the polls, but it is still the strongest opposition party at just under 10 per cent. The model-based line in the graph suggests that over the last couple of weeks, the party has recovered a wee bit, but that movement is negligible (have a look at the scale), just a teensy-weensy wiggle well within a wide credible band. Similarly, the Left is basically where it was two (and four, and six) weeks ago at 7.5 per cent.


The Greens and the FDP

The Greens had a bad start into the campaign. Just when they had selected two Spitzenkandidaten from the more conservative wing of the party (signalling that they might enter a CDU/CSU/Green coalition), the SPD pulled a Schulz on them by choosing a leader who would attract leftist voters and would at least ponder the prospects of an SPD/Left/Green coalition. The Greens lost a couple of points in January and early February but have been perfectly stable since. Conversely, the FDP may or may not have won a point since February. Like with the AfD, the wiggling is not impressive, given the width of the credible intervals. But importantly, this interval has not touched the electoral threshold of five per cent during the last three months, though the threshold is never that far away.



The polls are noisy as ever, but the model, which tries to account for house effects and random errors, suggests that the noise is just that, and that so far there was no real movement in political support in March. This picture is at least plausible: Apart from the AfD, all parties have selected their respective Spitzenkandidaten by now, it’s still six months to go,  and there have been no major domestic events.


At current levels of support, the Bundestag would have two (relatively) major parties that could continue the Grand coalition, although it’s not quite clear which one would hold the chancellorship, and four almost identically-sized minor parties, commanding between six and ten per cent of the vote. In such a relatively fragmented parliament, the chances (in a purely statistical sense) for forming a SPD/Left/Green coalition are slim: Only in 15 per cent of all draws from the (simulated) joint distribution of support is there a sufficient red-red-green majority. At 12 per cent, the chances of a numerical majority for a “Jamaica” coalition (Christian Democrats, FDP, Greens) are even lower. In none of the the simulations is either a SPD/Greens or Christian Democrats/FDP coalition feasible. It would be Grand Coalition, or new elections, or minority cabinet rule (a first). But remember the mantra: It’s six months to go, we’re very much talking about political mood, not firm intentions here, and this is just a model that tries to average over many different noisy and potentially biased polls.

Mar 102017

With just under seven months to go until the German federal election, I have recently begun once more to pool the pre-election polls from seven major survey firms. Since January, when the date for the election was set and the Spitzenkandidaten were selected, results from 35 polls with a median sample size of about 1900 have been published: nine apiece by Emnid and Forsa, five by Dimap, five by Insa, four by FGW, two by Allensbach, and a single one by GMS.

Easily the most exciting event in the (long) campaign so far has been the #Schulzzug: the mostly unexpected leak/announcement on January 24 that Sigmar Gabriel would be replaced as party leader and (presumptive) candidate by Martin Schulz, the former president of the European Parliament. Support for the SPD in the polls had hovered at historically low levels of just over 20 per cent for months, but the Schulz candidacy re-energised party members and resulted in lots of (mostly positive) media coverage so far. Subsequently, support for the party leaped up in the polls, even overtaking support for the Christian Democrats in some of them.

But most movement in the polls is noise, and so we would like to know if the Schulz bounce is real. The data basically say: yes.


The figure shows that support for the SPD begins to rise a couple of days before Schulz’s candidacy was announced, but this is probably an artefact. The model assumes that true support normally changes very little from one day to the next, but these are unusual circumstances, and so the ascent was probably steeper than the graph suggests. At any rate, the estimated level of support for the SPD in February was somewhere between 30 and 35 per cent, whereas it was between 20 and 24 per cent early in January. The model’s priors may play a role here (though they should be quickly overwhelmed by the data), but it is obvious that there was a gap of at least 10 percentage points between the two major parties in January that has essentially closed now. Support for the CDU and the SPD is virtually indistinguishable, and the Christian Democrats are rightfully worried.

What this means for the election is a different question. Estimated levels of support for both parties have been essentially constant for the last four weeks or so. The SPD has unexpectedly closed the gap, but it has stopped gaining. The Christian Democrats are not doing much worse than at the same point in the cycle four years ago. And once voters learn more about Schulz (who is a known unknown in Germany), the Schulz effect may wear off.

Mar 102017

It’s that time of the electoral cycle again: With just under seven months to go until the federal election in September, I feel the urge to pool the German pre-election polls. I’ve burnt my fingers four years ago when I was pretty (though not 100%) sure that the FDP would clear the five per cent threshold (they failed for the first time in more than six decades), but hey – what better motivation to try it again?

Why bother with poll-pooling?

Lately (see Trump, Brexit), pre-election polls have been getting a bad rap. But there is good evidence that by and large, polls have not become worse in recent years. Polls, especially when taken long before the election, should not be understood as predictions, because people will change their mind on how to vote, or will not yet have made up their mind – in many cases, wild fluctuations in the polls will eventually lead to an equilibrium that could have been predicted many months in advance. Rather, polls reflect the current state of public opinion, which will be affected by campaign and real-world effects.

Per se, there is nothing wrong with wanting to track this development. The problem of horse race journalism/politics is largely a problem of over-interpreting the result of a single poll. A survey of 1000 likely voters that measures support for a single party at 40% would have a sampling error of +/- 3 percentage points if it was based on a simple random probability sample. In reality, polling firms rely on complicated multi-stage sampling frames, which will result in even larger sampling errors. Then there is systematic non-response: some groups a more difficult to contact than others. Polling firms therefore apply weighs, which may hopefully reduce the resulting bias but will further increase standard errors. And then there are house effects: Some quirk in their sampling frame or weighing scheme may cause polling firm A to consistently overreport support for party Z. So in general, if support for a party or candidate rises or drops by some three or four percentage points, this may create a flurry of comments and excitement. But more often than not, true support in the public may be absolutely stable or even move in the opposite direction.

Creating a poll of polls can alleviate these problems somewhat. By pooling information from many adjacent polls, more accurate estimates are possible. Moreover, house effects may cancel each other out. However, a poll of polls will not help with systematic bias that stems from social desirability. If voters are generally reluctant to voice support for a party that is perceived as extremist or otherwise unpopular, that will affect all polls in much the same way.

Moreover, poll-pooling raises a number of questions to which there are no obvious answers: How long should polls be retained in the pool over which one wants to average? How can we deal with the fact that there are sometimes long spells with no new polls, whereas at other times, several polls are published within a day or two? How do we factor in that a change in polling that is reflected across multiple companies is more likely to reflect a true shift in allegiances?

The method: Bayesian poll-pooling

Bayesian poll-pooling provides a principled solution to these (and other) issues. It was pioneered by Simon Jackman in his 2006 article on “Pooling the polls over an election campaign”. In Jackman’s model, true support for any given party P is latent and follows random walk: support for P today is identical to what it was yesterday, plus (or minus) some tiny random shock. The true level of support is only ever observed on election day, but polls provide a glimpse into the current state of affairs. Unfortunately, that glimpse is biased by house effects, but if one is willing to assume that house effects average out across pollsters, these can be estimated and subsequently factored into the estimates for the true state of support for any given day.

The Bayesian paradigma is particularly attractive here, because it is flexible and because it easily incorporates the idea that we use polls to continuously update our prior beliefs about the state of the political play. It’s also easy to derive other quantities of interest from the distribution of the main estimates, such as the probability that there is currently enough support for the FDP to enter parliament, and, conditional on this event, that a centre-right coalition would beat a leftist alliance.

In my previous misguided attempt to pool the German polls, I departed from Jackman’s model in two ways. First, I added a “drift” parameter to the random walk to account for any long term trends in party support. That was not such a stupid idea as such (I think), but it made the model to inflexible to pick up that voters were ditching the FDP in the last two weeks before the election (presumably CDU supporters who had nursed the idea of a strategic vote for the FDP). Secondly, whereas Jackman’s model has a normal distribution for each party, I fiddled with a multinomial distribution, because Germany has a multi-party system and because vote share must sum up to unity.

The idea of moving to a Dirichlet distribution crossed my mind, but I lacked the mathematical firepower/Bugs prowess to actually specify such a model. Thankfully, I came across this blog, whose author has just done what I had (vaguely) in mind. By the way, it also provides a much better overview of the idea of Bayesian poll aggregation. My own model is basically his latent primary voting intention model (minus the discontinuity).

The one thing I’m not 100% sure about is the “tightness” factor. Like Jackman (and everyone else), the author assumes that most movement in the polls is noise, and that true day-to-day changes are almost infinitesimally small. This is reflected in the tightness factor, which he arbitrarily sets to 50000 after looking at this data. Smaller numbers make for more wiggly lines and wider confidence intervals, because more of the variability in the data is attributed to true change. Unfortunately, this number does not translate to a readily interpretable quantity of interest (say a true day-to-day change of 0.5 per cent).

After playing with smaller and even larger values, I came up with a cunning plan and made “tightness” a parameter in the model. For the first six weeks of polling data, the estimate for tightness is about an order of magnitude lower in a range between 3500 and 10000. Whether it is a good idea to ask my poor little stock of data for yet another parameter estimate is anyone’s guess, and I will have to watch how this estimate changes, and whether I’m better of to fix it again.

The data

Data come from seven major polling companies: Allensbach, Emnid, Forsa, FGW, GMS, Infratest Dimap, and INSA. The surveys are commissioned by various major newspapers, magazines, and TV channels. As far as I know, Allensbach is the only company that does face-to-face interviews, and INSA is the only company that relies on an internet access panel. Everyone else is doing telephone interviews. The headline margins are compiled and republished by the incredibly useful website:, which I scrape with a little help from the rvest package.

Replication (Updated)