polls Archives - kai arzheimer

How good/bad were the 2017 German election polls?

Political Science, Politics 12 Responses »

Sep 252021

In less than 43 hours, polling stations in Germany will close. The idea that the CDU/CSU is building momentum and could catch up with the SPD has been a conservative talking point for the last week or so. And right on cue, Allensbach has published a poll today that sees the SPD’s lead reduced to a single percentage point.

Over on twitter, people have been quite disparaging, pointing out that Allensbach has been perceived as close to the Christian Democrats for decades, uses methods that are now seen as outmoded, and considerably overestimated support for the CDU/CSU in 2017. While I tend to agree, I thought I might as well pass the time by having a look at how well the major pollsters did in 2017 using, you know, actual data.

For multi-party elections, the sum of squared differences between a pre-election poll and the final vote shares is a crude but intuitively plausible measure of accuracy. Thankfully, the good folks over at wahlrecht.de collect headline findings from the major houses going back all the way to the late 1990s. So I plugged the last surveys published immediately before the 2017 election into this shiny table and calculated the differences.

	CDU/CSU	SPD	Greens	FDP	Left	AfD	Others	Sum of squared differences
BTW 2017	32.9	20.5	8.9	10.7	9.2	12.6	4
Insa	34	21	8	9	11	13	4	8.56
Kantar/Emnid	35	22	8	9	10	11	5	14.56
FGW	36	21.5	8	10	8.5	11	5	15.96
Allensbach	36	22	8	10.5	9.5	10	4	19.56
Forsa	36	22	7	9.5	9.5	11	5	20.56
Infratest	37	20	7.5	9.5	9	12	5	21.86
GMS	37	22	8	9	9	10	5	30.56
YouGov	36	23	7	9	10	10	5	30.76

The numbers are not quite what I expected. Kantar/Emnid and FGW, who have been in the business for ages, are placed second and third with very similar deviations from the result. Fellow household names Allensbach, Forsa, and Infratest form a second, slightly worse performing but very homogeneous cluster. GMS and YouGov, on the other hand, were most (and similarly) off.

The best performer, and this is the surprising part, was INSA, who are, let’s say, are slightly less respected for both political and methodological reasons. According to wahlrecht.de, their final poll in 2017 was published on September 22, just two days before the election, with the data being collected on September 21/22. So it could be that they were simply interviewing closer to the event than the others and picked up some last minute swing away from the Christian Democrats. Or perhaps they were just lucky.

Coming back to Allensbach, the table shows that everyone overestimated the CDU/CSU (herding, anyone?), and that Allensbach was by no means an outlier. So if past (squared) performance can serve as a guide for the present, there is no particular reason to rubbish this latest Allensbach poll.

Mad Christian Democrats, the AfD’s internal conflicts, useless US polls, and the Wellcome Monitor: four links I liked

Links No Responses »

Jun 212019

Mad Christian Democrats, the AfD's internal conflicts, useless US polls, and the Wellcome Monitor: four links I liked 1

On a very slow news day, two third-tier politicians for the centre-right CDU phantasise over future coalitions with the “moderates/liberals” within the AfD (where would they find them?). Ah yes, they also want to re-unite “the National” and “the Social”, which, by the lego-like greatness of the German language, becomes the “National-Social”.

James F. Downes has an interesting long read on Alternative for Germany’s internal/inter-regional conflicts. It chimes with my own recent article on the development of the AfD and the normalisation of right-wing populism in Germany. Somewhat surprisingly, these contradictions are still not harming the party.

Sorry to disappoint: Brendan Nyhan explains why the current polls showing that Trump is trailing various Democrats are useless.

People in rich & healthy countries stop believing in vaccinations. In the so-called developing world, vaccinations are still trusted. Find these and other fascinating findings in the latest Wellcome Monitor

State of the German polls: It’s the final, final, final surveys

Data and Methods, Politics 7 Responses »

Sep 232017

State of the German polls: It's the final, final, final surveys 2

So: three more surveys. No kidding

Finality finally got a bit more final: just to annoy me (now here is a narcissist), three further surveys were published today (already yesterday in Germany). One of them is only new-ish: Emnid was in the field from September 14-21, so I take their data as a snapshot of the world as it was on September 17 (last Sunday). Forsa interviewed from September 18 to September 21, resulting in a mid-point of September 19 (Tuesday), while Insa did all their fieldwork on Thursday/Friday. But does this new information in any way alter the expectations? The short answer is:

It makes no difference

Here is a comparison of the overall estimates. They are virtually identical. The CDU/CSU is up by one point, but that is due to different rounding. The probability of the AfD coming third is now up at 99.6 per cent (from 96 per cent) and the point estimate for their lead over the Left is up, too, but again, that is due to rounding – the credible interval is much the same.

	yesterday		today
	Median	95 HDI	Median	95 HDI
CDU/CSU	35	[34-37]	36	[34-37]
CDU/CSU lead	14	[12-16]	14	[12-16]
SPD	22	[21-23]	22	[21-23]
FDP	9	[9-10]	9	[9-10]
Greens	8	[7-9]	8	[7-9]
Left	10	[9-10]	10	[9-10]
AfD	11	[10-12]	11	[10-12]
AfD lead	1 \| [0-2.4] \| 2 \| [0.4-2.7]

No more graphs, because they would look the same. Coalition options do not change. If the polls are right on average and the poll aggregation works, Grand Coalition and Jamaica are the mathematically possibilities. To be honest, in six per cent of my simulations a coalition of the Christian Democrats and the AfD would achieve a majority, but that is inconceivable.
That’s it. Move on. Nothing to see here until Sunday evening, which happens to be Sunday noon on my personal timescale.

State of the German polls: It’s the final surveys

Data and Methods, Politics 1 Response »

Sep 222017

I just suppressed the urge to insert the word ’countdown’ into the headline. See what I’m doing here? We have four more polls by Allensbach and Forsa (published on Tuesday), and by FGW and GMS (published on Thursday), and presumably, these are the last that we are going to see before election day. Do they change the story?

First, let’s note that FGW has the very latest data: they interviewed on Wednesday and Thursday and published the results immediately. A very short fieldwork period raises issues of representativeness, but they have been in the business for about 40 years now, so let us assume they know what they are doing, shall we? Second, unlike most pollsters, FGW always publishes both raw (but presumably weighted) data (what they call the political mood) and estimates that take into account party identification and other long-term factors (their ’projection’). So far, I have always used the former, but we have reached the point where the forecast becomes the nowcast, and so the only thing we get this time is their projection, which I treat like if they were raw data, using last week numbers of undecided and non-voters (both not very realistic, I suppose).

GSM was in the field from Thursday last week until Wednesday, but because I peg every poll to the mid-point of their fieldwork, their data are three days older than FGW’s for modelling purposes. Things get bit confusing then: Forsa were in the field from September 11 to September 15, and Allensbach even from September 6 to September 14, but then sat on their data. So their findings came out on Tuesday but are less recent than the Insa poll I talked about last time round. In other words: By putting this information in the model, I’m adjusting our estimate of where public opinion was a week ago, which then feeds into my guess where it is right now (or rather where it was two days ago). It’s a good thing that this is almost over.

Countdown

Ok, I succumbed. Couldn’t resist. etc.

The CDU/CSU maintain their lead

Support for the Christian Democrats has further declined. The last estimate is 35 per cent [34-37], which would be six points less than in 2013. But the Social Democrats are down, too. The estimate for their current level of support is 22 per cent [21-23], so the CDU/CSU’s lead is still 14 points [12-16].

The FDP bounce remains elusive, and the Greens are weak

If there is a last-minute rush towards the FDP, it’s not reflected in the polls. But the party (not currently in parliament) is doing well, and much better than a few months ago when it was far from certain that they would return to federal politics. Estimated support for them is 9 per cent[9-10], which puts them ever so slightly ahead of the Greens (8 per cent [7-9].

Is the AfD finally pulling ahead of the Left?

After going to great lengths to explain why the race for 3rd place is irrelevant and how the Left is better positioned to win it anyway, the AfD is finally pulling (or rather inching) ahead. The final estimate for their current support is 11 per cent [10-12] (which would be a far cry from the levels of support they enjoyed in 2016), while the Left is put at 10 per cent [9-10]. With the four new polls factored in, the chance of the AfD coming third is now a whopping 96 per cent. The size of their likely lead is a single point [0-2.4].

Overall estimates and coalitions

I (and the pollsters) have been embarrassingly wrong before, but it seems almost impossible that we are not heading for a six-party parliament. It’s also quite clear that there will be no SPD-lead coalition government (unless the SPD could somehow persuade the Greens, the FDP, and the Left to work with them, and even that might not be sufficient). Unless there is a last-minute bounce for the FDP or the Christian Democrats that does not affect the other party (i.e. a shift from the radical to the moderate right), there will be no centre-right government

The two most likely outcomes remain a continuation of the Grand Coalition (not necessarily in the best interest of the SPD), or a Jamaica coalition (if the FDP and the CSU and the Greens can work together). Interesting times ahead.

State of the German polls: No news is no news

Data and Methods, Politics 9 Responses »

Sep 192017

German Elections: Three more polls

We Anoraks are all getting a little jittery here. It’s 134 hours until closing time and there will be only a small handful of polls coming in in the next couple of days, so is there anything new that may be divined from the latest crop, published today (Insa), on Saturday (Emnid), and on Friday (FGW)? Not really. First, the Emnid poll is not new, but new-ish: fieldwork began on September 7, almost a week before Infratest’s (alleged) shock poll. Second, the three polls mostly agree:

	Emnid	FGW	INSA
CDU/CSU	36	36	36
SPD	22	23	22
GREENS	8	8	7
FDP	9	10	9
LEFT	10	9	11
AfD	11	10	11

Third, they are broadly in line with the last (Friday) set of estimates. Of course, that does not mean that the pollsters have it right. It just means that public opinion as measured by the various survey houses seems to be rather stable at the moment.

The Christian Democrats are still leading

Support for the Christian Democrats has been flagging recently, but they still have a solid lead of about 14 points over the Social Democrats. The credible interval for the gap is 13-16 per cent. The current estimate for the Christian Democrats is 37 per cent [36-38], which would make them the strongest party by far but would also imply a substantial loss compared to their result in the 2013 election (41.5%). The estimate for the SPD is 23 per cent [21-24], which is virtually identical to their worst ever result (in 2009).

The FDP and the Greens seem to be safely in

Speaking of virtual, it seems virtually impossible that these two minor parties will not clear the electoral hurdle. Then again, look at what happened in 2013. Right now, the FDP is ever so slightly ahead of the Greens, but the enormous attention they are currently getting from the chattering classes is not (yet?) reflected in the polls. Either way, their likely return from the electoral dead would be a significant event in German politics.

The Left and the AfD remain tied

Even the Wallstreet Journal is very excited about the idea of the AfD becoming Germany’s “third” party (technically, the CSU is competing for that title, too, but that is a different story). According to the model, however, the chances of the AfD ending up in this position are just 28%. Although predictions of support are almost identical – 9.5% [8.7-10.3] vs 9.7% [8.9-10.5] – the model gives the Left a much better chance (53%) of coming out tops. This is neatly illustrated here:

However, the relevant information (in my view) is still this: we are heading for a six/seven party parliament, with four minor parties of almost equal strength

Coalitions …

After factoring in the three latest polls, the options remain essentially the same: In all simulations there is a majority for both a Grand Coalition and a Jamaica arrangement. There is also tiny (0.5%) chance of a centre-right (CDU/CSU + FDP) coalition. If the polls are correct, nothing else will work. As I said before: Move on. Not much to see here.

State of the German polls: Is the SPD doomed and the AfD headed for third place?

Politics, Data and Methods 15 Responses »

Sep 152017

It’s just a single poll

Once more, repeat after me: It’s just a single poll. It’s also the time for horse-race journalism (and for horse-race blogging). In this specific case, the single poll is the most recent instalment of the “Deutschlandtrend”, a survey-series that Infratest-dimap runs for public broadcasting giant ARD. From the results (SPD: 20, AfD: 12), Focus Online has created a lovely headline: SPD in freefall, AfD at highest level of support in seven months. But is there really a story?

Horse-race blogging

Now for the horse-race blogging. Since my last blog (day before yesterday), three new polls have been published. Why bother to start the big and mysterious poll-pooling machine again? Because I can, because in a week or so, there will be no new polls, and because I want to see if there is anything to the Focus story.

First, a closer look at the Infratest-dimap poll, which is clearly the most recent piece of information: field time was only the last two days (September 12-13), and it was published immediately. The other two “new” polls are not really that new. They were in the field from September 8-11 (Insa) and September 4-8 (Forsa) and put the SPD at about 23 per cent and the AfD between 9 and 11 per cent for these slightly earlier time spans. Does that suggest some dramatic movement during the last couple of days? Not really. Infratest-dimap tends to produce somewhat low-ish estimates for the SPD, and rather high estimates for the AfD. The (estimated) house effects are -0.7 and +1.7 points, respectively. The house effects are not calibrated in any way, so Infratest-dimap’s estimates may be perfectly correct, but across all the polls in the model, their estimates for these two parties tend to be below/above average. This is neatly illustrated in the graph:

All Infratest-dimap polls (the hollow circles) put the AfD well above the model-based credible interval, and this one (the rightmost circle) is particularly far away from the envelope. The current credible interval for the AfD is 8.6-10.3%. The AfD’s mini-upward trend may be real, but this poll is probably exaggerating the development.

Infratest-dimap may also underestimate support for the SPD. The model currently puts the SPD between 21.4% and 23.6%. The Infratest-dimap poll (rightmost filled red circle) is well below the credible interval. Things don’t look great, but it’s not “freefall”. The credible interval for the gap between the the SPD and the AfD is 11.6-14.6 points, so the 8 point gap reported by Focus on the basis of a single poll looks like a bit of an over-dramatisation. The AfD is not (yet) catching up with the Social Democrats.

So what?

Will the AfD be Germany’s third party? In the model-based simulations, their chances have gone up from 18% to 39%, but that is still far from certain.In actual fact, according to the model, the Left has a better chance (50%) to become the largest of the minor parties. But that would be a far less dramatic story. And realistically, this is all by the by: the four minor parties enjoy virtually identical levels of support.

Coalition options are the same as they were three days ago. So what is the bottom line? This last poll (and the other two) make good headlines, but in terms of likely politically relevant outcomes, the situation has not changed at all.

State of the German polls: Move on, (almost) nothing to see here

Politics, Data and Methods 7 Responses »

Sep 122017

With less than two weeks until the election, we now have 153 surveys from seven different companies to pore over. The bulk of these (104) were produced by Emnid, Forsa and Insa. GMS and Allensbach have delivered only a handful of polls (seven and nine, respectively), while FGW (15) and Dimap (18) occupy the middle ground. Although this is the so-called “hot phase” of the campaign, with TV debates, tours of the country and whatnot, there is still very little movement in the (averaged) results. Unless my model is filtering out too much noise, or the polls are off, which are two entirely plausible and not mutually exclusive ideas.

The AfD and the Left are perhaps gaining some ground

Estimates for both parties have shown an upward trend for the last couple of weeks, but the gains are very moderate (a point apiece or so), and given the credible intervals, the movement is not necessarily real. But the AfD is probably doing a bit better than they did in late June, which marked their low point during the campaign.

Support for the Liberals and the Greens is mostly stable

The respective upward trends for the other two minor parties are even less pronounced. More importantly, both parties seem to have stabilised well above the electoral threshold

The tiniest of declines for the major parties.

Conversely, support for the two major parties may have fallen a bit. But the credible interval for the Christian Democrats is particularly wide because there is a lot of variation in their results, whereas numbers for the SPD are all very close to the credible envelope. Spare a thought for them: It’s abundantly clear that even with their recent relative weakness, the CDU/CSU are much stronger than the Social Democrats.

Overall estimates and possible coalitions

With about a fortnight to go (polls are published with a delay, and the model assigns each poll a notional date in the middle of the actual field phase) and many postal votes already cast, the overall picture looks very much like it has for weeks now. Support for all four minor parties is virtually identical and above the electoral threshold. The SPD is hovering somewhere between 20 and 25 per cent, while the Christian Democrats are located somewhere in the high thirties.

The Guardian may daydream about a black-green coalition, but that is not very plausible at the moment: not in a single one of 60,000 simulated outcomes would such a coalition achieve a majority. Obviously, a red-green coalition is even less probable, and a red-red-green majority is out of the question, too.

Nine days ago, there was at least a chance (23%) of a traditional centre-right majority, but with the (moderate) decline in support for the Christian Democrats, this looks highly unlikely (0.1%) now. However, both a “Jamaica” coalition and a Grand coalition are feasible in all simulations. So once more, a fourth term for Merkel seems to be inevitable.

State of the German polls: The initials of Germany’s next chancellor would seem to be A & M

Politics, Data and Methods 9 Responses »

Sep 022017

Eight months of polling

While I was doing other stuff, elsewhere, the German polling industry has been busy. Over the last eight months, the big seven have published results from 144 surveys with a total of 266,715 respondents. With just three weeks to go until election day (and postal voting well underway, what can they tell us?

They AfD is good at spinning

Much in line with the tenor of the international press, a British journalist asked me the other day how the AfD had managed to “bounce back”. Well, they have not. The height of their popularity was arguably in 2015/16, when they were solidly in double-digit territory for a while. Since the beginning of the long campaign in January (when the AfD did better than now), they have been stuck between seven and ten per cent in the polls. This pattern still holds. The AfD gets 10/11 per cent in two recent polls, but these were conducted by companies that tend to produce rather high estimates for the party’s support. In other equally recent polls, companies that tend to give lower estimates for the AfD put them at nine / eight per cent, respectively. Unsurprisingly, the credible interval for current AfD support ranges from 7.8 to 9.3 per cent.

Another object of rather intense media focus has been the question whether the AfD will be Germany’s third-strongest party on September 24. The aggregation model is not sure: In 10,645 of 60,000 simulated outcomes of the election, the AfD is the strongest of the minor parties. That’s 18 per cent of all runs, which roughly translates to “rather not”.

Either way, the question is quite irrelevant. The real issue here is that all minor parties enjoy very similar levels of support. If this support translates into real votes, there will be four minor parties in parliament, and coalition building is going to be a difficult, but not impossible (see below).

The Christian Democrats will be the strongest party (bloc) by a fair margin

Long gone are the days of the “Schulz Effect”, this dreamy moment early in the campaign when support for the SPD and the Christian Democrats became indistinguishable. For weeks on end, the CDU/CSU have been roughly 15 percentage points ahead of the SPD. One must admire the dedication of the SPD’s campaigners, but closing that gap looks like a very unlikely feat.

A look at possible coalitions

So far, the field has changed very little over the last weeks:

While tactical voting can mess with support for the FDP (that’s the way I burnt my fingers four years ago), not a single simulation out of 60,000 suggests that the FDP will remain below the electoral threshold. All four minor parties are currently well above the electoral threshold, and their respective levels of support are indistinguishable. Take that, media people.

In terms of possible coalitions, that means (amongst other things) that right now, there is no chance for a leftist (red-red-green) government: the combined vote share for the three left parties is in the range of 39-41 per cent. This implies that there is no majority for an SPD/Green coalition, too. There is also no majority for a “traffic light” (SPD/Green/FDP) government. In sum, on current polling the probability of an SPD-led coalition (and hence the probability of a Schulz chancellorship) is nil.

But there is a not-too-shabby chance for a traditional centre-right coalition. Thanks to the strong support for the Christian Democrats, the CDU/CSU and the FDP have a (narrow) majority in 23 per cent of the simulations. A “Jamaica” (CDU/CSU+FDP+Greens) coalition would have a solid majority in all of the simulations. And of course, there is always the prospect of yet another Not-so-Grand coalition.

So it looks that I was right to sell in May and go away: If the polls reflect the reality of German politics, and if that reality remains reasonable stable for another three weeks, there are 2.23 viable coalitions that would be led by Angela Merkel, and not a single one that would be headed by Martin Schulz.