How good/bad were the 2017 German election polls?

In less than 43 hours, polling stations in Germany will close. The idea that the CDU/CSU is building momentum and could catch up with the SPD has been a conservative talking point for the last week or so. And right on cue, Allensbach has published a poll today that sees the SPD’s lead reduced to a single percentage point.

Over on twitter, people have been quite disparaging, pointing out that Allensbach has been perceived as close to the Christian Democrats for decades, uses methods that are now seen as outmoded, and considerably overestimated support for the CDU/CSU in 2017. While I tend to agree, I thought I might as well pass the time by having a look at how well the major pollsters did in 2017 using, you know, actual data.

For multi-party elections, the sum of squared differences between a pre-election poll and the final vote shares is a crude but intuitively plausible measure of accuracy. Thankfully, the good folks over at wahlrecht.de collect headline findings from the major houses going back all the way to the late 1990s. So I plugged the last surveys published immediately before the 2017 election into this shiny table and calculated the differences.

CDU/CSUSPDGreensFDPLeftAfDOthersSum of squared differences
BTW 201732.920.58.910.79.212.64
Insa342189111348.56
Kantar/Emnid3522891011514.56
FGW3621.58108.511515.96
Allensbach3622810.59.510419.56
Forsa362279.59.511520.56
Infratest37207.59.5912521.86
GMS372289910530.56
YouGov3623791010530.76

The numbers are not quite what I expected. Kantar/Emnid and FGW, who have been in the business for ages, are placed second and third with very similar deviations from the result. Fellow household names Allensbach, Forsa, and Infratest form a second, slightly worse performing but very homogeneous cluster. GMS and YouGov, on the other hand, were most (and similarly) off.

The best performer, and this is the surprising part, was INSA, who are, let’s say, are slightly less respected for both political and methodological reasons. According to wahlrecht.de, their final poll in 2017 was published on September 22, just two days before the election, with the data being collected on September 21/22. So it could be that they were simply interviewing closer to the event than the others and picked up some last minute swing away from the Christian Democrats. Or perhaps they were just lucky.

Coming back to Allensbach, the table shows that everyone overestimated the CDU/CSU (herding, anyone?), and that Allensbach was by no means an outlier. So if past (squared) performance can serve as a guide for the present, there is no particular reason to rubbish this latest Allensbach poll.

12 thoughts on “How good/bad were the 2017 German election polls?”

  1. The interesting difference to 2017 is that this time around Allensbach is not herding with the others. They never had the CDU/CSU drop below 25%.
    Of course, if CDU/CSU end up with 23-24%, they will have been as close to truth as everyone else.

    Reply

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.