Sep 202013
 

We’re now officially in the final throes of a very bad case of election fever. Or at least the wonks are. The latest and probably last crop of major polls is in. Moreover, last Sunday brought us the Bavarian Land (i.e. State) election, which was obviously seen not as an election in its own right but rather as a Bellwether for the big one. This is nonsense.

What’s the Matter with Bavaria?

Last Sunday, the Bavarian Christian Democrats received a cool 47.7 per cent of the valid votes. That is enough to give them a majority of the seats in parliament. The Liberals, with whom they governed from 2008, are no longer required and, to add insult to actually injury, actually no longer represented in parliament, as they got a not-so-cool 3.3 per cent.

Just a week before the national election, journalists, politicians and pundits are desperate for every shred of insight into what’s to come, and so the idea of a Grand (CDU/SPD) Coalition gained a lot of currency very quickly. Predictably, the FDP launched a new plea for “loan votes” from CDU supporters. Slightly less predictably, the CDU declared that they had no votes to spare, and that their supporters should vote straight.

But the Bavarian election is not very informative. Bavarians made a judgement on a different government, voted for different parliament, under different rules and facing a different party system. Merkel’s CDU does not even exist in Bavaria, which has its own Christian Democratic party, the Bavaria-centric CSU. Both parties do not even form an electoral alliance for the Bundestag election. They have merely chosen to co-operate in parliament for the last 17 times (and will do so again). In a very real sense, the current coalition consists of three parties, and the CSU has often been the most quarrelsome of them.

The CSU is the party of Bavarian nationalism, and single-party rule by the CSU is more or less the norm. The CSU has been part of every post-war government with the exception of the 1954-57 gang-of-four coalition (an abomination), and has governed alone from 1947-50 and then again from 1966-2008. A liberal presence in the Bavarian parliament, on the other hand, is not the norm. The party was out from 1982 to 1990, and then from 1994 to 2008. The Greens have been doing better than the Liberals in every election since 1982, and that is saying something in Bavaria. And yet, there is a whiff of change in the air.

The Final (?) Polls Are Coming In

In Germany, the two major public TV channels are the main sponsors for frequent large-scale polling. In the past, they stopped publishing results ten days before the election, but this is not a legal requirement: The law only bans the publication of findings from exit polls before the polling stations close on Sunday at six. Publishing results based on interviews conducted just before voters entered the polling station would be technically legal.

This year, broadcaster ZDF broke with tradition and published their final poll yesterday. Moreover, findings from three other major polls were published today, bringing the total of new polls since my last post to five. Two of them are straddling the allegedly ominous Bavarian election, while three of them were conducted this week. What do they tell us?

The Final Estimates

file:///home/kai/Work/btw-aggregator/figures/majorparties-week-38.png

This close to the election, the model is becoming a lot more confident, i.e. credibility intervals for Sunday are now pretty narrow. The upward trend for the SPD (Fingergate notwithstanding) and the corresponding downward trend for the CDU that became visible last week are now more pronounced, but this should be seen in proportion: Predictions for both parties are within a point or two where they were seven weeks ago.

Things look a bit different for the smaller parties. The Greens are now projected to garner just under 10 per cent. In early August, the model was giving them about 13 per cent (with a very wide credibility interval). Looking at the curves, one can be pretty certain that there has been real movement: They have lost about one quarter of the support they had in mid-August. Conversely, the lot of the Left has improved. They are now projected to end up with about eight per cent, up from roughly six per cent that were predicted in August.

The most interesting case is of course the FDP. Back in August, the model thought they would scrape by with about five point something per cent. The probability that they would enter parliament at all was estimated at roughly 70 per cent. Seven weeks on, the estimate for their vote share has hardly changed, but the credibility interval is much more narrow. The model is now 95 per cent certain that they will garner at least five per cent. Reports of their death seem slightly exaggerated. Or my model could be totally wrong.

file:///home/kai/Work/btw-aggregator/figures/minorparties-week-38.png

The probability of a majority for the current coalition is now estimated at 82 per cent. Whether the CDU likes it or not, voters will not ask for permission to vote tactically. If they do, the probability goes up to 85 per cent. These figures are down by seven/nine points from last week, reflecting the (modest) decline of the CDU and the slightly better performance of the SPD and the Left. But the probability of a Red-Green government is still zero per cent.

The probability of a (politically infeasible) Red-Red-Green majority is 15/18 per cent (lower if CDU supporters help the FDP). In line with previous post, I assume that there is a one in ten chance that the SPD might go back on their word, and a nine in ten chance that they would form a Grand Coalition with the CDU. This puts the Merkel-O-Meter (TM) at 98 per cent (irrespective of tactical voting).

Finally, the AfD is still making an awful lot of noise. While their party leader has recently lost a libel suit against a leading pollster, their supporters in the social media tend to say things that would make you want to avoid them in dark alleys. So far, one single poll has put them at exactly five per cent, and that was based on an online access panel. Everyone else has them at four per cent or less. If 40 per cent of the current support for “other” parties is for them, their probability of getting into parliament is zero. If their share of the “other” vote was more like 60 per cent, they would be almost certainly in.

The Outlook

While the pundits’ and politicians seem to brace themselves for a Grand Coalition and the death bell is ringing for the FDP, the model says that a new mandate for the current coalition is still by far the most likely outcome. That feels distinctly odd.

German politicians have a mantra that goes “Polls are not actually election results”, and pollster routinely claim that they are not making predictions. But colleague Bernhard Weßels over at the WZB has an excellent blog about the fit between polls and election results (in German). Usually they are very close. But there are some embarrassing exceptions. So stay tuned for post-election analysis, subterfuge and model tweaking.

Sep 192013
 
Thank you for more than 28,000 clicks on that photo!  Dome of the Reichstag building - La cúpula del Reichstag - Reichstagskuppel Berlin
alles-schlumpf / Foter / CC BY-NC-SA

Why Would I Want to Pool the Polls?

Pre-election polls are noisy for a number of reasons. First, there is sampling error. For n=1000, the confidence interval for a party whose true support is 40 per cent ranges from 37 to 43 per cent, which is more than most people would think. And this assuming simple random sampling. For multi-stage sampling, you could end up with one to two extra points at each end. Then there are house effects: Pollsters dress up their raw figures and different ways, use different sampling frames and slightly different modes and questions. And finally, political events and media coverage on the day of the poll will have effects, especially early on when many voters are undecided.

Combining results from different polls is one obvious strategy to deal with these problems: The combined sample size is bigger, and there is hope that the various sources of bias might offset each other. Hopefully.

Where Do the Data Come From?

The very useful site wahlrecht.de publishes margins from seven large German pollsters. Excluding INSA (they use an online-access panel), I check this site regularly and generate a data set from it that you can download here. To companies post (relatively) raw data, which I find preferable. What the others do to their figures, we cannot know.

How Does It Work?

The most straightforward idea in poll-pooling is calculating a moving (and possibly weighted). A more principled approach is model-based. My model borrows heavily from Simon Jackman’s (2005) paper and from Chris Hanretty’s application of a similar model to Italy, but differs in some respects. First, I treat the polls as draws from a multinomial distribution to account for Germany’s moderate multi-partyism. The parameters of this distribution depend on the relative strength of latent support for each party. Modelling the results as multinomial implies the constraint that the estimated shares must some to unity, which is useful. Second, like Jackman and Hanretty, I assume that latent support for each party follows a random walk (today’s support is yesterdays support plus a random quantity), but I allow for a drift: a linear trend in latent support over the course of the campaign. Third, I assign each poll to a week, because there are relatively few polls, and field-times are relatively long. Put differently, I assume that public opinion moves from week to week (but not from day to day).

The model estimates latent party support since January 2013 and makes predictions for the outcome of the election. The code (R & Bugs) is here.

Does It Work?

Honestly, I have no idea. This is work in progress, so take the findings with a pinch of salt.

What Could Possibly Go Wrong?

Everything? Anything? The most obviously dubious assumption of the model is that the polls are unbiased on average. Latent linear trends are a close second.

Sep 152013
 

State of Play: From Momentum to Peer’s Finger

The German media have been particularly excitable this week. They kicked off with floating the idea that there was momentum in the polls (“SPD up by almost two points!”) and concluded with covering the infamous magazine cover portraying Peer Steinbrück flaunting his finger. Moreover, the idea that the eurosceptic AfD might enter parliament has gained some traction. Of course, there is zero new evidence, but some days ago a leading pollster said that he could not rule out a result north of five per cent (the threshold).

Given that they have been polling between three and four per cent and considering the margin of error, that is a very sensible statement. By this morning, it had morphed into “Pollsters Expect that AfD will force Grand Coalition”. My totally scientifically objective hybrid human-computer qualitative analysis of the press (i.e. me browsing the internet) shows that the more pro-business papers are pushing that story – presumably because the AfD’s leader, a professor of macro economics, has the ear of a few editors.

The party does have, however, an unusually strong social media presence. If that is indicative of anything is anyone’s guess.

The Polls

The most recent iteration includes three new polls from week 36 (the week after the debate) as well as two from this week, and they are as noisy as ever. Ratings for the CDU are particularly variable (between 39 and 44 per cent), which is to be expected since the sampling error is bigger for parties that are close to 50 per cent. But according to model, this is mostly noise, and the party is still solidly somewhere in the area of 40 per cent (or slightly more).

majorparties-week-37.png

Estimates/Predictions for the Major German Parties (2013). Click for larger image

The SPD’s support has indeed risen a bit over the last couple of weeks (all polls before finger-gate). But in the great scheme of things, that does not seem to matter much: The party is still stuck well under 30 per cent. Looking on the bright side, they will probably do better than the abysmal 23 per cent they got in 2009.

The decline of the Greens seems to be levelling out. Their recent drop in popularity not withstanding, the model predicts that the party will come third at about ten percent.

Support for the Left is basically constant and stable, while the FDP continues to inch upward. According to the model, Merkel’s preferred coalition partner will almost certainly make it into parliament.

minorparties-week-37.png

Estimates/Predictions for the Smaller German Parties (2013). Click for larger image

The estimated probability of a new mandate for the current coalition is now 89 per cent. If tactical voting is taken into account, that number goes up to 94 per cent. The probability of a red-green majority is constant at virtually zero per cent. The SPD has repeatedly ruled out that they would accept any arrangement involving the Left. In line with previous posts, I assume that there is a 10 per cent chance that they nonetheless consider the unspeakable. Plugging this into the calculation, the probability of Merkel winning a third term is estimated at 98.9/99.4 per cent (with/without tactical voting for the FDP).

The Outlook

Over the next six days, the CDU’s main problem will be the complacency of their voters: With the race virtually run, they might simply be too lazy to turn out to vote. The SPD, on the other hand, seems to be making very small noises that imply that they might be interested in entering another Not-So-Grand Coalition. And the FDP will make a desperate appeal to their supporters (whom?) while trying to convince at least some CDU voters that yellow-blue is the new black.

Much will depend on the outcome of today’s state election in Bavaria: While Bavaria is basically representative for, well, Bavaria, politicians, journalists and voters will inevitably take it as a bellwether for the big one. Let’s wait and see then, shall we?

Sep 142013
 

I finally got around to clean/update the code for my model that pools the 2013 German pre-election polls, estimates latent party support and comes up with the most marvellous predictions (TM). You may download it (complete with the most recent version of the polling data) from my dataverse. If you spot a bug or some other problem, please drop me a line.

I will be back soon with the latest polls.

Sep 012013
 

The Polls

majorparties-week-35

Support for the Major German Parties, Estimates and Predictions (Week 35, 2013)

Exactly three weeks before the 18th Bundestag election, it’s time for another look at the polls. This weekend brings six new entries: One late result from week 33 that was only published a week ago, three polls from week 34, and two that were conducted this week, with fieldwork done from Monday/Tuesday to Wednesday. For all purposes and intents, that means that any possible fallout from the Western (non-)intervention in Syria will not be reflected in the polls.

Raw Figures, Estimates and Predictions

As always, there is a good deal of variation in the published figures. The range for Merkel’s Christian Democratcs, for example, is 41 to 46 per cent. But for what it is worth, the model is ever more confident about the outcome of the election: The estimated probability of victory for the governing coalition is now 85 per cent (up from 78 per cent) even if one ignores tactical voting by CDU supporters. If this “loan vote” is factored in, the probability of a coalition victory is 94 per cent (up from 90). Unsurprisingly, the probability of a Red-Green majority is still estimated as zero.

minorparties-week-33

Support for the Minor German Parties, Estimates and Predictions (Week 35, 2013)

The  one remarkable change is the modest slump in support for the Greens, which have lost about two points over the last four weeks and are now well below their peak support of about 15 per cent in March. The slow upward trend of the Liberals is unbroken, and the Left is safely above the electoral threshold. Support for the two major parties is perfectly stable.

Since my interest here is (mostly) academic, I also began comparing past predictions (from week 33) with current estimates. The differences are small, but there is one interesting exception: Support for the Greens is now estimated to be 0.8 points lower than it should have been, given the information that was available two weeks ago. So it would seem that their support is indeed suffering from some random shock.

The Outlook

Today is the day of the televised debate between Steinbrück and Merkel (in Germany, known as “the Duel”). While we are professionally obliged to watch it, I don’t think that it will make much of a difference. Both candidates are extremely well known knowns. I also don’t think that Syria will matter for this campaign.

Have I just shot myself in the foot? Probably. Come back next week for the latest batch of surveys.

Aug 242013
 

The State of Play, Four Weeks Before the Election

Last week’s post on Merkel’s very good chances to win a third term created a bit of a stir. This week, I’m back with nine new polls (conducted between August 6 and August 19 by six different companies), which all point into the same direction.

What the Pollster Saw

On average, polls are in the field for five days (with a standard deviation of three days), so I continue to anchor each poll to a specific week in the calendar. Along with the raw data, the graphs show estimates for the true support for each party over 32 weeks, starting from Monday, the 31st of December. Eight of the new polls cover week 31 and week 32, while one is a late addition to estimate for week 30.

majorparties-week-33

Estimated/predicted Support for Major German Parties (2013 election). Click for Larger Image.

 

Support for Merkel’s Christian Democrats is between 39 and 47 per cent. The model, which accounts for previous levels of party support and variation across pollsters, puts them at 41 per cent. Findings for the major opposition party, the Social Democrats, are less variable at 22 to 25 per cent. The model places them at the upper limit of these current polls.

Results for the Greens are even more unanimous (12-13.5 per cent). The model agrees, confirming that their support has come down a tick or two over the last weeks.

The same cannot be said for the Left, which is almost static at seven per cent (polls: 6-8.1). That is well below their 2009 result, but also well above the electoral threshold of five per cent.

Finally, for the Liberals, Merkel’s coalition partner, things have improved ever so slightly. While the polls vary from three to seven per cent, the Liberals’ true level of support is currently estimated at 5.2 per cent. More importantly, after months of continuous near-death experiences, there seems to be an upward trend.

minorparties-week-33

Estimated/predicted Support for Smaller German Parties (2013 election). Click for Larger Image.

 

What Does That Mean for September 22 and Beyond?

This is my first shot at pooling the pre-election polls, so all predictions should be taken with a very generous pinch of salt. The model is possibly misspecified and rests on an number of questionable assumptions. The most obviously problematic amongst these is that polls are, on average, unbiased over the whole January-September timeframe. But hey, this is a blog, so let’s ignore this (and all other) problems for a second and believe that the trend-lines and credible intervals for the next four weeks are indeed credible.

Once we make this leap of faith, the probability of a return to a Red-Green coalition is approximately zero. Amongst 10000 simulations of week 38/39 (the election is on a Sunday), there is not a single one that gives a parliamentary majority to this prospective coalition.

The FDP, on the other hand, makes it past the electoral threshold in 83 per cent of my simulations, and in 78 per cent, there is a parliamentary majority for the present coalition. The true probability will be higher, as some CDU supporters will vote strategically for the FDP to help them across the threshold. If we assume that this behaviour is virtually guaranteed to succeed (it would be enough if about one in 40 CDU would cast a “loan vote”), the probability of a majority for the present coalition goes up to 90 per cent.

Put differently, the probability of a Red-Red-Green coalition (SPD, Left, Greens) is between 22 per cent (no loan votes for FDP) and 10 per cent (loan vote strategy works perfectly). But even if there was a majority for the three opposition parties, a coalition (or rather a toleration arrangement with the Left) would be highly unlikely (say p=0.1), making a Grand Coalition led by the CDU the default option. That again means that the probability of any government not being headed by the present chancellor is between one and two per cent (down from four per cent last week).

What About the Smaller Parties (AfD, Pirates, etc.)

For several months, most pollsters did not publish separate results for smaller parties such as the eurosceptic AfD or the internet-centric Pirates. Some have resumed giving itemized counts for “other” parties, and it currently seems safe to assume that neither will enter parliament. If they did, the Pirates would probably take away votes from the left parties, whereas the AfD would most likely weaken the two major parties. In either case, a Grand Coalition would become more likely.

What Everyone Else Thinks

The July issue of PS has two short pieces on forecasting models for the September election. Both pick Merkel as winner. So do Bundewahltrend (average over the six most recent polls), pollytix (weighted average of 15 most recent polls), and wahlistik (a poll aggregator run for the Zeit weekly). Las weekend, politicians in both major parties began floating the idea of a Grand Coalition, but given the latest polls, four more years of centre-right government seems to be the most likely option by far.

Stay Tuned

This post comes with lots of health warnings attached. In the past, forecasts have failed, faces have turned red, majorities have collapsed well before election day. I’ll be back once I have collected the next batch of polls.

Aug 162013
 

The Polls Are Noisy

Just over five weeks before the Bundestag election, there is much merriment about the current state of play. Support for the Liberals has been consistently below the electoral threshold of  five per cent for months, which implies that Merkel’s coalition would not be able to continue after September. Consequently, everyone is very excited about a more recent series of polls, which put the party at exactly five per cent. But even with n=2000, an exact confidence interval would range from 4 to 6 per cent. Add multistage sampling, house effects, and the fact that people do not necessarily know how they will vote in September, and you end up with a lot of noise.

Support for Major German Parties (2013)

Support for the Two Major German Parties in 2013

How much noise exactly? The good folks at wahlrecht.de publish marginal distributions for six major companies that regularly conduct random polls. I wrote a little program to collects everything published since January (the exact election date was officially determined on February 8 but was negotiated between the parties in January). Here are the results (click to enlarge).

Most polls are in the field for two to seven days, so I anchored them at their midpoints. My current data set spans 31 weeks, with just under 4 polls conducted each week.

Polling the German Election Polls

It’s obvious that there is a lot of variation in these 120 data points, making claims that this party has declined while another one surges rather dubious (though they still make excellent headlines). Poll aggregation is one possible and increasingly way out of this conundrum, so I decided to get my hands dirty, install (r)jags and bite the Bayesian bullet (something I have meant to do for years).

Support for Smaller German Parties in 2013

Support for three of the Smaller German Parties (2013)

My model is rather simplistic (I’ll post the code once it has stabilised). I assume that reported voting intentions are distributed multinomial, and that they depend on a) latent party support and b) house effects. I further make the rather heroic assumption that house effects are random with a mean of zero. Latent party support, on the other hand, follows a random walk, possibly with a drift: This week’s support is last week’s support plus some random change due to political events, plus some constant that accounts for steady up- or downward trends.

The Bayesian framework seems particularly appropriate here as it is technically and conceptually easy to come up with predictions for September 22, but I refrain from incorporating any prior beliefs and put vague distributions on all parameters. As far as I can tell (and that does not mean a lot), the model seems to converge without problems.

Estimated Support for Major German Parties 2013

Estimated and Predicted Support for Major German Parties (2013)

Somewhat surprisingly,the 95 per cent credibility interval (the shaded area around the trend line) is rather narrow for both the CDU and the SPD, implying that we can learn a lot from pooling many noisy polls. Support for Merkel’s Christian democrats was largely stable over the last seven months at about 42 per cent. This would make them the strongest party by far, although they are far away from the lofty 50 per cent they reached in some polls in April and June. According to the model, support for the Socialdemocrats is similarly stable, though at a much lower level of 26 per cent.

The predictions (to the right of the vertical line that marks the beginning of the last week included in the model) are less precise than the estimates, and become more vague as they extend towards election day, but it seems almost certain that the CDU will be the strongest party in the new parliament by a fair margin.

Predicted Support for Smaller German Parties

Estimated and Predicted Support for Smaller Parties in Germany (2013)

The model is also very confident about levels of support for the smaller parties. Green support peaked in March, but current and predicted levels are still above 12 per cent, which would be an improvement on the 2009 result. But since the SPD is so weak, the probability of a “Red-Green” majority in the next Bundestag is estimated to be (much) less than one per cent.

Support for the Left is estimated at about seven per cent, well above the threshold (the dashed line), but also well below their very strong result in 2009.

Finally, the FDP has shown an upward trend over the last 10 weeks or so and is projected to cross the threshold just in time for the election. The model estimates the probability of the FDP returning to parliament at 67 per cent.

Predicting the Inevitable

In reality, a sufficient number of potentially CDU voters might support the FDP for tactical reasons, pushing up that number towards certainty. But the coalition could come to an end even if that manoeuvre succeeds: The odds that the coalition garners more votes than the three left parties together are only slightly better than even at 58 per cent.

A “Red-Red-Green” coalition (or rather a Red-Green government tolerated by the Left), however, seems politically infeasible, suggesting a return to a Great Coalition lead by Angela M. with a subjective probability of at least 90 per cent.

If (if!) I take these estimates seriously for just one moment, that means that probability of Ms Merkel retaining her office is roughly 96 per cent. Let’s see how the next batch of polls plays havoc with this figure, shall we?

Jul 172013
 

Replication data for our forthcoming Political Analysis paper on our new, multinomial accuracy measure for bias in opinion surveys (e.g. pre-election polls) has just gone online at the PA dataverse. So if you want to gauge the performance of French pollsters over the 2012 presidential campaign (or cannot wait to re-run our simulations of B_w‘s sampling distribution), download our data and start playing.

A current version of our Stata add-on surveybias is included in the bundle. Alternatively, you can install the software into you personal ado dir by typing ssc install surveybias.

Jul 102013
 
Used Punchcard
BinaryApe / Foter / CC BY

In a recent paper, we derive various multinomial measures of bias in public opinion surveys (e.g. pre-election polls). Put differently, with our methodology, you may calculate a scalar measure of survey bias in multi-party elections.

Thanks to Kit Baum over at Boston College, our Stata add-on surveybias.ado is now available from Statistical Software Components (SSC).  The add-on takes as its argument the name of a categorical variable and said variable’s true distribution in the population. For what it’s worth, the program tries to be smart: surveybias vote, popvalues(900000 1200000 1800000), surveybias vote, popvalues(0.2307692 0.3076923 0.4615385), and surveybias vote, popvalues(23.07692 30.76923 46.15385) should all give the same result.

If you don’t have access to the raw data but want to assess survey bias evident in published figures, there is surveybiasi, an “immediate” command that lets you do stuff like this:  surveybiasi , popvalues(30 40 30) samplevalues(40 40 20) n(1000). Again, you may specify absolute values, relative frequencies, or percentages.

If you want to go ahead and measure survey bias, install surveybias.ado and surveybiasi.ado on your computer by typing ssc install surveybias in your net-aware copy of Stata. And if you use and like our software, please cite our forthcoming Political Analysis paper on the New Multinomial Accuracy Measure for Polling Bias.

Update April 2014: New version 1.1 available