May 102014

In a recent publication (Arzheimer & Evans 2014), we propose a new multinomial measure B for bias in opinion surveys. We also supply a suite of ado files for Stata, surveybias, which plugs into Stata’s framework for estimation programs and provides estimates for this and other measures along with their standard errors.  This is the first instalment in a mini series of posts that show how our commands can be used with real-world data. Here, we analyse the quality of a single French pre-election poll.

Installing surveybias for Stata

You can install surveybias directly from this website (net from, but it may more convenient to install from SSC ssc install surveybias

Assessing Bias in Presidential Pre-Election Surveys

. use onefrenchsurvey

The French presidential campaign of 2012 attracted considerable political interest. Accordingly, numerous surveys were fielded. onefrenchsurvey.dta (included in our package) contains data from one of them, taken a couple of weeks before the actual election. The command I will discuss in this post is called (*drumroll*) surveybias and is the main workhorse in our package. surveybias needs exactly one variable as a mandatory argument: the voting intention as measured in the survey, which is appropriately called “vote” in this example. Moreover, surveybias requires an option through which must submit the true distribution of this variable. Absolute or relative frequencies will do just as well as percentages, since surveybias will automatically rescale any of them.

Ten candidates stood in the first round of the French presidential election in 2012, but only two of them would progress to the run-off. While surveybias can handle variables with up to twelve categories, requesting estimates for very small parties increases the computational burden, may lead to numerically unstable estimates and is often of little substantive interest. In onefrenchsurvey.dta support for the two-lowest ranking candidates has therefore been recoded to a generic “other” category. The first-round results, which serve as a yardstick for the accuracy of the poll, are submitted in popvalues(). For other options, have a look at the documentation.

. surveybias vote, popvalues(28.6 27.18 17.9 9.13 11.1 2.31 1.15 1.79 0.8)
______________ ________________________________________________________________
vote       Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
______________ ________________________________________________________________
Hollande   -.0757639   .0697397    -1.09   0.277    -.2124512    .0609233
Sarkozy    .0477294   .0689193     0.69   0.489    -.0873499    .1828087
LePen   -.0559812   .0823209    -0.68   0.496    -.2173271    .1053648
Bayrou    .3057213   .0953504     3.21   0.001     .1188379    .4926047
Melenchon   -.0058251   .0988715    -0.06   0.953    -.1996096    .1879594
Joly   -.0913924   .2154899    -0.42   0.671    -.5137449      .33096
Poutou   -.8802476   .4482915    -1.96   0.050    -1.758883   -.0016125
DupontAigna   -.5349338   .3031171    -1.76   0.078    -1.129032    .0591648
other    .1841789   .3177577     0.58   0.562    -.4386147    .8069724
______________ ________________________________________________________________
B    .2424193   .0767485     3.16   0.002     .0919949    .3928437
B_w    .0965423    .039022     2.47   0.013     .0200605    .1730241
______________ ________________________________________________________________

Ho: no bias
Degrees of freedom: 8
Chi-square (Pearson) = 18.695468
Pr (Pearson) = .01657592
Chi-square (LR) = 19.540804
Pr (LR) = .01222022

The top panel lists the Ai for the first eight candidates plus the “other” category alongside their standard errors, z- and p-values, and confidence intervals. Ai is a party-specific, multi-party version of Martin, Traugott, and Kennedy’s measure A and reflects bias for/against any specific party. By conventional standards (p 0.05), only two of these values are significantly different from zero: Support for François Bayrou was overestimated (A4 = 0.31) while support for Philippe Poutou was underestimated (A7 = -0.88).

Poutou was the little known candidate for the tiny “New Anticapitalist Party”. While he received more than twice the predicted number of votes (1exp(-0.88) 2.4), the case of Bayrou is more interesting. Bayrou, a centre-right candidate, stood in the previous 2007 election and came third with a very respectable result of almost 19 per cent, taking many political observers by surprise. In 2012, when he stood for a new party that he had founded immediately after the 2007 election, his vote effectively halved. But this is not fully reflected in the poll, which overestimates his support by roughly a third (exp(0.31) 1.35). This could be due to (misguided) bandwagon effects, sampling bias, or political weighting of the poll by the company.

The lower panel of the output lists B and Bw, a weighted version of our measure. B, the unweighed average of the Ais absolute values, is much higher than Bw. This is because the estimates for all the major candidates with the exception of Bayrou were reasonably good. While support for Poutou and also for Dupont-Aignan was underestimated by large factors, Bw heavily discounts these differences, because they are of little practical relevance unless one is interested specifically in splinter parties.

As outlined in the article in which we derive B, B’s (and Bw’s) sampling distribution is non-normal, rendering the p-value of 0.002 somewhat dubious. surveybias therefore performs additional χ2-tests based on the Pearson and the likelihood-ratio formulae, whose results are listed below the main table. In this case, however, both tests agree that the null hypothesis of no bias is indeed falsified by the data.

While their p-values are clearly higher than the one resulting from the inappropriate z-test on B, they are close to the p-value for Bw. This is to be expected, because the upward bias and the non-normality become less severe as the number of categories increases, and because the weighting reduces the impact of differences that are small in absolute numbers but associated with large values on the log-ratio scale.

surveybias leaves the full variance-covariance matrix behind for your edification. Parameter estimates, chi-square values and probabilities are available, too, so that you can easily test all sorts of interesting variables about bias in this poll.

Apr 052014

The first round of the French local elections created quite a stir, but the second round of the French local elections was not a bad day for the anoraks either. While the initial focus was on the not totally unexpected success of the Front National, most of my correspondents agree that the real news is the annihilation of the governing Parti socialiste. Amongst the many posts, here are the ones I find most interesting:

7481490802 6fd4a85ab5 m marine le pen French Local Elections Round 2 Link RoundupPhoto by Mashthetics cc French Local Elections Round 2 Link Roundup

  1. Over at the LSE blog, Jocelyn Evans and Gilles Ivaldi, the grand seigneurs of Front National blogging, argue just that: The shock of Sunday’s French municipal elections was the Socialist defeat (this is actually reblogged from their own 500signatures site). The Guardian agrees.
  2. For Art Goldhammer, it is the end of municipal socialism.
  3. Part of the fallout was the appointment of Manuel Valls as new French PM. John Gaffney thinks that this was a desperate move that will come back to haunt Hollande.
  4. But Art Goldhammer links to a source claiming that Hollande actually has been wanting Valls to be PM for a long time.
  5. And here is an “interview” on the Monkey Cage – interesting questions/answers on Hollande’s electoral disaster, but in a slightly odd format.
Mar 242014

Predictably, the Front National has done well in yesterday’s local elections in France, and predictably, everyone is very excited about. Much has been said about the FN’s performance, but not yet by everyone. Here is a bunch of useful links to bring your punditry up to scratch.

  1. First, the electoral system for the locals. It is slightly weird (no surprises here). La Jeune Politque has an explainer (not sure how complete it is), and also a map identifying some of the interesting contests. They were also live-blogging the first round of the French locals.
  2. EUobserver has a short summary of the results. The Guardian has some additional coverage.
  3. Writing ahead of the election, James Shields (over the LSE blog) thinks the FN will remain isolated and hampered by the electoral system.
  4. The 500 signatures blog by Messrs. Evans and Ivaldi has forecasts (now backcasts) of the national result as well as background stories on the races in the big three (Paris, Lyon, Marseilles).
  5. Euractiv has another short summary and outlook for the second round.
  6. Art Goldhammer looks at the (slightly) longer game, i.e. the implications for the Hollande adminstration.

Have I missed something important?

Jan 282013

Radical Right buffs out there, have you submitted your paper proposal for Bordeaux yet? As you may or may not know, Liz Carter and I have put together a six-panel-section on the New Right for the 7th ECPR General Conference in September. The deadline is February 1, i.e. in just four days – submit your proposal now if you are interested at all (requires myECPR registration).

Extreme Right and Ethnic Politics in Eastern Europe
Radical/Extreme Right Party Ideology, Strategy and Organisation
The Eurozone Crisis and the Radical Right
The New Right and the 'Squeezed Middle': Service Sector Vulnerability and Populist Appeal
The Populist Voter
The Radical Right in the Post-Communist Context: New Perspectives on an Old Phenomenon


Oct 292012

Like social networks, multilevel data structures are everywhere once you start thinking about it. People live in neighbourhoods, neighbourhoods are nested in municipalities, which make up provinces – well, you get the picture. Even if we have no substantive interest in their effects, it often makes sense to control for structures in our data to get more realistic standard errors.

Now the good folks over at the European Social Survey have reacted and spent the Descartes Prize money on compiling multilevel information and merging them with their own data. So far, the selection is a little bit disappointing in some respects. Homicide rates, for instance, are reported on the national level only. But there are some pleasant surprises (I guess due to Eurostat, who collect such things): We get unemployment, GDP growth and even student numbers at the NUTS-3 level. Since you asked, NUTS is the Nomenclature of (subnational) Territory, and level 3 is the lowest level for which comparative data are normally published.

Regrettably, the size and number of level 3 units is not necessarily comparable across countries: For Germany, level 3 corresponds to about 400 local government districts, while France is divided into 96 European Departments. But if you need to combine top-notch survey data with small(ish) regional data, it’s a start, and not a bad one.

Jun 112012

Colleague Rainbow Murray is in Paris to do a little bit of observing. Her personal account of the count in one Parisian ward is quite intriguing. I had no idea that each candidate has to provides their own ballot paper. Not a very green thing, it would seem. Art Goldhammer puts Laurent Joffrin’s comment on Mélenchon’s failture/LePen’s success in perspective. ‘We all tend to overinterpret the results of elections’. Can’t argue with that. Meanwhile, Matt Goodwin ponders the question if Marine LePen’s ‘detoxified’ version of her father’s Front National is serving once more the blueprint for the (West) European Extreme Right, with Greece providing the counterpoint.And my own thoughts? Looking back, perhaps the most remarkable fact is how much our collective excitement has waned since the presidential election, although legislative elections really arereally important. Is this just because hundreds of multi-person races can simply not compete with the drama of the shoot-out between Sarkozy and Hollande, or just another piece of evidence of the internet’s detrimental effects on our attention spans? Continue reading »

Jan 172011

If this post’s title does make any sense to you, chances are that you are one of us anoraks who had a brilliant weekend of extreme right spotting. In France, Jean-Marie Le Pen stepped down as leader of the Front National, just under 40 years after he founded the party. He is succeeded by his youngest daughter, who is portrayed as a moderniser (hey, she’s twice divorced) and a moderate (by FN standards). While this story might conjure the image of Prince Charles, Marine’s rise through the Front’s ranks was quick, largely unexpected and a major source of aggravation for Bruno Gollnisch, the controversial academic who became the party’s number two after the old number two, Bruno Mégret, left the party to found the MNR in 1999. If Gollnisch (who was soundly beaten by LePen the younger in the leadership contest) aims to repeat that stunt remains to be seen. I’m sure there is a silly story about men named Bruno who turn out to be the real Princes Charles here (both spent a lot of  time eyeing the leadership and are in their early 60s now ), but more importantly, Marine is going to change le Front, though her father might be tempted to meddle. These right-wingers know a thing or two about family values.

Meanwhile, in Germany the NPD, arguably the most radical amongst the electorally viable right-wing parties in Germany has celebrated its merger with its old rival DVU. The DVU was founded in the early 1970s as a marketing device for right-wing books, journals and paraphernalia and became a party in the 1980s. For nearly 40 years, it was completely dominated by its founder Gerhard Frey, who finally stepped down in 2009, aged 76. While has successor planned for the merger that became effective from January 1, some regional leaders are less than happy and seem willing to either take the issue to the courts over some alleged irregularities, or to set up a new party of their own. Either way, it would seem that the German extreme right remains divided, as it has been since the 1980s.

 Marine anointed, DVU and NPD merged
Aug 012009

These days, a bonanza of political information is freely available on the internet.  Sometimes this information comes in the guise of excel sheets, comma separated data or other formats which are more or less readily machine readable. But more often than not, information is presented as tables designed to be read by humans. This is where the gentle art of screen scraping, web scraping or spidering comes in. In the past, I have used kludgy Perl scripts to get electoral results at the district level off sites maintained by the French ministry of the interior or by universities (very interesting if you do not really speak/read French). A slightly more elegant approach might be to use R’s builtin Perl-like capabilities for doing the job, as demonstrated by Simon Jackman. Finally, Python is gaining ground in the political science community,  which has some very decent libraries for screen/web scraping – see this elaborate post on Drew Conway’s Zero Intelligence Agents blog. But, let’s face it: I am lazy. I want to spend time analysing the data, not scraping them. And so I was very pleased when I came across outwit, a massive plugin for the firefox browser (Linux, Mac and Windows versions available) that acts as a point-and-click scraper.

outwit 1 300x178 Web scraping made easy: outwit

French Départements (from Wikipedia)

Say you need a dataset with the names and Insee numbers for all the French Départements. The (hopefully trustworthy) Wikipedia page has a neat table, complete with information on the Prefecture and many tiny coats of arms which are of absolutely no use at all. We could either key in the relevant data (doable, but a nuisance), or we could try to copy and paste the table into a word processor, hoping that we do not lose accents and other funny characters, and that WinWord or whatever we use converts the HTML table into something that we can edit to extract the information we really need.

Or you we could use outwit. One push of the button loads the page

outwit 22 300x178 Web scraping made easy: outwit

Scraping a table with outwit

into a sub-window, a second push (data->tables) extracts the HTML tables on the page. Now, we can either mark the lines we are interested in by hand (often the quickest option) or use a filter to selfect them. One final click, and they are exported as a CSV file that can be read into R, OpenOffice, or Stata for post processing and analysis.

While I’m all in favour of scriptable and open-source tools like Perl, Python and R, outwit has a lot to go for it if all you need is a quick hack. Outwit also has functions to mass-download files (say PDFs) from a page and give the unique names. If the job is complex, there is even more functionality under the hood, and you can use the point-and-click interface to program you own scraper, though I would tend use a real programming language for these cases. At any rate, outwit is a useful and free tool for the lazy data analyst.

 Web scraping made easy: outwit
Sep 292008

In a recent article in the European Journal of Political Research, Kestilä and Söderlund claim (amongst other things) that in the French regional elections of 2004, turnout and district magnitude have significant negative effects on the extreme right vote whereas the effects of the number of party lists and unemployment are positive and significant. Most interestingly, immigration (which is usually a very good predictor for the radical right vote) had no effect on the success of the Front National. More generally, they argue that a subnational approach can control for a wider range of factors and provide more reliable results than cross-national analyses (now the most common approach to this phenomenon). My colleague Liz Carter and I disagreed and engaged in a massive replication/re-analysis endeavour. The outcome is a critique of the KS model of subnational political opportunity structures in regional elections. In this paper, we dispute Kestilä’s and Söderlund’s claims on theoretical, conceptual and methodological grounds and demonstrate that their findings are spurious. Today, the European Journal has accepted the article for publication (probably in 2009) icon smile Does Immigration help or hurt the Front National in France?

Technorati-Tags: , , , , , , , , , , , ,