Back in the mist of time, when I should have been was working on my PhD, I found a blue book on the shelf that a previous occupant of the office had left there. As I learned later, it was The Blue Book that introduced the S language, the predecessor of R. I got sidetracked (as you do) and taught myself how to produce beautiful graphs in what is now known as base R, and how to run poorly understood time series analyses (impossible in SPSS at this point).
A little later, I got hooked on Stata, and to the present day, I refuse to be Stata-shamed, as Ben Stanley put it. 95 per cent of the time, it does the job, and quickly so. Also, the documentation is simply excellent.
But every now and then, I came back to R because I needed something specific. And it was mostly fun. Having access to all these APIs (in fact, concurrently having more than one data set in memory) was exciting. Having a real, reasonably straightforward scripting /programming language at my disposal instead of Stata’s hodgepodge of three (four if you count the graph language) half-baked syntaxes was exhilarating. Having a go at the latest methods on the basis of nothing more than skimming a working paper (skipping every non-trivial equation) was… I guess a little bit like trimming your hair with a chainsaw.
But finding, installing, updating and then loading three packages, just to make recoding a little more intuitive? Seriously, R? Not so cool. In fact, finding a variable (whose name and data set must be given in full) was usually enough to reduce me to tears. Attach() somehow never does what I think it should do. And so, I would return to Stata once more, like <insert awkward metaphor>.
Then, during one of my last forays, I began playing with the tidyverse. And as the young ones are prone to say: my mind was blown. Tibbles! Pipelines! Lots of yummie helper functions! Going from long to wide format and back (in various different ways)! Grouping, summarising, and even some pythonesque list traversing. This was no longer the fascinating but slightly stroppy R I used to know.
Compared to the handful of letters and abbreviations that I use in Stata to get things done, recoding-wise, this is still quite verbose, and I have to look up just about everything. But I really like it. Like, really like it. And so doing more stuff in R is firmly on the endless List Of Things I Want To Look Into. To end on the most positive note possible, here is a gratuitous picture of a cat.
Dealing with packages in R can be somewhat painful, at least for me. Here is pak, a package that claims to make installing more packages less of a hassle, once you’ve managed to install it.
With less than two months to go until the EP elections, it is time to stir up some social media moral panic. And there is good reason for that. Here is an interesting piece by someone who claims to be involved in the development of youtube’s recommendation engine. Shock, surprise: apparently he helped create a monster that has learned that the average youtube viewer wants to see more, more, more anti-media content. Judging by my own recommendations, the monster delivers.
Staying with this theme, here is a post on social-media marketing your research. I read it twice to make extra sure that is indeed satire. YOU WILL BE SHOCKED. And now click on that darn link to demonstrate that the trick works.
It is a warm but grey and gloomy weekend in Germany, so here are three links I enjoyed:
In case you were wondering whether Trump is a) evil, b) senile or c) a master strategist: here is a piece arguing that c) is unlikely, though a) and b) could easily be true at the same time.
This is surprisingly accurate: academic life told through Wallace & Gromit gifs
Terminology matters for science. If people use different words for the same thing, or even worse, the same word for different things, scientific communication turns into a dialogue of the deaf. European Radical Right Studies are a field where this is potentially a big problem: we use labels like “New”, “Populist”, “Radical”, “Extreme” or even “Extremist” with abandon.
Topic modelling does not work well for (my) research paper abstracts
The Radical Right Research Robot is a fun side project whose life began exactly one year ago. The Robot exists to promote the very large body of knowledge on Radical Right parties and their voters that social scientists have accumulated over decades. At its core is a loop that randomly selects one of the more than 800 titles on my online bibliography on the Extreme/Radical Right every few hours and spits it out on twitter.
Yet the little android’s aim was always for some sort of serendipity, and so it tries to extract meaning from the abstracts (where available), sometimes with rather funny consequences. The robots’s first idea was to make use (structural) topic modelling. There are some implementations available in R and the first results looked promising, but in the end, topic modelling did not find meaningful clusters of papers that could easily be labelled with a common theme. One possible reason is that the abstracts are short, and that there are relatively few (less than 400) of them. And so the Robot reverted to using a small and fairly arbitrary set of keywords for identifying topics.
This approach produced some embarrassing howlers like this one:
the media and the #RadicalRight D. Halikiopoulou and T. Vlandas. “Risks, Costs and Labour Markets: Explaining Cross-national Patterns of Far Right Party Success in European Parliament Elections”. In: Jcms-Journal of Common Market Studies 54.3 (2016), pp. 636-655. https:// pic.twitter.com/K4YHAoZvNL
There are two problems here: first, even a single instance of a keyword in a given abstract is enough to trigger a classification, and second, the bot’s pedestrian implementation would classify an abstract using the last keyword that it detected, even if it was the most peripheral of several hits. Not good enough for world domination, obviously.
Newsmap works reasonably well for classifying topics in research paper abstracts
Looking for an alternative solution, the robot came across newsmap (now also available within quanteda), a geographical news classifier developed by Kohei Watanabe. Newsmap is semi-supervised: it starts with a dictionary of proper nouns and adjectives that all refer to geographical entities, say
'France': [Paris, France, French*]
'Germany': [German*, Berlin]
...
But newsmap is able to pick up additional words that also help to identify the respective country with high probability, e.g. “Macron”, “Merkel”, “Marseille”, “Hamburg”, or even “Lederhosen”. In a (limited) sense, it learns to identify geographical context even when the country in question is not mentioned explicitly.
But the algorithm is not restricted to geographical entities. It can also identify topics from a list. An so these days, the robot starts with a dictionary of seed words that is work in progress but looks mostly like this at the moment:
Results are not perfect, but at least they are less embarrassing than those from the simple keyword approach. One remaining problem is that newsmap tags each abstract with (at most) one topic. In reality, any given article will refer to two or more themes in the literature. Topic models are much more attractive in this respect, because they treat each text as a mixture of topics, and so the robot may have to revisit them in the future.
Co-citations within top 20 titles in Extreme / Radical Right studies
The titles are arranged in groups, with the “Extreme Right” camp on the right, the “Radical Right” group in the lower-left corner, and a small number of publications that is committed to neither in the upper-left corner. The width of the lines represents the number of co-citations connecting the titles.
What does the pattern look like? The articles by Knigge (1998) and Bale et al. (2010) are both in the “nothing in particular” group, but are never cited together, at least not in the data that I extracted. One potential reason is that they are twelve years apart and address quite different research questions.
Want to watch a video of this blog?
The Extreme / Radical Right network of co-citations
Apart from this gap, the network is complete, i.e. everyone is cited with everyone else in the top 20. This is already rather compelling against the idea of a split into incompatible two incompatible strands. Intriguingly, there are even some strong ties that bridge alleged intellectual cleavages, e.g. between Kitschelt’s monograph and the article by Golder, or between Lubbers, Gijsberts and Scheepers on the one hand and Norris and Kitschelt on the other.
While the use of identical terminology seems to play a minor role, the picture also suggests that co-citations are chiefly driven by the general prominence of the titles involved. However, network graphs can be notoriously misleading.
Modelling the number of co-citations in European Radical Right studies
Modelling the number of co-citations provides a more formal test for this intuition. There are counts of co-citations amongst the top 20 titles, ranging from 0 to 5476, with a mean count of 695 and a variance of 651,143. Because the variance is so much bigger than the mean, a regression model that assumes a negative binomial distribution, which can accommodate such overdispersion, is more adequate than one built around a Poison distribution. “General prominence” is operationalised as the sum of external co-citations of the two titles involved. Here are the results.
Variable
Coefficient
S.E.
p
external co-citations
0.0004
.00002
<0.05
same terminology
0.424
0.120
<0.05
Constant
2.852
0.219
<0.05
The findings show that controlling for general prominence (operationalised as the sum of co-citations outside the top 20), using the same terminology (coded as “extreme” / “radical” / “unspecific or other” does have a positive effect on the expected number of co-citations. But what do the numbers mean?
The model is additive in the logs. To recover the counts (and transform the model into its multiplicative form), one needs to exponentiate the coefficients. Accordingly, the effect of using the same terminology translates into a factor of exp(0.424) = 1.53.
What do these numbers mean?
But how relevant is this in practical terms? Because the model is non-linear, it’s best to plot the expected counts for equal/unequal terminology, together with their areas of confidence, against a plausible range of external co-citations.
Effect of external co-citations and use of terminology on predicted number of co-citations within top 20
As it turns out, terminology has only a small effect on the expected number of co-citations for works that have between 6,000 and 8,000 external co-citations. From this point on, the expected number of co-citations grows somewhat more quickly for dyads that share the same terminology. However, over the whole range of 6,000 to 12,000 external co-citations, the confidence intervals overlap and so this difference is not statistically significant.
Unless two titles have a very high number of external co-citations, the probability of them being both cited in a third work does not depend on the terminology they use. Even for the (few) heavily cited works, the evidence is insufficient to reject the null hypothesis that terminology makes no difference.
While the analysis is confined to the relationships between just 20 titles, these titles matter most, because they form the core of ERRS. If we cannot find separation here, that does not necessarily mean that it does not happen elsewhere, but if happens elsewhere, that is much less relevant. So: no two schools. Everyone is citing the same prominent stuff, whether the respective authors prefer “Radical” or “Extreme”. Communication happens, which seems good to me.
Arzheimer, Kai. “Conceptual Confusion is not Always a Bad Thing: The Curious Case of European Radical Right Studies.” Demokratie und Entscheidung. Eds. Marker, Karl, Michael Roseneck, Annette Schmitt, and Jürgen Sirsch. Wiesbaden: Springer, 2018. 23-40. doi:10.1007/978-3-658-24529-0_3 [BibTeX][Download PDF][HTML]
@InCollection{arzheimer-2018,
author = {Arzheimer, Kai},
title = {Conceptual Confusion is not Always a Bad Thing: The Curious Case of
European Radical Right Studies},
booktitle = {Demokratie und Entscheidung},
publisher = {Springer},
address = {Wiesbaden},
pages = {forthcoming},
year = 2018,
url =
{https://www.kai-arzheimer.com/conceptual-confusion-european-radical-right-studies.pdf},
doi = {10.1007/978-3-658-24529-0_3},
pages = {23-40},
html =
{https://www.kai-arzheimer.com/conceptual-confusion-european-radical-right-studies},
editor = {Marker, Karl and Roseneck, Michael and Schmitt, Annette and Sirsch,
Jürgen},
dateadded = {01-06-2018}
}
Short of training a hypercomplex and computationally expensive neural network (i.e. a grad student) to look at the actual content of the texts, analysing citation patterns is the most straightforward way to address the research question. Because I needed citation information, I harvested the Social Science Citation Index (SSCI) instead of my own bibliography. The Web of Science interface to the SSCI lets you save records as plain text files, which is all that was required. The key advantage of the SSCI data is that all the sources that each item cites are recorded, too, and can be exported with the title. This includes (most) items that are themselves not covered by the SSCI, opening up the wonderful world of monographs and chapters. To identify the two literatures, I simply ran queries for the phrases “Extreme Right” and “Radical Right” for the 1980-2017 period. I used the “TS” operator to search in titles, abstracts, and keywords. These queries returned 596 and 551 hits, respectively. Easy.
But how far separated are the two strands of the literature? To find out, I first looked at the overlap between the two. By overlap, I mean items that use both phrases. This applies to 132 pieces, or just under 12 per cent of the whole stash. This is not a state of zilch communication, yet by this criterion alone, it would seem that there are indeed two relatively distinct literatures. But what I’m really interested in are (co-)citation patterns How could I beat two long plain text lists of articles and the sources they cite into a usable data set?
When you are asking this kind of question, usually “there is an R package for that”™, unless the question is too silly. In my case, the magic bullet for turning information from the SSCI into crunchable data is the wonderful bibliometrix package. Bibliometrix reads saved records from Web of Science/SSCI (in bibtex format) and converts them into data frames. It also provides functions for extracting bibliometric information from the data. Before I move on to co-citations, here’s the gist of the code that reads the data and generates a handy list of the 10 most-cited titles:
library(bibliometrix)
D <- readFiles("savedrecs-all.bib")
M <- convert2df(D, dbsource = "isi", format = "bibtex")
# remove some obviously unrelated items
M <- M[-c(65,94,96,97,104,105,159,177,199,457,459,497,578,579,684,685,719,723),]
M <- M[-c(659,707),]
M <- M[-c(622),]
results <- biblioAnalysis(M, sep = ";")
S=summary(object = results, k = 10, pause = FALSE)
#Citations
CR <- citations(M, field = "article", sep = ". ")
CR$Cited[1:10]
So what are the most cited titles in Extreme/Radical Right studies?
The ten most cited sources in 726 SSCI items
Source
Number of times cited
Mudde (2007)
160
Kitschelt (1995)
147
Betz (1994)
123
Lubbers et al. (2002)
97
Norris (2005)
90
Golder (2003)
86
R.W. Jackman & Volpert (1996)
77
Carter (2005)
66
Arzheimer & Carter (2006)
65
Brug et al. (2005)
65
Importantly, this top ten contains (in very prominent positions) a number of monographs. The SSCI itself only lists articles in (some) peer-reviewed journals. Without the citation data, we would have no idea which non-peer-reviewed-journal items are important. Having said that, the situation is still far from perfect: We only observe co-citation patterns through the lens of the 1,000+ odd SSCI publications. But that’s still better than nothing, right? What about the substantive results of this exercise? The table clearly shows the impact that Cas Mudde’s 2007 (“Populist Radical Right”) book had on the field. It is the most cited and at the same time the youngest item on the list, surpassing the much older monographs by Betz (“Radical Right Wing Populism”) and Kitschelt (“Radical Right”). Two other monographs by Carter (“Extreme Right”) and Norris (“Radical Right”) are also frequently cited but appreciably less popular than the books by Betz, Kitschelt, and Mudde. The five other items are journal articles with a primarily empirical outlook and mostly without conceptual ambitions. Taken together, this suggests that the “Extreme Right” label lacked a strong proponent whose conceptual work was widely accepted in the literature. Once someone presented a clear rationale for using the “Radical Right” label instead, many scholars were willing to jump ship.
Getting to the co-citation network: are the Extreme / Radical Right literatures separated from each other?
If this was indeed the case, the literature should display a low degree of separation between users of both labels. Looking for co-citation patterns is a straightforward operationalisation for (lack of) separation. A co-citation occurs when two publications are both cited by some later source. By definition, co-citations reflect a view on the older literature as it is expressed in a newer publication. When two titles from the “Extreme Right” and “Radical Right” literatures are co-cited, this small piece of evidence that the literature has not split into two isolated streams. The SSCI aims at recording every source that is cited, even if the source itself is not in the SSCI. This makes for a very large number of publications that could be candidates for co-citations (18,255), even if most of them are peripheral European Radical Right studies, and a whopping 743,032 actual co-citations.
To get a handle on this, I extracted the 20 publications with the biggest total number of co-citations and their interconnections. They represent something like the backbone of the literature. How did I reconstruct this network from textual data? Once more, R and its packages came to the rescue and helped me to produce a reasonably nice plot (after some additional cleaning up)
NetMatrix <- biblioNetwork(M, analysis="co-citation",network = "references", sep = ". ")
# Careful: we are not interested in loops and not interested in separate connections between nodes. We convert the latter to weights
g <- graph.adjacency(NetMatrix,mode="max",diag=FALSE)
# Extract the top 20 most co-cited items
f <- induced_subgraph(g,degree(g)>quantile(degree(g),probs=(1-20/ length(V(g)))))
# Now build a vector of relevant terms (requires knowledge of these titles)
# 1: extreme, 2: radical, 3:none/other
# Show all names
V(f)$name
term <- c(3,2,1,1,2,1,1,2,1,2,3,2,2,2,3,1,1,1,1,1)
mycolours <- brewer.pal(3, "Greys")
V(f)$term <- term
V(f)$color <- mycolours[term]
Co-citation analysis: results
So, what are the results? First, here is the top 20 of co-cited items in the field of Extreme/Radical Right studies:
The twenty most co-cited sources in 726 SSCI items
Source
Co-citations within top 20
Total co-citations
Kitschelt (1995)
745
7700
Mudde (2007)
740
8864
Lubbers et al. (2002)
600
5212
Norris (2005)
568
5077
Golder (2003)
564
4687
Betz (1994)
542
6151
R.W. Jackman & Volpert (1996)
477
4497
Brug et al. (2005)
462
3523
Arzheimer & Carter (2006)
460
3551
Knigge (1998)
445
3487
Carter (2005)
389
3291
Arzheimer (2009)
376
3301
Ignazi (2003)
344
2876
Ivarsflaten (2008)
334
3221
Ignazi (1992)
331
3230
Rydgren (2007)
300
3353
Bale (2003)
297
3199
Brug et al. (2000)
276
2602
Meguid (2005)
246
2600
Bale et al. (2010)
134
2449
Many of these titles are familiar, because they also appear in the top ten of most cited titles and are classics to boot. And here is another nugget: for each title, a substantial share of about 10 per cent of all co-citations happen within this top twenty. This is exactly the (sub)network of co-citations I’m interested in. So here is the plot I promised:
Co-citations within top 20 titles in Extreme / Radical Right studies
Arzheimer, Kai. “Conceptual Confusion is not Always a Bad Thing: The Curious Case of European Radical Right Studies.” Demokratie und Entscheidung. Eds. Marker, Karl, Michael Roseneck, Annette Schmitt, and Jürgen Sirsch. Wiesbaden: Springer, 2018. 23-40. doi:10.1007/978-3-658-24529-0_3 [BibTeX][Download PDF][HTML]
@InCollection{arzheimer-2018,
author = {Arzheimer, Kai},
title = {Conceptual Confusion is not Always a Bad Thing: The Curious Case of
European Radical Right Studies},
booktitle = {Demokratie und Entscheidung},
publisher = {Springer},
address = {Wiesbaden},
pages = {forthcoming},
year = 2018,
url =
{https://www.kai-arzheimer.com/conceptual-confusion-european-radical-right-studies.pdf},
doi = {10.1007/978-3-658-24529-0_3},
pages = {23-40},
html =
{https://www.kai-arzheimer.com/conceptual-confusion-european-radical-right-studies},
editor = {Marker, Karl and Roseneck, Michael and Schmitt, Annette and Sirsch,
Jürgen},
dateadded = {01-06-2018}
}
counts of co-citations amongst the top 20 titles, ranging from 0 to 5476, with a mean count of 695 and a variance of 651,143. Because the variance is so much bigger than the mean, a regression model that assumes a negative binomial distribution, which can accommodate such overdispersion, is more adequate than one built around a Poison distribution. “General prominence” is operationalised as the sum of external co-citations of the two titles involved. Here are the results.
