A short scale for measuring political secularism
Social scientists have long been aware that religion is a powerful political force and that by implication, significant changes to the religious makeup of a society will have far-reaching political repercussions. The growing number of people who identify as Muslims is one such change that has become a hot button issue in Europe. However, Muslims currently make up just five per cent of Europe’s population, and even under a high immigration/high fertility scenario, their share will rise to less than 15 per cent over the next three decades (Center 2017). At least numerically, secularisation is a much more important development in Europe.The script refused to produce HTML tables. The full-fat numbers are in the PDF.
Secularisation is a multi-faceted term that denotes both social and intrapersonal processes (Sommerville 1998) that diminish the role of religion at the societal, group, and individual level (Stolz and Tanner 2019, 2). At the micro-level, applied research usefully distinguishes between a decline in membership and practice on the one hand and a decline in individual beliefs on the other. The former may outpace the latter, creating “unchurched” segments of the population that still believe but no longer belong (Davie 1990). However, in their global survey of religious life, Norris and Inglehart 2011 demonstrate that in some parts of the world and most notably in Western Europe, both membership and belief have been on the retreat for decades. These trends continue to the present day, making Western Europe “one of the world’s most secular regions” where “many do not describe themselves as Christian” (Center 2018).
For many of these citizens (in Europe and elsewhere), being secular is not just the absence of religion: they embrace secularism as a non-religious world-view. In the broadest possible sense, secularism is an idea that “seeks to restrict religious elements” (Stolz and Tanner 2019). While such an outlook has multiple dimensions, one particularly interesting one is political secularism, a preference to limit religion’s role in politics (Beard et al. 2013).
One prominent account (Voas 2009) of religious change in Europe that is well supported by empirical data holds that long-running societal secularisation brings about individual secularism, via a mechanism of changing socialisation patterns and generational replacement. At the same time, individual (political) secularism may lead to demand for policy changes (e.g. full legal recognition of same-sex partnerships) that in turn drive further societal secularisation. Somewhat paradoxically, secularisation in Europe is also contributing to a re-politicisation of religion: it makes societies more culturally diverse and results in polarisation between religious organisations and citizens on the one hand and their political secular opponents on the other (Pickel 2017).
To gauge the importance of these issues on the micro-level, a survey measure for political secularism is required. Yet, although there is a plethora of scales for measuring religiousness, secularism as such is rarely studied, and not a single psychometrically evaluated instrument for measuring politically secular attitudes exists.
In this research note, I briefly review the concept of political secularism and present a cluster of items which target it. Using data from four large population representative samples, I show that these items form a short internally consistent scale, and that this scale also displays convergent and discriminant validity.
2. What is political secularism, and how can it be measured?
Rates of religious membership and participation have been a concern for social scientists since the inception of sociology in the 19th century. Attempts to complement them by instruments that capture more private religious practice as well as the structure of religious beliefs and attitudes hark back to at least the 1950s. Allport and Ross’s distinction between intrinsic and extrinsic orientations (Allport and Ross 1967) is perhaps the most famous example of this early research. Less than two decades later, Gorsuch 1984 reviewed the many instruments already existing at the time and strongly argued against their further proliferation. Yet after another quarter-century, Cutting and Walsh 2008 counted nearly 180 scales for measuring religiousness, and their number is still growing.
Almost all of these scales implicitly follow Sommerville’s classic notion that secular attitudes represent a “shift of attention from ultimate (religious) concerns to proximate concerns” (Sommerville 1998), i.e. a lapse from or a neglect of religion. While such “passive secularism” is an important phenomenon in itself, “active secularism”, i.e. “the affirmation of secular identity and beliefs” (Campbell et al. 2018) is arguably a more interesting cluster of attitudes. While passively secular citizens are by definition indifferent towards religion but may still accept its traditional role in society and politics, actively secular citizens are positively opposed to religious arguments and authority, which may spur them into political activity and lead them to adopt more extreme positions (see, e.g. (Layman and Weaver 2016; Brockway 2018; Layman et al. 2021)).
Just like religion, secularism has several “dimensions” (Campbell et al. 2018) or “facets” (Beard et al. 2013). These include philosophical convictions, social values, secular identities, and negative emotions towards religious authority and doctrine, all of which can have political consequences.
In a narrower sense, political secularism refers to one’s attitudes on “the proper role of religious beliefs in political life” (Beard et al. 2013). More succinctly political secularism holds that religious beliefs should not be used to justify curbing individual freedoms, and should play little, if any role in political debates.
As a political and legal idea, political secularism is well-researched ((Bhargava 2008; Cliteur and Ellian 2020)). But there is not one validated scale for political secularism at the micro-level. This is problematic, because politically secular views are believed to become more and more widespread in Europe and even in the US, and because they represent a particularly disruptive type of active secularism. This is because they are conceptually linked to a host of polarising “morality policies”, which pit religious views and actors against notions of individual freedom (Mourão Permoser 2019). If one wants to assess the alleged spread of politically secular views as well as their impact on politics, it is therefore desirable to have an instrument that is both reliable and valid and preferably comprises multiple indicators.
But even single-item questions or short batteries that tap into politically secular views are rare in the extant literature. Beard et al. 2013, who prominently make the case for researching political secularism at the micro-level, propose only two items that fit their own definition: “Do you worry the government is getting too involved in the issue of morality?” and “Should churches and other houses of worship keep out of political matters” (Beard et al. 2013).1
Brockway 2018, in his useful study of “committed seculars”, relies on a single item that refers to one important but very specific aspect of political secularism, namely the separation of church and state. Similarly, Castle 2015 measures opinions on “religious establishment” using an index comprising four items that refer to government institutions and officials engaging in and supporting religious activities.
Finally, Ribberink, Achterberg, and Houtman 2018 measure “anti-religiosity” by combining four items that were included in the 1998 and 2008 waves of the ISSP survey. Two of these items refer to respondents’ views of believers and their creeds as intolerant and belligerent. Such opinions may be secular, but they are not politically secular in the aforementioned sense. The two other items tap into attitudes on the role of religious leaders in politics. These items are a better fit for the concept of political secularism but again address a single, rather narrow and specific aspect of it.
Conversely, a valid and reliable scale for measuring political secularism should consist of several items that cover the breadth of the relationship between religion and politics. While developing such a scale from scratch would have its own obvious benefits, re-using existing items from large-scale surveys has at least three important advantages. First, it is already clear that these items “work” in surveys of the general public. Second, replicating single items or group of items opens up the possibility of meaningful comparisons between new and existing surveys. Third, if a whole set of items is replicated that form a scale, its reliability and validity can be assessed across different settings.
The questionnaire of an older survey (“Church and Religion in an enlarged Europe” (CREE)), which ran in 2006 in various European countries and targeted attitudes on “religion, society, and life in general” (Pollack, Pickel, and Müller 2007), provides a useful starting point. Its large pool of questions contains five items that closely fit Beard et al.’s ((2013)) definition of political secularism.
One of these items (Symbols, see below) was taken from the earlier “Religious and Moral Pluralism” survey. Another one (Debates) came from the ISSP survey series and is one of the two that were also used by Ribberink, Achterberg, and Houtman 2018. The remaining three were written by members of the CREE team.2
These five (and also some of the other) items were then subsequently included in the 2012 iteration of Germany’s biannual General Social Survey (ALLBUS) in somewhat modified form:
- “The EU treaties should contain a reference to the Christian god” (God EU) (reversed)
- “Religious symbols such as crucifixes should be banned from state schools” (Symbols)
- “Education in state schools should be free of religious elements” (Education)
- “Scientific research on humans should not be limited by religious norms and values” (Science)
- “Religious norms and values have no place in public debates on political issues” (Debates)
When the “Problems of Representation in the Domain of Biopolitics” (PRDB) project set out in 2016 to measure political secularism and its effects at the micro-level, it adopted these five items for its nationally representative survey. For comparability with the ALLBUS and because the rewritten items are even better suited for measuring political secularism,3 the PRDB project also used the new wording.
The PRDB project chose these items because collectively, they capture different aspects of political secularism as defined by (Beard et al.). The first question was highly controversial in European Union politics in the 2000s and is still useful as a supranational variation of the classic question about the separation of church and state. The second and third items relate to the role of religion in state schools, which is another contentious issue for secularists in many societies. The fourth item alludes to the role of religious authority in one specific domain of regulatory politics where appeals to religious doctrine are frequent. More generally, it also taps into secular support for science and rationality. Finally, the fifth item spells out the core tenet of political secularism: that religion should play no role in politics.
The analyses presented in the remainder of this research note are based on data from the 2012 ALLBUS round (Leibniz-Institut für Sozialwissenschaften 2019) and the 2016 PRDB survey (Arzheimer 2016). Both datasets are available in their entirety from Germany’s national data archive, where they can be located by following their respective DOIs.
Because of the persistent political, social, and economic east-west differences within Germany, both studies drew separate and disproportionately large samples of respondents from Germany’s eastern states. For the present research question, this is particularly apt because West German politics was dominated for decades by a Christian Democratic party, whereas Eastern Germany was a self-avowed irreligious state. As a consequence of this research design, the analyses are effectively based on four large (n=
740–2538), independent samples.
While the ALLBUS usually uses an odd (five, seven, or eleven) number of categories for measuring agreement, they retained the CREE’s original four-point format. Conversely, the PRDB survey employed five categories with a neutral mid-point for almost all items and adjusted the items’ response options accordingly. Otherwise, the wording of the items is identical across the samples.
4. Reliability and Dimensionality
Any presumptive scale must be internally consistent, i.e. its items should tap into a single common dimension. Internal consistency is one important type of reliability, which in turn is a necessary but insufficient condition for validity. Traditionally, internal consistency is assessed by calculating Cronbach’s alpha (Cronbach 1951), which can be interpreted as the mean of all possible split-half reliabilities across a given set of items.
Confirmatory Factor Analysis (CFA) provides a complementary perspective that is more nuanced and can also help mitigate some of the well-known problems4 linked to the exclusive reliance on alpha (Viladrich, Angulo-Brunet, and Doval 2017). However, the common practice of establishing unidimensionality mechanically by assessing the fit of a single-factor model may also be misleading, because established fit indices as well as newer measures may be too sensitive to trivial violations under some circumstances while failing to identify clearly multidimensional structures in others (Reise et al. 2013). Therefore, a detailed inspection of the measurement model that takes into account not just various fit indices and newer variations of alpha but also the factor loadings and possibly alternative specifications of the measurement model is required.
One final complexity arises from the ordinal nature of the survey responses. While items with as few as five categories are sometimes treated as if they were continuous (Viladrich, Angulo-Brunet, and Doval 2017), the WLSMV estimator, which was specifically developed for large samples of ordinal measurements, was employed for both surveys. Therefore, the factor loadings are probit coefficients, and variants of alpha that account for the ordinal nature of the data (Green and Yang 2009) were calculated. Threshold parameters are omitted from the main tables but documented in the appendix.
shows the estimates for a preliminary measurement model for political secularism. All five items display substantial factor loadings in the correct direction. (Ordinal) alpha varies between 0.7 and 0.83, a range that is usually seen as acceptable to good. Because the measurement model assumes that the items are congeneric whereas alpha presupposes tau-equivalent items, (ordinal) omega is a more appropriate measure of internal consistency (Viladrich, Angulo-Brunet, and Doval 2017). The respective row in Table ???
shows that values for omega are indeed slightly higher, but the differences are minuscule.
The evidence from the fit indices, however, is more mixed. The CFI and the SRMR show an acceptable fit in all four samples. Conversely, the TLI and the RMSEA indicate that model fit is unsatisfactory for all respondents but those interviewed in the east in 2012.
A closer look at the bivariate correlations reveals that there is a particularly close relationship between the two items referring to the role of religion in state schools, which can be explained by their very similar content. This in turn suggests that allowing their error variances to correlate could improve the model fit substantially.
From a substantive point of view, however, it is preferable to remove the Symbols item from the scale. First, having two items that refer to public education gives disproportionate weight to this specific aspect of political secularism. Second, while both items are similar, “religious elements” is more general and already encompasses “religious symbols”. Third, the meaning of the former item is also clearer, because almost all federal states provide some form of religious education in state schools, whereas crucifixes in schools (and other public buildings) are closely linked to the Bavarian Christian Democrats (CSU) and their representation of Bavarian identity. Moreover, because “religious symbols” also include the hijab, this item may also tap into anti-Muslim sentiment. And finally, short(er) scales have clear advantages: they often provide a more realistic assessment of reliability, increase the motivation of and decrease the burden on respondents, and are cheaper to administer (Gogol et al. 2014).
shows the estimates for the four-item measurement model. The factor loadings remain high and their direction is still correct. Omega and alpha are somewhat lower, which is expected because they partly depend on the number of items, but still acceptable. Finally, the CFI, TLI, and SRMR indicate near perfect fit in all four samples. The RMSEA indicates a very good fit for all samples but the eastern one in 2016.
Taken together, these findings suggest that the four items form a consistent scale that works reasonably well across slightly different formats (four vs five scale points) and cultural settings (Germany’s staunchly secular eastern states vs the more religiously diverse states in the west). The next section will assess the question whether the scale is also valid.
Many different types of validity have been proposed in the literature. However, they all relate to the same underlying question: to what degree does an instrument tap into what it is supposed to measure? Thus, they should be seen as “tests” that provide (sometimes conflicting) evidence on the question of validity, which “can never be answered with absolute certainty” (Bollen 1989).
This uncertainty is fundamental: in an ideal world, validity could be quantified simply by the correlation of a scale with some universally accepted external criterion, but in reality, there is usually no such gold standard. Like reliability, validity is therefore best investigated making use of CFA, which opens up many sophisticated possibilities (see e.g. (O’Leary-Kelly and Vokurka 1998)).
In this section, I follow a comparatively simple approach (see e.g. (Hayes, Glynn, and Shanahan 2005)): I demonstrate that scores on the secularism scale are highly correlated with measurements that they should correlate with (convergent validity), yet remain distinguishable from them (discriminant validity). This provides a more rounded view of the scale’s validity than the relationship with a single criterion and is a first step towards a full nomological validition, which would test a whole network of theoretical propositions and auxiliary assumptions (see, e.g., (Hagger, Gucciardi, and Chatzisarantis 2017)). There are two constructs available across the four samples that can be used for validation, one rather specific, the other more general.
Attitudes on a specific policy, namely legal access to abortion, form the first of these constructs. The regulation of abortion is widely seen as one of the most prominent “morality issues” (Mooney and Schuldt 2008; Studlar, Cagossi, and Duval 2013) worldwide. Efforts to limit legal access to abortions are almost universally spearheaded by religious organisations, who explicitly claim that they act on of their beliefs. This suggests that political secularism should be positively associated with more permissive views on abortion. But because this is a single specific policy, because many other factors may shape one’s views on abortion, and because the items used in the surveys make no reference to religious doctrine, this correlation should be far from perfect.
All surveys include four identical or very similar items on the regulation of abortion (see appendix appendix-measurement-abortion). A preliminary analysis shows that all items display substantial loadings on the common factor. Ordinal alpha is high and ordinal omega is acceptable across all four samples (see Table ???
in the appendix) so that the construct may be used for validation.
The second, more general construct is a composite measure of religiousness that encompasses individual and collective religious practice on the one hand (prayer and attendance) and self-conception as more or less religious on the other (see appendix appendix-measurement-religiosity). All three items display substantial loadings on the common factor. Alpha is high, and omega is acceptable across all four samples (see Table ???
in the appendix).
Following (Beard et al.), I have argued in section theory that political secularism should not be understood as the absence of religiousness but rather as an attitude on the proper role of religion in political life. For the validity of the political secularism scale, its correlation with religiousness is therefore crucial. Overall, a strong negative relationship is to be expected: non-religious citizens should be opposed to religious interference in the public sphere, whereas believers should support institutional arrangements that make outcomes which align with their beliefs more likely. Having said that, many religious citizens (particularly in some protestant traditions) may still favour a clear separation of church and state. Conversely, lukewarm believers and even non-believers may see a societal value in religion.
To investigate the validity of the secularism scale, a joint model of secularism and attitudes on abortion was estimated (see the tables in appendix appendix-full-models-abortion for the full results). Table ???
shows that the two-factor models, which treat secularism and abortion attitudes as separate constructs, fit the data very well. In line with expectations, the correlation is clearly positive (more secular respondents do indeed have more permissive views on legal access to abortion) but far from perfect, with values between 0.35 and 0.53. Conversely, assuming that all items measure a single common factor results in a dramatic decline of model fit to values that would be unacceptable by common standards. This provides some initial evidence for the validity of the secularism scale.
shows that a two-factor solution again achieves an acceptable fit across all four samples.5 In line with expectations, the estimated correlation between secularism and religiousness is negative and quite substantial, with values ranging from -0.56 to -0.70. To ascertain that political secularism and religiousness are indeed closely related but not identical, again all items were forced to load on a single common factor. Once more this results in a dramatic increase in χ2
and a decline in all measures of model fit to values that would be unacceptable by common standards.
Taken together, these findings already provide strong evidence for the convergent and discriminant validity of the political secularism scale. Ideally, they would be complemented by a full nomological validition in future research.
Recent research highlights the importance of political secularism as an individual attitude that affects political behaviour. While secularism as a political and philosophical concept has existed for centuries, and while there is a plethora of measures for individual religiousness, there is no established instrument for measuring political secularism at the micro-level.
In this research note, I introduce a cluster of items which tap into individual political secularism. Applying CFA, I show that these items form an internally consistent scale. Importantly, good internal consistency can be achieved with just four items. Such short scales are becoming increasingly popular, because they reduce both surveys costs and the burden on respondents.
The secularism scale also displays a moderate to strong positive correlation with permissive views on abortion and a strongly negative correlation with religiousness, yet it remains clearly distinguishable from both. This suggests that the scale is not just a reliable but also a valid instrument for measuring political secularism.
However, there are some limitations. First, the analyses presented in this note rely on surveys that were carried out in Germany only, albeit both in the territory of the former Federal Republic of Germany and the former German Democratic Republic. In future research, they should be complemented by comparative survey data to ascertain that measurements are invariant (Davidov 2009) across cultural, political and linguistic borders. Ideally, they would be included in large-scale project such as the European Social Survey.
Second, and relatedly, the item referencing the EU treaties should obviously be confined to the EU’s 27 member states. In an even broader context, it would have to be adapted, e.g. by instead referring to the national constitution.6
Third, while internal consistency is a central aspect of the scale’s reliability, it is unrelated to test-retest reliability (McCrae et al. 2011). Political secularism itself should be a relatively stable attitude. If the scale is reliable, measurements of secularism should be relatively stable, too, at least in the short and medium term. Ideally, future surveys should therefore administer the items repeatedly so that we may learn more about the scale’s properties.
But ultimately, the usefulness of the scale can only be gauged in applied research: including the scale in rich and substantively interesting models of political attitudes and behaviour will yield the true test of nomological validition.
7.1. Thresholds for the measurement models from section findings
7.2. Measurement model for attitudes on abortion
The ALLBUS 2012 included an item battery – first developed in the 1970s for the American General Social Survey (Bumpass 1997) — that taps into attitudes on abortion. Respondents were presented with a set of vignettes and asked whether abortion should be legal under these circumstances. Four of these questions were replicated in the PRDB study. They read as follows:
Should abortion be legal …
- … if there was a high probability that the baby would suffer from a serious birth defect (Birth Defect)
- … if the pregnancy posed a serious health risk to the mother (Health)
- … if the pregnancy was the result of rape (Rape)
- … if the woman wanted an abortion, whatever her reasons (Choice)
A slight complication arises from a split-half experiment in the 2012 ALLBUS that varied the situations. As a result, about 50 per cent of the respondents were not presented with the Rape item. Because their answers are missing completely at random (MCAR) and the estimator handles missing values transparently, this poses no special problem. In the same experimental group, respondents were given the opportunity to qualify their affirmative answer so that they had three options instead of two: “yes”, “yes, but only during the first three months of pregnancy”, and “no”. For the analysis, this was recoded as just “yes” versus “no”. Compared to the respondents who only had a binary choice in the first place, this does not change the marginal distribution.
7.3. Measurement model for religiousness
Religiousness was measured by three items from the ALLBUS study :
- Would you describe yourself as (Intensity: 1 extremely non-religious … 10 extremely religious)
- About how often do you pray (Prayer: 1 never, 2 less than several times a year, 3 several times a year, 4 between one and three times a month, 5 once a week, 6 more than once a week, 7 daily)
- As a rule, how often do you go to church – or to the mosque, synagogue or other house of God? (Church Attendance: 1 never, 2 less than several times a year, 3 several times a year, 4 between one and three times a month, 5 once a week, 6 more than once a week)
The PRDB study used the same wording but a slightly different answer format. For Church Attendance, there was an additional category “daily”. Because this was chosen by just 5 respondents, it was recoded to “more than once per week”, which brings the variable in line with the ALLBUS. For Intensity, the PRDB used a scale with the same labels for the endpoints, but eleven instead of ten steps. Because of this slightly greater spread, factor loadings for this item are not strictly comparable across time.
7.4. Convergent and discriminant validity: full models
7.4.1. Attitudes on abortion
A third item (“Do religious beliefs most influence your thinking about government and public affairs?”) is more a matter of self-perception than a statement about the desired role of religion in public life.
Personal communication with Olaf Müller, one of the former CREE project members.
The Debates item originally more specifically referred to “religious leaders” trying to influence public debate. Because the new wording is more general, it better captures political secularism. Other modifications are very minor.
Alpha can be misleading if the items are not tau-equivalent or if the structure of the underlying measurement model is not simple. In its original formulation, it is also not adequate if measurements are ordinal. Finally, striving for a particularly high value of alpha during scale development may lead to the inclusion of redundant indicators, because alpha is also a function of the number of items.
For the West German sample from 2012, the TLI is below the usual cut-off value of 0.95, but the other indices still suggest a reasonable fit.
The CREE project used this variation of the item in Russia.
Created: 2021-12-22 Wed 16:12