Measuring Non-electoral Political Participation: Bi-factor Model as a Tool to Extract Dimensions

While conducting research on political participation, scholars oftentimes face the decision of how many dimensions to extract. This is a crucial step regardless of whether we want to model political participation as a latent variable in a structural equation model or to build a composite score and run a regression. Even though we could find authors who argue for the unidimensional structure of political participation (e.g. van Deth 1986), various participatory forms have been usually divided into two or more dimensions, also known as modes (Barnes and Kaase 1979; Inglehart and Catterberg 2002; Teorell et al. 2007; Marien et al. 2010). The preference for more than one dimension has been backed with numerous empirical tests. Yet, once we take a closer look at those testing procedures, we may have doubts if the alleged multiple dimensions are really mirrored by the data. If they are not, conclusions from numerous studies may need to be reconsidered.

To investigate the number of dimensions of political participation, researchers commonly use two methods: principal component analysis (PCA) and factor analysis. In the case of the PCA, they usually run a model and they check the number of eigenvalues greater than 1.0 (e.g. Teorell et al. 2007; Theocharis and van Deth 2018). But as shown by Russell (2002) and van der Eijk and Rose (2015), this method often leads to solutions which suffer from overdimensionalisation, i.e. extracting too many factors, and, therefore, should not be used. As to the factor analysis, the procedure frequently consists in checking the model fit (e.g. Gibson and Cantijoch 2013; Ohme et al. 2018; Talò and Mannarini 2015). However, in the psychometric literature, where seeking dimensionality is a core enterprise, determining dimensionality via a test of model fit has been criticized for being oversimplistic (e.g. Bentler 2009; Berge and Sočan 2004) and for wrongly conflating model fit with model validity (Reise et al. 2018). The problem is that “every set of responses by real individuals is multidimensional to some degree” (Harrison 1986). In practice, whenever we have more than three items, perfect unidimensionality is impossible (Berge and Sočan 2004) and the one-factor model can rarely describe data with a reasonably large number of items (Bentler 2009). Still, this does not automatically imply that the construct we are interested in is multidimensional. To treat a construct as having two or more distinct dimensions, these should uniquely explain variation in item responses. However, even when a multidimensional model fits better than a model with a single dimension, the factors can offer a very little unique (unshared) contribution (Gignac and Kretzschmar 2017). If we ignore this and model more than one dimension, we may end up explaining largely the measurement error and not the variance attributable to each dimension. Consequently, any estimate of the association between that dimension and an external variable will be attenuated or inflated and so will the measures of uncertainty of that estimate (Wiernik et al. 2015).

To find out how many dimensions of political participation to extract, I propose to use a bi-factor modeling strategy along with factor strength, internal consistency reliability, and construct replicability indices: Explained Common Variance, omega coefficients, and the H index. Bi-factor model allows us to inspect if we deal with a truly multidimensional construct or a construct that is essentially unidimensional, i.e. having one major latent dimension and a couple of minor dimensions that can be ignored (Strout 1990). With a bi-factor model, we can disentangle various sources of variance, which otherwise often remain indistinguishable. I apply this strategy to the European Social Survey, a dataset frequently employed by the students of political participation.

The results show that once we take into account the common variance among the indicators, additional dimensions of political participation explain very little variance in most of the countries and are likely to be unreplicable across different samples. If we were to build separate subscales for each dimension and score individuals on them, the scores would be severely unreliable as most of the variance in the unit-weighted composite scores is due to the general trait of political participation. Overall, additional dimensions of political participation do not acquire sufficient specificity nor stability to represent empirically identifiable constructs in most cases. They are virtually impossible to interpret and when regressed on external variables (e.g., in structural equation modeling context), the coefficient will be unreliable at least.

These findings have direct consequences for how to model political participation irrespective of whether this is done in a structural equation model framework or in the form of composite scores and regression analysis.

When trying to measure political participation, researchers often are not interested in just one form, like signing petitions, but they want to reveal a broader picture of how citizens participate in politics. Among others, they want to know whether young people participate differently than adults (García-Albacete 2014) or if political trust impacts the choice of forms of participation (Hooghe and Marien 2013). To capture the whole phenomenon of political participation, scholars usually treat political participation as a continuous latent variable and the multiple participatory forms as manifestations of this latent quantity (e.g. Marsh 1974; Verba et al. 1978; Vráblíková 2014). The relation between an observable indicator, \({x}_{i}\), and the unobserved quantity, \({\xi }_{i}\), can be expressed with the following equation:where \({\delta }_{i}\) denotes a measurement error, \(i\) refers to the unit of observation. Latent formulation of political participation allows us to incorporate into the analysis the idea that survey measures do not resemble the real-world behavior of individuals perfectly and that these measures come with an error. Additionally, and more importantly, the formulation reminds us that an indicator, \(x\), is not the latent variable of interest, \(\xi\) (Jackman, 2008). In the case of political participation, a reported signing of a petition (\(x\)), for instance, even if accurately reported, is not political participation (\(\xi\)); it is related to political participation and the Eq. (1) shows this relationship. The latent construct of political participation is comprised of multiple observable indicators and not just one. If we believe that political participation is made up of many different acts and, still, we equal it to one indicator, we make a translation error, i.e. we fail to translate the theoretical concept into the measurement specification (Fariss et al. 2020; Trochim andDonnelly 2007).

The core issue about conceiving of political participation as a latent variable is the question of the number of dimensions. Definitions of political participation are agnostic as to whether there are single or multiple dimensions. Most of the efforts have been devoted to imposing clear boundaries on what political participation is and what is not (Almond and Verba 1963; Conge 1988; Fox 2014; Pattie et al. 2004; Verba and Nie 1972). For example, in the recent work by van Deth (2014), political participation is defined as an activity or action that is carried out voluntarily by non-professional citizens, and which is located within the realm of politics, government, or the state. Alternatively, the action should be targeted at a subject from one of those realms or aimed at solving a community problem. When even the target is unclear, one should look at the political context or the motives of participants (van Deth 2014; van Deth and Theocharis 2018). By doing so, we can capture all forms of political participation, both traditional like voting and new ones like online participation.

In practice, these various forms are often divided into two or more dimensions, also known as modes of participation. There is an underlying assumption that people choose forms from either of those dimensions and rarely people’s participatory repertoires consist of acts that stem from different modes. In other words, people tend to participate in forms from either of the modes. This view on political participation as a multidimensional construct has been popularized by numerous authors. They differ as to the number of dimensions and the criteria according to which we should divide the forms (e.g., Barnes and Kaase 1979; Inglehart and Catterberg 2002; Teorell et al. 2007; Marien et al. 2010; Hooghe and Marien 2013). In this article, I concentrate on the division on institutionalized and non-institutionalized forms which is prevalent in the empirical research and which builds on the proposition of Barnes and Kaase (1979). Institutionalized forms are directly related to the institutional process. These forms are usually regulated by the members of the political elite, primarily by political parties. By contrast, non-institutionalized forms are conceived of as having no direct link to the electoral process or to the functioning of the political institutions. They are used predominantly by actors who are not members of the political elite, who want to manifest their dissatisfaction with the elite’s actions or to impact the decision process. The impact of non-institutionalized forms on the political system is usually indirect (Hooghe and Marien 2013; Marien et al. 2010).

However, some scholars indicate that talking about multiple dimensions is essentially dated. Today, participatory repertoires of citizens are made up of multiple diverse forms, like taking part in demonstrations, consumerism, and petitioning (Norris 2007). Having multiple forms at their disposal, people opt for acts which allow them to express themselves, which match their interests and resources (Bang 2009; van Deth and Theocharis 2018). Politically skilled and policy-oriented citizens will choose whatever means appropriate to influence policies they care about (Dalton 2008). They may combine different forms, like voting and demonstrating, which is not a rare practice nowadays (Norris et al. 2005; van Aelst and Walgrave 2001). Given all this, it makes sense to treat political participation as an essentially unidimensional latent construct.

To solve the controversy around the number of dimensions of political participation, I analyze the battery of participatory items of the 8th edition of the European Social Survey (ESS) (2016) for each country. I use ESS for two reasons. First, ESS is frequently used by scholars researching political participation (e.g., Bäck and Christensen 2016; Kostelka 2014). Second, it includes numerous participatory items, representing both institutionalized and non-institutionalized forms.

From the whole battery of items, I utilize eight items measuring political participation: contacting politicians or government, working for a political party or action group, working for another organization or association, badging, signing petitions, taking part in lawful demonstrations, boycotting products, and posting or sharing things online. Each respondent was asked whether he or she had participated in each form in the past 12 months. Thanks to this, the variation in the levels of political activity across individuals only due to different timespans is greatly limited. The ESS battery of political participation items was intended to measure a variety of political participation acts. The items were not meant to conform to any particular conceptualization of political participation and, therefore, to form any particular scale (Thomassen 2001). Descriptive statistics for the items can be found in Table 3 in the "Appendix".

For the purpose of this analysis, I exclude voting. Many studies have shown a special character of voting: it is a very frequent activity and therefore an exception on other indicators (Parry et al. 1992; Marien et al. 2010). The difference in prevalence between voting and other forms of participation often leads to problems with estimating measurement models. For these reasons, voting is often left-out from testing political participation scales in practice (e.g. Hooghe and Marien 2013; Theocharis and van Deth 2018; Vráblíková 2014).

The items which measure political participation were recoded into dichotomous variables, with values 1 “participation” and 0 “no participation”. Refusals, “No answer”, and “Don’t know” answers were recoded as missing values and were not used in the analysis. In addition, all interviewees-minors were excluded from the analysis as they could not participate in some participatory forms for legal reasons.

Table 2 shows the indices for the bi-factor model of political participation. In 15 out of 24 countries, the ECV value for the general trait of political participation is higher than 0.7, meaning that in those countries the general factor of political participation accounts for more than 70% of common variance. This is a strong indication that in these countries political participation is essentially unidimensional. Also, we should be able to fit a simpler unidimensional model for these countries without much bias introduced. In the other countries, the general factor explains still more than half of the common variance, except Finland, where the ECV is below 0.5. The amount of common variance explained by the specific factors is small. On average, the factor representing institutionalized mode explains 13% of the common variance and the factor representing non-institutionalized mode—15%. Only in Finland and Iceland the ECV for a specific mode (non-institutionalized mode) is noticeably higher, exceeding 30%.

H values show that the general trait of political participation is not only strong but also reliable and well-defined. In almost every country this index is higher than 0.8, and only in Finland it is below that value but still acceptable. By contrast, H values for specific modes of political participation are all below 0.7, with the exception of the non-institutionalized mode in Finland. It means that they are poorly-defined by the set of indicators and that the group factors representing the modes are likely to lack stability across samples. Overall, ECV and H values suggest that there exists one strong general trait of political participation, and two weak and unreliable group factors—specific modes of political participation. Based on this, we can say that common variance among all the indicators cannot be ignored if political participation were to be modeled in the structural equation modeling framework and that we should not interpret the structural coefficients between an external variable and specific modes of participation.

As to the omega ratios, the results are rather unequivocal. In 20 out of 23 countries, the ratio of omega hierarchical to omega is above 0.8. It means that more than 80% of reliable unit-weighted total score variance is due to the general factor. Subscales in all the countries are largely confounded by the general factor. By using the rule of thumb, i.e. assuming that at least 50% of reliable variance in unit-weighted subscale scores is needed to score individuals on the subscales (Canivez 2015), we could do it in just five countries: in Germany, Norway, and Slovenia on the subscale “institutionalized mode”; in Finland and Iceland on the subscale “non-institutionalized mode”. Still, this would be risky and it is up to the researcher if he or she wants to take that risk.

Hence, in the case of political participation unit-weighted subscale scores generally do not provide meaningful and reliable information about the modes that is unique from the general trait of political participation. Given the results, building composite scores for subscales cannot be justified, with five possible exceptions mentioned earlier. If the subscale scores, e.g. as composite scores, were to be correlated with an external variable, the correlation will be largely inflated if the external variable predicts the general trait of political participation and a specific mode in the same direction (i.e., enhancing conflation) or attenuated if the external variable predicts in opposite directions (i.e., suppressive conflation) (Wiernik et al. 2015). Inflation and attenuation would impact the correlation between the total score and an external variable too, but the impact would be significantly lower since the general trait of political participation accounts for most of the variance.

Lastly, as a robustness check, I repeated the procedure on the 9th edition of the ESS (2018). The analysis corroborates the findings based on the 8th edition. Details can be found in the Online Resource 2.

Previous research relied strongly on procedures which produced a false impression that we can and should model political participation as a multidimensional construct, composed of separate modes, with the common variance being ignored. Those procedures usually involved a simple inspection of the model fit (e.g. Talò and Mannarini 2015) or other shortcuts in deciding about the optimal number of dimensions like using PCA with the Kaiser criterion (e.g. Theocharis and van Deth, 2018). But as argue a growing number of methodologists (Bentler 2009; Berge and Sočan 2004; Reise et al. 2018; van der Eijk and Rose 2015), these often employed methods suffer from some flaws which may lead to erroneous conclusions about the number of dimensions that we should extract. To model political participation as a multidimensional construct, these dimensions should offer some unique (unshared) contribution in explaining the variance in item responses (Gignac and Kretzschmar, 2017).

If we were to decide how to build unit-weighted scores of political participation, i.e. by using the total score (political participation as a whole), subscale scores (institutionalized and non-institutionalized modes), or both, the results are rather unequivocal. In 20 out of 23 countries, 80% of reliable total score variance was due to the general factor. Subscales in all the countries were largely confounded by the general factor and only in five instances we could try to score individuals on the subscales. Therefore, if modeled as a unit-weighted composite score, political participation should be treated as a whole in most countries. Otherwise, that is by using subscales for each mode, we would incorporate plenty of measurement error and the observed score correlations will be largely inflated or attenuated as estimates of relations between the specific modes and external variables. Total scores are affected significantly less by the attenuation and inflation.

If we were to decide how to model political participation in the structural equation modeling framework, the most optimal solution is to use a bi-factor model. However, only the general trait of political participation from this model is reliable and well-defined in most of the countries. By contrast, the specific factors representing institutionalized and non-institutionalized modes of participation are poorly-defined and unreliable, and, therefore, very likely not replicable across samples. With the bi-factor model, we can regress the general factor on external variables and account for the noise caused by the specific factors, which may distort the structural coefficients with external variables. Any structural coefficient between a specific factor and an external variable would be unreliable at least and cannot be interpreted.

Overall, the analysis shows that only the general trait of political participation is interpretable in most countries. The specific factors representing institutionalized and non-institutionalized modes of participation, generally, do not acquire sufficient specificity nor stability to represent empirically identifiable constructs. This makes the interpretability of the specific modes virtually impossible. The results lend strong evidence that in most countries political participation is an essentially unidimensional construct, with negligible group factors representing specific modes. The multidimensionality of political participation cannot be backed by sufficient empirical evidence and the multiple dimensions seem to be methodological artifacts.

Future research should concentrate on testing political participation with larger sets of items, and using other theoretical divisions of forms of political participation. Moreover, researchers should systematically investigate the structure of political participation in time—an issue that was beyond the scope of this article.