On-Site Citizenship Education: An Effective Way of Boosting Democratic Engagement and Reducing Inequalities Among Young People?

Citizenship education refers to efforts aimed at developing attitudes and behaviors that contribute to active, informed and engaged participation in a pluralistic democracy (Galston, 2001; Lawy & Biesta, 2006). It can be divided into four categories: formal instruction, open classroom climate, voluntary activities, and compulsory activities (e.g. Campbell, 2019; Dassonneville et al., 2012; Geboers et al., 2013). Formal instruction refers to citizenship related courses within the classroom context, such as civics or social studies. The open classroom climate refers to the atmosphere in and organization of the classroom, exemplified by active participation, student-driven learning, and interaction. Voluntary activities encompass student council membership, mock elections, or participation in a debating team. Voluntary activities are subjected to selection effects: particularly youngsters that are already politically engaged will participate in voluntary activities (Glanville, 1999; Hoskins & Janmaat, 2019).

On-site citizenship education falls within the fourth category, i.e. compulsory activities that usually take place outside the classroom context. In most Western European countries, governments provide on-site citizenship education (e.g. the UK Parliament Education and Engagement Service in the United Kingdom) or subsidize independent organizations to offer on-site citizenship programs (e.g. the organization ProDemos—House of Democracy and Rule of Law in the Netherlands). On-site citizenship programs usually include a visit to a government institution (such as the House of Representatives, country hall, court, prison, police station), interaction with government officials (e.g. Members of Parliament, city councilors, judges, police officers) or the reenactment of government institutions (e.g. debating and voting on a law, reenacting a lawsuit). It may also cover visiting an (interactive) democracy museum, or participating in informative quizzes and games about democracy and politics.

The aim of (on-site) citizenship education is twofold. First, it aims to enhance democratic engagement, that is, to transfer knowledge about democracy and its institutions, raise political interest and efficacy, stimulate political participation, and increase democratic values (e.g. Hoge, 2002; Langton & Jennings, 1968). Second, it aims to assist particularly adolescents from politically disadvantaged backgrounds to be democratically involved at the same level as those from politically advantaged backgrounds (e.g. Campbell, 2008; Neundorf et al., 2016).

However, it remains unclear whether on-site citizenship education actually contributes to stimulating democratic engagement and compensates for inequalities, as is assumed by both governments and civic education organizations. Remarkably, on-site citizenship education has remained understudied empirically.

On-site citizenship programs, such as the ones mentioned above, generally aim to influence democratic engagement rather than civic engagement (such as volunteering, see Henderson et al., 2007) or political engagement (such as voting, see Keating & Janmaat, 2016). Yet, there is no consensus in the literature on how to define or measure democratic engagement among adults (Ekman & Amnå, 2012; Keating & Janmaat, 2016), let alone among youngsters. Given that they are not yet in the position to formally participate in the democratic process due to their lack of voting rights,¹ many items that are regularly asked among adults are irrelevant for youngsters (Deimel et al., 2020).

The categorization of Ekman and Amnå (2012) is useful, as it does not only consider manifest political behavior, but also more latent, pre-political forms of engagement. This is a fruitful strategy, especially for research among youngsters, since there are many attitudes that are of great importance for future democratic engagement. Therefore, I define democratic engagement broadly, encompassing political knowledge, political interest, (internal and external) political efficacy and (intended) political participation (anticipated voting intention). Since on-site citizenship education defines the stimulation of democratic engagement as a core aim, I also include support for representative democracy.

There are multiple reasons to believe that on-site citizenship education effectively contributes to democratic engagement. First, the organizers of on-site citizenship programs design the content of the program in such a way that it boosts democratic engagement.² Second, the specific set-up of the programs is conducive to effective learning. On-site citizenship education is based on the idea of experience learning: seeing and experiencing the political arena in real life renders politics less abstract and is assumed to leave a longer lasting impression than reading from a text book (Dassonneville et al., 2012). On-site citizenship education also combines education and entertainment, which is a very effective way of learning (Rapeepisarn et al., 2006). The mere location of on-site citizenship programs outside the school creates an entertaining atmosphere. Third, youngsters generally participate in on-site citizenship education as part of their formal school curriculum. On-site citizenship education is therefore not troubled by selection effects in the way that voluntary activities are.

Yet, the high expectations of the effects of on-site citizenship education on democratic engagement need qualification. First of all, on-site citizenship programs generally only take one day, which is not very long compared to other forms of citizenship education. Their potential to boost democratic engagement is therefore limited. Second, it is likely that the aspects of democratic engagement listed above are not affected to the same extent by on-site citizenship education. Some aspects of democratic engagement, such as political knowledge, are more easily changed than others, such a political values, or participatory intent (Finkel & Ernst, 2005; Prior, 2010; Van Ingen & Van der Meer, 2016). In fact, the aspects of democratic engagement studied in this paper (i.e. political knowledge, political interest, political efficacy, intended political participation, and support for representative democracy) are to some extent causally related (Delli Carpini & Keeter, 1996; Hooghe and Dassonneville 2011; Van Ingen & Van der Meer, 2016). In line with this argument Finkel and Ernst (2005) find that visits to government institutions increase political knowledge, but do not affect political values. Similarly, Dassonneville et al., (2012) find no effect of government visits on political values and behavior, such as political interest, political efficacy, political trust and political participation. On the basis of these findings, I expect that on-site citizenship education affects political knowledge first, political attitudes second, and political behavior last. This causal order is reflected in the first hypothesis, which sorts the aspects of democratic engagement on the basis of expected effect sizes.

Youngsters are not blank slates when they participate in on-site citizenship education. The variation in political attitudes and behaviors that is observed among adults is mirrored among youngsters (Schulz et al., 2018). Already at a young age political knowledge, political interest, political efficacy, (intended) political participation, and support for democracy differ by gender, educational level, socioeconomic status, and migration background (e.g. Abendschön & Tausendpfund, 2017; Geboers et al., 2015; Schulz et al., 2018). These differences are particularly problematic when they reflect structural inequalities in society. Especially citizens with lower education, lower socioeconomic status, migration background, and women belong to minoritized groups. In many countries, they gained suffrage later, and are still in an underrepresented position in the political arena today (Bovens and Wille 2011; Schakel & Hakhverdian, 2018).

The literature on the effects of citizenship education finds that well-designed citizenship education in general compensates inequalities (e.g. Campbell, 2008, 2019; Campbell & Niemi, 2016; Deimel et al., 2020; Gainous & Martens, 2012; Hoskins et al., 2017; Langton & Jennings, 1968; Neundorf et al., 2016; Robinson, 2019). Compensation occurs when citizenship education has a greater impact on adolescents from disadvantaged backgrounds than on adolescents from advantaged backgrounds (Campbell, 2008; Neundorf et al., 2016), thereby narrowing the gap between the two groups. The main mechanism underlying compensation is the catch-up effect: adolescents with disadvantaged backgrounds gain more from citizenship education, because they lacked opportunities to develop democratic engagement at home.

Yet, the literature also points towards other ways in which citizenship education might affect pre-existing inequalities. Acceleration takes place when adolescents from advantaged backgrounds benefit more than youngsters from disadvantaged backgrounds from on-site citizenship programs, while reproduction occurs when on-site citizenship education does not have a differential effect on adolescents from advantaged and disadvantaged backgrounds. Support for these differential effects in the literature is mixed. While Neundorf et al., (2016) and Deimel et al., (2020) find no evidence of acceleration, there is at least one study that does show an accelerating pattern: Hooghe and Dassonneville (2011) observe that students that know more about politics learn more from group projects in class. Deimel et al., (2020) also investigate whether reproduction takes place, finding that students’ socioeconomic background does not affect the positive relation between an open classroom climate and intended voting.

On-site citizenship education aims to compensate inequalities in democratic engagement between advantaged and disadvantaged groups by adjusting programs to specific groups, instead of a one-size fits all approach. For example, students with a lower educational level participate in a different program than those with a higher educational level. However, whether on-site citizenship education indeed succeeds in compensating inequalities and preventing acceleration in democratic engagement is an empirical question. All in all, the second hypothesis is:

The Netherlands has assigned the legal task to develop citizenship competences to schools. Yet, the definition of this task is procedural rather than substantive. Schools can define citizenship to match the (pedagogical or religious) character of their school. Consequently, there is a large variety in the ways that schools organize citizenship education. Yet, many secondary schools of different backgrounds participate in the main Dutch on-site citizenship program. This program is provided by ProDemos—House of Democracy and Rule of Law (an organization that is subsidized to a large extent by the Dutch government), with the explicit aim to stimulate democratic engagement and compensate inequalities in democratic engagement. The on-site citizenship program of ProDemos is representative of many other such programs in scope and activities. The program takes one day and adolescents generally participate in the program as part of their formal school curriculum. In 2018, more than 99.000 adolescents participated in the program (ProDemos, 2018). Part of the program is located in and around parliament: here, students visit the House of Representatives and/or Senate, interact with a Member of Parliament (when possible) and do a politics-related digitally-assisted scavenger hunt.³ The second main part of the education program takes place in an interactive democracy museum near parliament that is designed specifically for the on-site citizenship program. Here, students participate in interactive instruction (e.g. reenactment of parliament, debate and discussion, quizzes, escape room).

In the literature on the effect of citizenship education, most studies are based on observational data (Campbell, 2019). Be it cross-sectional or longitudinal, usually they rely on questions on past exposure to citizenship education, without a pre-test measuring democratic engagement before this exposure (e.g. Dassonneville et al., 2012; Finkel & Ernst, 2005). Moreover, due to the lack of random assignment they have to make strong assumptions about the comparability of students in their study. To enhance my causal claims, I employ a quasi-experimental panel design. I selected all schools that participate in the main Dutch on-site citizenship program with a random selection of classes at the beginning of the school year (treatment group) and with a random selection of classes at the end of the year (control group). To test whether students who took part in the program were affected differently from those that did not (yet) do so, this study relies on Difference-In-Difference (DiD) estimators. This method compares the differences between the pre- and post-test in the treatment group to the differences between the pre- and post-test in the control group.

In order to ensure that potential effects are truly attributable to the on-site citizenship program, I take great effort in ruling out alternative explanations. First, I minimize selection effects at the school, class and student level. Because schools that choose to include on-site citizenship programs in their curriculum likely differ from schools that do not choose to do so, I rely on control groups within the same school. I eliminate selection at the class level by only including schools that participate in on-site citizenship programs with all their classes. Selection effects at the student level are minimized by selecting schools in which participation in the on-site citizenship program is not voluntary, but part of the school’s formal curriculum.

Second, to rule out that any patterns are artefacts of pre-test sensitizing or period effects, I extended the data collection to a Solomon four-group design (Solomon, 1949). A pre-test may influence participants’ sensitivity to the treatment, for example by causing participants to contemplate a specific issue. I therefore add a control group that does not fill in a pre-test. Participants may also be subject to intervening influences at the time of the post-test. For example, if there is an election campaign going on at the time of the posttest, one can wrongly attribute an observed rise in political interest to the intervention. I therefore include a control group that takes part at the time of the post-test only.

Finally, I assess whether the possible effects of participation in on-site citizenship programs last over time by adding an additional wave. In total, I carried out four waves among the same students over the course of the school year 2017–2018. The control groups did not participate in the on-site citizenship program during the time of the study, but only later after T4. The full design is presented in Table 1.⁴

In the school year 2017–2018 three schools met the demands that I defined above and were hence selected for this study. In the Netherlands there are three main educational tracks: pre-vocational education (preparing for vocational education), senior general education (preparing for university of applied sciences) and pre-academic education (preparing for university). In this study, two schools offer pre-vocational education and participated with all third grades (equal to ninth grade in the U.S. system; 15 years old). One school combines senior general and pre-academic education and participated with all fourth grades (equal to tenth grade in the U.S. system; 16 years old). The schools are comparable in the sense that they are all relatively large public schools (600–1500 students) based in urban areas.

Within these three schools, I randomly allocated the total amount of thirty classes to the treatment group or one of the control groups, while taking into account the planning of ProDemos.⁵ In total 585 students (x̄_T4 = 14.95 years, s.d. = 0.76) received a number of paper-and-pencil surveys during class, ranging from one to four times.⁶ The survey took approximately 30 min. The first wave took place 2 to 31 (x̄ = 7) days before participating in the on-site citizenship program, the second wave 15 to 48 (x̄ = 30) days after participation, the third wave 81 to 157 (x̄ = 129) days after participation and the fourth wave 112 to 249 (x̄ = 173) days after participation. This resulted in 2340 observations, nested in 585 students, in thirty classes and in three schools.

The choice for the dependent variables is driven by the range of outcomes that most on-site citizenship education programs aim to affect, that is, democratic engagement. As argued in the theory section, I define democratic engagement broadly, encompassing political knowledge, political interest, (internal and external) political efficacy and (intended) political participation (voting intention). Since on-site citizenship education aims to stimulate democratic engagement, and not solely political engagement, I also include support for representative democracy. The descriptive statistics for the dependent variables are presented in Table 2.

Political knowledge was measured by students’ sum score of four multiple-choice questions (designed for this study): “Do you know how many seats there are in the Houses of Parliament?”, “Do you know how many seats there are in the Senate?”, “Do you know who can propose laws?” and “Do you know what is contained in Article 1 of the Constitution?” For each good answer, students could earn 1 point. Since there were 5 good answers,⁷ the maximum number of points is 5. ‘I don’t know’ answers get 0 points.

Political efficacy, the feeling that one can influence politics, has both an internal and external component (Balch, 1974). Internal political efficacy refers to confidence in one’s own competence to understand and effectively participate in politics. It was measured by agreement with two items on a 5-point Likert scale: “I don’t understand much about politics” (reverse coded, so that a higher score refers to a greater feeling of understanding) and “I understand more about politics than most people my age” (Cronbach's α = 0.64).⁸External political efficacy refers to beliefs about the governments’ responsiveness to citizens’ needs and demands. It was measured by means of a three-item scale: “I think politicians don’t care much about what people like me think”, “People like me have no influence on what happens in politics” and “Politicians are not interested in my opinion” (Cronbach's α = 0.76). In all cases items were measured on a 5-point scale and reverse coded, so that a higher score refers to a greater feeling of external political efficacy.

Voting intention was measured by a single item: “Do you intent to vote when you are 18 years old?” (1 = yes, 0 = no). As discussed in the theory section, given that youngsters are not yet allowed to vote, it is not possible to assess their actual voting behavior. Voting intention is a good and frequently used alternative to measure formal political participation among this age group (Deimel et al., 2020; Glasford, 2008).

To test the second hypothesis whether on-site citizenship education has any differential effects, I include the main sources of structural inequalities in society. That is, gender (ref: boy), educational level (pre-vocational education—senior general and pre-academic secondary education), socioeconomic status as indicated by the highest education of students’ parents (did not attend education, did not finish primary education, did not finish secondary education, did finish secondary education, did vocational training, did higher professional or university education) and the number of books at home (no books, 1–10, 11–50, 51–100, 101–200, 201–500, more than 500 books),⁹ and migration background (no migration background, first generation migrant, one parent born abroad, both parents born abroad).

As control variables, I include age of the respondent (in years) and dummies for the school they are in.

The first hypothesis states that on-site citizenship education has a positive effect on democratic engagement. This hypothesis is tested by means of Difference-in-Difference (DiD) estimation, which compares differences in the pre- and post-test of the treatment group to differences in the pre- and post-test of the control group. The results are presented in Table 3.

What stands out from Table 3 is the high number of non-effects. The DiD estimators of political interest, internal and external political efficacy, voting intention and support for democracy are not only insignificant, they are also substantially small. This implies that the main Dutch on-site citizenship program does not have an effect on political interest, internal and external political efficacy, voting intention and support for democracy.

However, the DiD estimator of political knowledge is significant, indicating that there are differences in the growth trajectories of the students that did and did not yet participate in the on-site citizenship programs. Therefore, this finding is scrutinized further.

First, Fig. 1 shows that students that participated in the on-site citizenship program (treatment group) experience a strong increase in their political knowledge between time 1 (before participation) and time 2 (after participation) (b = 0.856, p = 0.000), while the political knowledge of students that did not participate in the program (control group) remains the same (b = 0.053, p = 0.367). Students that participated in the on-site citizenship program have a 0.809 higher increase in their political knowledge score compared to students that did not participate. This means that students almost score one point higher on the knowledge scale due to their participation in the on-site citizenship program.

To rule out that this effect is caused by pre-test sensitization, the treatment and control group are compared to a group of students that did participate in the on-site citizenship program, but did not receive a pre-test (pre-test group). The increase in political knowledge is not caused by taking the pre-test, when the difference between the pre-test sensitization control group and treatment group is not significant, but the difference with the ‘regular’ control group is significant. In line with expectation, the results (see Table 4) show no significant difference between participants with and without a pre-test at T2; yet they reveal significant differences between participants without a pre-test and non-participants. This suggests that the increase in political knowledge is not affected by taking the pre-test.

In addition, the effect of participation in the on-site citizenship program lasts over time, given that the knowledge score of the treatment group is still higher at time 3 (b = 0.822, p = 0.000) and 4 (b = 0.806, p = 0.000) than before their participation.¹⁰ However, although the political knowledge of the control group does not increase from T1 to T2 (b = 0.053, p = 0.367), it does increase from T2 to T3 (b = 0.683, p = 0.000) and remains at the same level after that (T3-T4: b = -0.045, p = 0.381). Also, the DiD estimators of T1-T3 (b = -0.077, p = 0.350) and T1-T4 (b = -0.119, p = 0.268) are not significant, implying that there is no difference in the growth trajectories of the treatment and control group between T1 and T3/T4. This suggests that students’ political knowledge increases over the course of a schoolyear anyway, but that students that participate in the on-site citizenship program simply gain political knowledge earlier.

The second hypothesis is that on-site citizenship education compensates structural inequalities in youngsters’ democratic engagement. The first step is to examine what kind of inequalities in democratic engagement are observable among youngsters. The descriptive statistics of democratic engagement among the various groups are presented in Table 5. Some differences are significant. Girls have less internal political efficacy and are more supportive of democracy than boys. Moreover, higher educated students¹¹ have more political knowledge, political interest, internal political efficacy and support for democracy than lower educated students. Furthermore, students with higher educated parents¹² have more internal political efficacy than those with lower educated parents and students with more books at home¹³ are more politically interested than those with less books at home. Lastly, those with a migration background have less external political efficacy, and are less likely to vote than those without a migration background.

In a next step, I investigate how various subgroups are affected by the on-site program by means of three-way interactions. Since my primary interest are the outcomes rather than the underlying pathways, the three-way interactions (Table 6) and resulting differential effects (Table 7) are modeled in separate models and without covariates.

Again, the large number of non-effects stands out. For all aspects of democratic engagement, there is not a single significant three-way interaction with regard to the number of books at home and migration background. This means that the pre-existing inequalities based on the number of books at home (i.e. political interest) and migration background (i.e. external political efficacy and voting intention) are reproduced by on-site citizenship education. Of course, there is also reproduction in the aspects of democratic engagement in which there were no inequalities beforehand. There are, however, some significant three-way interactions for gender, educational level and parental education.

There is no differential effect on the aspects of democratic engagement in which there were no gender based inequalities at time 1 (i.e. political knowledge, political interest, internal political efficacy¹⁴ and voting intention). However, given the significant three-way interactions, the on-site citizenship program does have a differential effect on boys’ and girls’ support for democracy and external political efficacy. The results point to a slight acceleration and creation of inequalities due to the on-site citizenship program. For the exact pattern, I turn to differences in the marginal effects.

First, participation in the on-site citizenship program accelerates differences based on gender with regard to support for democracy (see Fig. 2). In the control group (dy/dx = 0.183, p = 0.080) and the treatment group (dy/dx = 0.176, p = 0.063), there is a marginal significant difference between boys and girls in support for democracy at time 1. At time 2, the difference in the control group disappears (dy/dx = -0.030, p = 0.421), while remaining in the treatment group (dy/dx = 0.286, p = 0.011). This implies that participation in the on-site citizenship program accelerates the difference in support for democracy.

Second, participation in the on-site citizenship program creates differences based on gender with regard to external political efficacy (see Fig. 3). There are no differences in external political efficacy between boys and girls in the control group at time 1 (dy/dx = -0.142, p = 0.165) and time 2 (dy/dx = 0.150, p = 0.192). Also in the treatment group there are no differences between boys and girls before their participation in the on-site citizenship program (dy/dx = 0.005, p = 0.486). However, after their participation, the external political efficacy of girls seems to be marginally higher than that of boys (dy/dx = 0.192, p = 0.085), thereby creating an inequality that was not there before.

There is no differential effect on the aspects of democratic engagement in which there were no inequalities based on educational level before participation in the on-site citizenship program (i.e. external political efficacy and voting intention). Since participation in the on-site citizenship program has no differential effect on the pre-existing inequalities in political interest and support for democracy with regard to educational level, these inequalities are reproduced. In addition, given the significant three-way interactions, participation in the on-site citizenship program compensates inequalities based on educational level with regard to internal political efficacy, but also accelerates inequalities based on educational level with regard to political knowledge. Again, I turn to the differences in the marginal effects for the exact pattern.

First, in the control group there are no differences with regard to internal political efficacy between higher and lower educated students at time 1 (dy/dx = 0.199, p = 0.138) and time 2 (dy/dx = 0.282, p = 0.072) (see Fig. 4). In the treatment group, higher educated students have more internal political efficacy than lower educated students before they participate in the on-site citizenship program (dy/dx = 0.634, p = 0.000). However, lower educated students show a steeper growth curve, which decreases the difference between lower and higher educated students after participation (dy/dx = 0.349, p = 0.010) and leads to a compensation effect.

With regard to political knowledge, there are no differences between higher and lower educated students in the control group, both at T1 (dy/dx = 0.129, p = 0.291) and T2 (dy/dx = -0.090, p = 0.363) (see Fig. 5). In the treatment group, higher educated students are already more politically knowledgeable than lower educated students before they participate in the on-site citizenship program (dy/dx = 0.851, p = 0.000), but this difference increases after their participation (dy/dx = 1.411, p = 0.000). Higher educated students show a steeper growth curve than lower educated students, suggesting an acceleration effect.

There is no differential effect on the aspects of democratic engagement in which there were no inequalities based on parents’ educational level at time 1 (i.e. political knowledge, political interest, external political efficacy, voting intention and support for democracy). However, given the significant three-way interaction, the on-site citizenship program does have a differential effect on the internal political efficacy of youngsters with lower and higher educated parents. The results point to an acceleration of inequalities due to the on-site citizenship program. Again, I turn to differences in the marginal effects for the exact pattern.

First, in the control group there are no substantial differences with regard to internal political efficacy between students with higher and lower educated parents at time 1 (dy/dx = 0.280, p = 0.050) and time 2 (dy/dx = -0.126, p = 0.257) (see Fig. 6). In the treatment group, students with higher educated parents have more internal political efficacy than those with lower educated parents before they participate in the on-site citizenship program (dy/dx = 0.559, p = 0.000). Students with higher educated parents also show a steeper growth curve, which increases the difference between students with lower and higher educated parents after participation (dy/dx = 0.632, p = 0.000) and leads to an acceleration effect.