Before presenting and commenting on the empirical dimensions emerging from the analysis, some clarifications are needed. First, it must be underlined that what is presented in the following tables simply represents the frequency with which the six fundamental components of policymaking (and the related empirical dimensions) recur in the analysed sample, according to the conceptualisation framing of the theoretical papers and empirical evidence coming from the analytical papers. Thus, the data simply show how the specialised literature has analysed the use of knowledge from a subjectivist perspective, and thus, they should be understood as the output of the specific theoretical approach adopted in both the theoretical and analytical papers. These data should thus not be considered objective evidence regarding barriers and facilitators but rather an illustration of how the theoretically driven and empirically oriented literature treats the relevant components of policymaking. This means that frequency cannot be considered an indicator of relevance. This is particularly true if we consider that, for example, among the 183 empirical cases, 172 are qualitative case studies. At the same time, these data support some thoughts and comments about the different propensities of the six constitutive elements of policymaking to facilitate and constrain contextualised knowledge in policymaking.
It is interesting that both interests and institutional and processual arrangements are not as appreciated as barriers. Perhaps interests might have been underestimated (due to their close relationship to values/ideology/beliefs), but their low score in the analysed sample, together with institutional arrangements, raises the question of whether the role of knowledge and evidence depends less on institutions and self-interest and more on capacities, relationships and ideas.
It must be underlined that, also seen from a diachronic perspective, it emerges that values, ideologies and beliefs have been considered to be barriers since the first decade of the considered time span, although equated over time by actors’ relations and, to a lesser extent, policy capacities. Moreover, the policy capacity dimension gradually emerges as a facilitator over the examined timeframe (see Tables E1, E2, and E3 in Appendix I).
It is possible to obtain a more fine-grained view of this evidence by focusing on the occurrence of single factors that are the empirical ways through which the six general categories work in the reality of policymaking (see Table F in Appendix I). Our analysis discloses that in a context of high variety, there is a significant concentration of the operational dimensions of both barriers and facilitators. Analytical capacity is ranked first both among the barriers and the facilitators. Regarding the barriers, three of the first seven factors belong to the values/ideology/beliefs category and thus proved to be construed as challenging constraints in the reviewed literature. This also shows that the role of evidence and knowledge might be significantly bordered when issues are treated ideologically and become a matter of political conflict, divided values and ideational bias. Interestingly, among the first seven factors, there is one (nature of scientific inquiry) that belongs to the type of evidence category. On the facilitators side, four of the first seven factors belong to the actors’ relationships category, which is therefore conceptualised in the reviewed literature as the most promising or manipulable in improving the role of knowledge in policymaking. Moreover, the role of the individual capable of exercising the role of leader or broker does not appear to be conceptualised as a significant facilitator, which is surprising given the large body of literature on leadership.
The values/ideology/beliefs category ranks first as a barrier in most policy sectors and is absent in only a few sectors. At the same time, there are other policy fields with few selected papers (such as regulation, drug policy and sports policy) in which the prevalent barrier is ideational. Among the facilitators, there is no general pattern of prevalence. Policy capacities are ranked first in education and justice, while actors’ relationships are ranked first in climate and environment. It is interesting that a few sectors, such as energy, fisheries, gender, sports and transport, do not have any suggested facilitators, which may, however, be due to the low number of papers analysed.
Discussion
The data presented above are rich and could be discussed from different points of view. Here, we focus on those that are more relevant from a public policy perspective.
First, the field of knowledge utilisation is elusive and slippery, including in metareviews. It is relevant to emphasise that the potential results depend on the research questions and thus on the keyword search. We used keywords that would allow a wide-mesh sieve. Thus, we could include relevant items that could have been excluded with more specific targeting. In this way, it was possible to build a detailed map of how knowledge utilisation is treated in various streams of literature and thus to determine how much of this literature must be considered of partial, if not marginal, interest for policy scholars.
The different focus adopted by this metareview—based on the distinction between objective perspective (knowledge for policymaking) and subjective perspective (knowledge in policymaking) and the consequent choice to privilege the latter for deeper analysis—shows different results in terms of barriers and facilitators compared to those of previous studies as presented in Sect. 2.2, mostly belonging to the objective perspective and thus emphasising those factors that constrain or facilitate the use of knowledge (considered external to the policy process): clarity of evidence and proper channels for knowledge translation.
By adopting a subjective perspective, we have assumed that barriers and facilitators are not objective elements in policymaking but that the constitutive components of policymaking could constrain and facilitate the use of knowledge according to the context (and thus also to the researchers’ theoretical approach).
Being aware of these assumptions, it should be noted that, according to the picture emerging from the papers belonging to the subjective perspective, values/ideology/beliefs, problems regarding the relationships between actors and (low) policy capacities emerge as pivotal factors constraining the proper use of knowledge (see Table
6). With regard to the first mentioned barrier (values/ideology/beliefs), it was noted, for instance, that “[i]ndividuals and groups with contrasting belief systems interpret and use evidence differently—not just because they are actively pursuing particular policy results, but because of how existing beliefs, motivations, and values structure the cognitive processes through which evidence is understood and applied” (Parkhurst,
2016: 384). In terms of actors’ relations, some of the reviewed literature has detected difficulties in science communication to nonscientists. For instance, it has been pointed out that “[t]he rising number of involved actors in communication on the science-policy interface raises new challenges when speaking about quality: scholarly information has to be understood by politicians and the general public (who) do not necessarily engage in scholarly activities themselves” (Sokolovska et al.,
2019: 5). The difficulties in actors’ relations, as emerged from our metareview, are twofold and mainly caused by the differences between the academic and policymaking communities, as “academics have difficulty communicating research in an applicable manner, and policymakers, in turn, tend to focus on operational motivations” (Wilkinson et al.,
2017: 88). Finally, in terms of policy capacity, our reviewed literature shows that its low level is a key constraint to improving evidence-based policymaking, with an example of this being found in the short-term focus of the actors involved in policy analysis (Howlett,
2009).
Thus, according to our reviewed literature, ideational factors and a lack of trained administrators of appropriate organisations seem to matter more than the type of evidence available. The skills of policymakers are important, but they should be included in the more general capacities (individual, organisational, systemic) that characterise a specific policy setting, the government and its bureaucracy. Furthermore, when relations between actors are analysed, there is an emphasis not only on the interaction between scientists and policymakers but also on the relevance of other stakeholders, such as the community, specific groups of citizens and vested interests. For instance, in the case of water governance in the city of Ahmedabad, one of our reviewed items identified insufficient stakeholder involvement as a factor contributing to explaining the weak linkage between research and policy and, conversely, stakeholder involvement broadening as a factor that might facilitate “a long-term joint knowledge production process” (Aartsen et al.,
2018: 2455).
Finally, we find that financial issues do not stand out in the selected sample. This indicates that the problem of the impact of knowledge utilisation is not related to the existence of sufficient evidence (whose lack would justify the consideration of funding as a barrier) but to other factors.
Regarding facilitators, this metareview clearly provides a very different picture than the literature based on other reviews, which are very often based on an objective perspective (cf. Section 2). On the one hand, this work reinforces in a detailed way those perspectives that call for better governance of evidence (Parkhurst,
2017). In fact, what emerges is not only that in the concrete dynamics of policymaking, the triggers to improve the role of knowledge involve factors related to policy capacities and actors’ relations but also that some specific factors could matter more than others, such as analytical capacities, the direct involvement of citizens and local stakeholders and coproduction. The importance of analytical capacities in policymaking is supported by scholars who state that the policymaking process needs and produces knowledge and that during this process, some (analytical) questions must be answered, such as “Which aspects of the problem should be considered more important? What information is relevant and should be used? Who are the principal stakeholders? Who else is affected by the situation and the possible policies? Which resources are allocated, where how and when? What matters in terms of potential consequences?” (De Marchi et al.,
2016: 23). To answer these questions, policymakers need analytical capacities. These capacities are also key elements of co-production practices, which are, in turn, functional to the use of knowledge in policymaking. Our reviewed literature also outlines other elements needed for these practices to work well. Along with the development of capacity and social capital, these elements include the active participation of knowledge users (e.g. policy implementation actors such as clinicians) in the creation of knowledge, the equality of researchers and policy actors in the conduct and application of research, the reciprocity of benefits between researchers and policy actors, and the setup of infrastructures that incentivise coproduction relations (Heaton et al.,
2015: 1489). Finally, some of the literature we reviewed argues that public (citizens’) participation should be reconsidered, not only to solicit valuable information from key stakeholders but also to leverage participation itself as empirical evidence. This reviewed literature identifies practical rules for conducting meetings with citizens, such as recording them to avoid losing important information, as well as the benefits of reconsidering public engagement as research, which might stem from improving science communication to broad audiences and demonstrating to citizens that policy decisions are made in consideration of their input (Hall et al.,
2016). Again, however, such efforts to employ public engagement as evidence require adequate analytical capacities.
A significant element emerging from the analysis of barriers and facilitators is the asymmetric role of ideas/values and ideology: this necessary, and inescapable, component of policymaking acts more as a barrier than as a facilitator. It must be noted that values and beliefs can be only partially manipulated, as the Advocacy Coalition Framework convincingly underlines (Sabatier,
1993; Weible & Jenkins-Smith,
2016). Thus, this asymmetry could be justified exactly because values and ideas are less changeable or manipulable in policy dynamics than other dimensions such as actors’ relationships and policy capacities. However, it could be clearly observed that the main elements of policymaking do not act in isolation and thus that this asymmetry could depend on the theoretical/analytical perspective of the authors but also on the specific policy fields analysed or on the high level of politicisation of the treated case.
Thus, this asymmetric role of ideas/values and ideology cannot be taken for granted, but the reviewed literature gives this general signal that is reinforced by other evidence coming from the analysis. In fact, for example, from a focused analysis of our final sample of 224 papers, it seems that ideational maps that hamper the use of knowledge in policymaking—for instance, international assessment results in the field of education—could be overcome by means of enhanced policy capacity, such as in the form of an improvement in strategic leadership (De Lisle et al. 2014). At the same time, and more representatively in terms of the literature reviewed, divergences in values, priorities and beliefs can be surmounted by means of enhanced actor relations. For instance, the pragmatic “forma mentis” of many policymakers in the healthcare sector, who tend to look mainly at the knowledge that they find immediately usable—which can represent an obstacle to the absorption of knowledge that they do not perceive as such—can be overcome by means of improved relationships between actors, where researchers proactively learn more about policymakers’ priorities and seek to develop mutually beneficial relations with them (Wye et al.,
2015). The same happens when the priorities, ideas and/or values and knowledge systems of policymakers and researchers are not the same, as is sometimes the case in the field of environmental protection. This ideational difference is indeed surmountable, according to the reviewed literature, through preemptive behaviour on the part of researchers who take action to know in advance what these priorities are (Karam-Gemael et al.,
2018); by means of procedures and practices that unlike the search for consensus that is the prevalent orientation in some fora (e.g. the Intergovernmental Science–Policy Platform on Biodiversity and Ecosystem Services—IPBES) are able to accommodate pluralism, contestation and distinct perspectives and knowledge systems (Díaz-Reviriego et al.,
2019); by means of collaborative, resource-intensive co-learning arrangements between policymakers and scientists that are properly organised in terms of governance (Morgan,
2017); or even by attempting at co-production, which some of the reviewed literature presents as an important practical arrangement of the relationships between actors—although not as a panacea (Sylvester & Brooks,
2020).
This evidence invites us to take seriously the hypothesis that knowledge production should be appropriate and localised (Parkhurst,
2017) and that policy capacities could also be a pivotal resource for any kind of policy process (Wu et al.,
2018). Overall, the present metareview on this point indicates that to increase the likelihood of a more effective role of knowledge in policymaking, inclusive policymaking processes should be introduced, bureaucrats should be better trained and the organisation’s capacity to collect and analyse data should be increased. Again, this evidence may depend on the characteristics of the case studies analysed by the reviewed literature and thus should not be taken for granted. However, these findings indicate a path for further analysis and in-depth empirical research.
The third relevant point that emerges from our metareview is that three constitutive elements of policymaking (values/ideology/beliefs, actors’ relations and policy capacities) prevail as barriers and/or facilitators, while the other three categories (including the factors considered more relevant by the other sectoral reviews as summarised in Sect. 2.2) are residual or relevant in only a few policy fields. The limited number of papers found and scrutinised on these aspects indicates the small number of interested scholars working on them. The question is thus raised regarding why interests, institutional and processual arrangements and types of evidence emerge as less relevant in favouring or constraining the use of knowledge in policymaking. While the low relevance of the type of evidence in policy dynamics can be related to the fact that, according to a subjective perspective, it has a high chance of being an intrinsically weak component of policymaking due to the intensity and strength of the others, it is less easy to justify the low relevance of the other two components. Obviously, the usual disclaimer can be applied, i.e. the specific subjective perspective of the reviewed works: for example, it could be the case that the authors have not clearly distinguished the ideas and interests in their research or have focused more on the institutionalised actors’ interactions and less on the institutional arrangements in which these interactions happen. This could also depend on similar mistakes in coding the data. However, the evidence emerging in the review significantly indicates this difference among the six components of policymaking, and this is something that should be addressed by further research. If corroborated by further empirical investigation, this difference could represent a significant “discovery” with respect to the determinants of the use of knowledge in policymaking.
The fourth relevant point is why there are so many sectors in which there is insufficient analytical attention to the role of knowledge in policymaking. We counted 18 policy sectors that are clearly underrepresented in the sample, while constant attention was given over time to sectors such as environmental and health policy. Sectors such as labour, social work, public administration reform, transport, energy, higher education, urban development and migration emerge as uninteresting to scholars who are interested in the policy use of knowledge. This may be due to the keywords we used or the fact that the role of knowledge is less relevant in policy fields where it is more difficult to objectify it, which seems to be the case for health, the environment and climate, where the bulk of knowledge is apparently more homogeneous and internally more shared by scientists. Thus, this underrepresentation may be a consequence of the more disputable nature of the issues at stake and of the different theoretical approaches and empirical evidence coming from scholars. However, it must be emphasised that many scholars of public policy are more interested in the dynamics of policymaking, whereas the different types of knowledge utilisation are usually treated as one of the possible factors influencing policymaking. Among the various literature streams in public policy that pay attention to the role of knowledge in policymaking, the following can be noted:
-
The stream focusing on the ideational dimension of policymaking, thanks to which it is possible to grasp whether and how ideas can be constraints or facilitators (Swinkels,
2020);
-
The recent Policy Design perspective, which underlines how the technical capacities of government can make a difference in terms of the quality of policy design and analyses the factors that can constrain or facilitate it (Capano & Mukherjee,
2020; Howlett,
2019);
-
The Policy Capacity literature, which focuses on defining the different capacities that are necessary to produce good policies (Mukherjee et al.,
2021);
-
The Advocacy Coalition Framework, which emphasises that there are certain conditions under which deep beliefs can be overcome by evidence-based knowledge and that experts are not neutral (Jenkins-Smith et al.,
2018);
-
The stream that focuses on co-production and collaborative governance, which is fundamental to understanding how the involvement of stakeholders is essential for good governance and thus for an appropriate role of knowledge in policymaking (Ansell & Gash,
2008; Poocharoen & Ting,
2015).
The richness of these approaches in public policy, therefore, could justify the fact that so many policy fields lack consistent literature on knowledge utilisation: these approaches consider knowledge as an element of their designed policy setting, even if the role of knowledge is not directly enlightened as pivotal.
At the same time, this richness calls for a higher integration with studies specifically devoted to analysing knowledge utilisation in policymaking. In fact, on the one hand, these studies, such as the public policy streams of research listed above, share the common subjective perspective that assumes the complexity of policymaking as an inescapable feature and consequently that knowledge is not a phenomenon that can be considered external to the policy dynamics themselves. On the other hand, this metareview clearly shows that the three fundamental components of policymaking (values/ideology/beliefs, policy capacities, actors’ relations) that emerge as more relevant in constraining and/or facilitating knowledge in policymaking are exactly those on which there is a flourishing public policy literature. Greater integration would help better clarify the drivers and activated mechanisms through which this set of factors can constrain or facilitate a better use of knowledge and thus find and potentially design possible solutions to make knowledge more relevant in policymaking. In addition, overall, due to the emerging relevance of policy capacities and actors’ relations, this greater integration would help demonstrate ways to better design actors’ relations and improve policy capacities to eventually make the role of knowledge in policymaking more effective.