Methodological Challenges in Sustainability Science: A Call for Method Plurality, Procedural Rigor and Longitudinal Research

Sustainability science has gained increased momentum for dealing with wicked problems [1, 2]. It acknowledges that cause and effects of complex problems are difficult to unravel, judgments over potential solutions are value laden, and tackling the wickedness demands integration of various knowledge domains [3]. However, often such observations remain rhetorical arguments and little attention is given to the implications this has for the practices of a science dealing with wicked problems. This contribution aims to increase clarity over the methodological challenges that sustainability science needs to address to effectively move forward in dealing with wicked problems.

The methodological challenges of sustainability science have diverse rooting. Some scholars have emphasized shortcomings in developing a new community of practices [4] that share an agreed upon set of principles [5] for generating knowledge on the interactions between nature and society. Jerneck et al. [6] noticed the tension between natural and social science. While natural science strives for scientific portrayal with little focus on matters such as justice and power, social science seeks to understand knowledge as contextually constructed, often with insufficient comprehension of natural science research [7]. In fact, much like other disciplines (e.g., health science, agricultural science) the common ground for sustainability science does not emerge from shared methods or understandings but from the problems research addresses and the purpose to developed solutions [8, 9]. We hope to contribute to this debate through discussing the methodological challenges of sustainability science through the problems it addresses. Accordingly, we focus on the nature of these problems as urgent and multifaceted rather than on the specific domains in which they occur (e.g., climate change, biodiversity loss, social injustice, etc.) and guide our discussion through the solution orientation of sustainability science.

The nature of sustainability problems is best conceptualized through defining them as wicked problems [10, 11]. Wicked problems are novel combinations of complex problems that are only understood after a solution is found [12]. It is controversial as to whether solutions can actually be reached or wicked problems are at best mitigated since they are likely to be continuous and have long-lasting impacts [1]. Since wicked problems depict real-world challenges, repeated solution attempts are inconceivable because working to implement a solution alters a given situation significantly, thus demanding a suite of solution attempts [13]. Furthermore, the complexity of wicked problems makes it even more challenging to develop transferable solutions because the contextual specifics of the problem vary, thereby creating novel combinations of problems features. Additional complexity is seen in the existence of ‘super-wicked problems’, an example of which is climate change. Research on climate change suggests that time to find a solution is running out, and the creators of the problem are also the same as those with the potential to solve it [1]. While we acknowledge the importance of super-wicked problems (e.g., [1]) our goal in this manuscript is not to address them per se, but rather focus on a more modest goal of exploring how inter- and trans-disciplinary methodological approaches can be used to progress the sustainability agenda. Particularly, because super-wicked problems as a concept does not significantly add to or change the conceptual thinking we propose in this manuscript, we find increased merits in focusing more narrowly on sustainability as a wicked problem.

Current problems such as climate change, changes in biosphere integrity, and land use change, share the key characteristics that define super-wicked problems [14]. These problems are typically multifaceted and complex, impeding comprehensive understanding and demanding solutions beyond mere panaceas [15]. While problems need to be urgently addressed, potential solutions need to be salient, credible, and legitimate to have relevance outside of science (e.g., for policy [16]) and require novel approaches because existing knowledge does not address their systemic causes [17], [18], [19]. Many of these problems span the whole globe in their impact but demand recognition of local context and the integration of stakeholder interests for approximating viable solutions [9]. For instance, while climate change occurs on a global scale, the negative impact of climate change is often observed and felt at the local scale [1]. Just as with climate change and adaptation to it, solution attempts to wicked problems leave no room for errors as only one solution attempt is possible. Yet these problems are continuous in nature and solutions can therefore only be approximated on an ongoing path [1]. Besides the practical challenges presented by wicked problems, they also require sustainability science to benefit from advances in other domains [20]. In order to effectively address wicked problems integration and cooperation between scholars and practitioners are crucial to establish a solution-oriented agenda [21] that is use-inspired and addressing real world place-based problems [22].

The tension between the disciplinary roots of sustainability science and its solution-oriented agenda creates methodological challenges, among other problems [14]. The field comprises two stands of research, the disciplinary rooted descriptive-analytical and the solution-oriented transdisciplinary science [8]. With regard to the first, research comprises disciplinary and interdisciplinary approaches that focus primarily on describing and understanding sustainability problems through descriptive analysis and advanced modeling [23]. This research follows a knowledge-first mode for understanding the root-causes and underlying mechanism, with its core interest focus on the problem [24]. Transdisciplinary sustainability science encompasses research on evidence-supported solutions generating actionable knowledge to solve and mitigate context specific problems [23]. This research focuses on transition pathways towards solutions, with its core interest laying on the solution(s) itself [24], yet we acknowledge the complexity of consolidating a project team [25]. While both strands of research have their merits for sustainability science [8], the methodological gaps between disciplinary and integrated approaches hamper efficient knowledge production. There is a need to effectively move from a mere problem analysis to creating solutions. In addition, there is insufficient exchange of methodological approaches between different fields. Thus, while necessary approaches to generate knowledge might exists in one field, they are unknown in other fields.

In this discussion paper, we focus on three aspects where we highlight methodological challenges in sustainability science.

i) descriptive-analytical research often relies on established methods despite diversity and complementary approaches that are available to address wicked problems;

ii) solution-oriented research may potentially lack the required scientific rigor in procedures and ethical standards to account for the unique features of sustainability problems;

iii) both standards of research have failed to deliver on their promise to facilitate longitudinal research which is necessary for wicked problems, as the creation of solutions does not contain stopping rules, since solution can only be approximated, but never reached.

Taken together, there is a need to consider diverse perspectives and coherent procedural frameworks in order to enable knowledge generation on a given problem state, desired futures, and solution implementation.

Solutions to wicked problems require three types of knowledge. The first is systemic knowledge that involves diagnostic understanding of the underlying dynamics and conditions. The second is normative knowledge to indicate the direction of change; and the third is transformative knowledge and solution strategies that effectively transform the problem at hand. Thus the three-fold knowledge can be pragmatically differentiated into system, normative and transformational knowledge [26]. System knowledge refers to rigorous empirical research on the multi-scalar nature, facets and complexity of wicked problems. This knowledge often has a profound disciplinary rooting and is usually generated through specific disciplinary lenses and sets of methods. Normative knowledge relates to norms, values and objectives that define how desirable a situation is, and thus provides a specific orientation and aim in decisionmaking and the development of solution options. Normative knowledge deals with the human perception of the situation demanding increased input from the social science and humanities for evaluating and valuing decision stakes. Finally, transformative knowledge requires development of solutions strategies and instructions to transform the analyzed problem for making progress towards greater sustainability. Transformative knowledge is least often generated and is typically unique to solution-oriented transdisciplinary research in sustainability science.

In the following sections we employ this knowledge typology in order to facilitate discussion of the methodological challenges in sustainability science. With regard to wicked problems we discuss, first, the need for diversity in methods, second, procedural and ethical challenges, and third, the demand for and necessity of longitudinal research.

The three types of knowledge—system, normative and transformative—foreshadow the need for diversity of approaches to work on wicked problems [27]. In fact, borders between natural and social sciences are not helpful to holistically understand environmental characteristics and societal dynamics of wicked problems (systemic knowledge) [28]. Explicit normativity in judging trade-offs in decisionmaking on wicked problems is crucial for orienting actions toward solutions. This requires methods that capture the multiplicity of values in order to disentangle the politics of change and clarifying whose values count (normative knowledge) [29]. Novel approaches and methods are needed for advancing “evidence-supported strategies that match the complexity of the problems they address” because the characteristics of what transformational knowledge entails and its implementation remains vague [30].

The differentiation into knowledge types illustrates the difficulties in linking results obtained through different methodological approaches because the application of specific methods and generation of knowledge are often subject to specific disciplines. A key challenge in the generation of system knowledge is that few studies apply interand transdisciplinary approaches. However, the use of boundary objects—which are defined as plastic objects or concepts that can integrate across different disciplines or knowledge domains [31]—can bridge perspectives and disciplines. Prominent examples of this approach include the application of frameworks such as the Ostrom framework for diagnosing and analyzing social-ecological systems [32].

Within the realms of wicked problems typical boundary objects were coined “big hairy audacious goals” [33], which indicate problems that can help to align and integrate a diversity of people towards achieving these goals. A classic example would be a lake that has collapsed and that scientists and stakeholders try to restore. While we acknowledge the existence of a wide array of methods to approach such problems, it remains a key challenge to ensure that diverse approaches are undertaken to generate a foundation for possible solutions.

Mixed method approaches (which we define as a combination of different methods, often qualitative and quantitative methods) could allow researchers to generate knowledge across different disciplines. This would, however, require research teams with competencies in and across different disciplines [34]. We therefore propose that in order to approach wicked problems the reflection on knowledge diversity, which we define as taking advantage of sources that generate a wide array of knowledge is a key component in any research project.

Normative knowledge gives rise to questions over which values are important and how to account for these [29]. In addition, solution-oriented research on wicked problems demands increased reflexivity. Researchers need to critically reflect on underlying worldviews of proponents that engage in sustainability transitions, while being sensitive to co-optation of experimentation and reflect on who benefits from solutions [35]. Besides reflexivity, a solution orientated agenda would also benefit from an increase in the accountability and legitimacy of generated knowledge [18]. A clear understanding of the different possible modes of exchange and communication needs to be facilitated. This demands an active integration of stakeholder knowledge in the research process across different scales [36], enabling joint problem framing, as well as the co-creation of solutions [34]. We recognize that for many topics and branches of science, a recognition or exploration of normative knowledge is less prevalent (e.g. [31]). However, system understanding and normative knowledge needs to be effectively linked, since most systems are managed and dominated by humans, which is why it is important to account for human perceptions of system dynamics.

Since existing knowledge has not led to addressing the systemic causes of wicked problems, transformational knowledge demands development of new methods that allow for generating of evidence-based solutions. This again is challenging since transformation per se needs also to be facilitated and the implementation of changes cannot be made by scientists, but involves a wider array of actors and recognition of power structures. Methods to support transformation are, to date, probably the weakest link within the three types of knowledge (system, normative and transformative), and we have only begun to explore the methodological approaches necessary to gain transformative knowledge and understand and act using such novel associated approaches [37].

Wicked problems demand higher diversity and complementarity in methodologies to allow generating evidence across the three knowledge types, which we propose is still widely lacking to date. As such, wicked problems might as mentioned serve as boundary objects to integrate different disciplines, raising awareness of the resources and approaches needed to approximate solutions. In order to serve as boundary objects, wicked problems would need to be sufficiently understandable to all stakeholders [38] because the recognition of a wicked problem can also be variable within a system [39]. While a concrete goal can aid to integrate diverse stakeholders, many wicked problems are more complex and harder to communicate and explain. Challenges such as the refugee crisis, resource depletion, or loss of biodiversity can be seen with this logic as wicked problems. All these problems are essentially novel, continuous, and demand the approximation of solutions. For instance, a large body of knowledge exists on climate change, yet mitigation (preventing climate change), and adaptation (implementing solutions for the impacts of climate change), remain as challenges [40].

The generation of transformational knowledge should therefore consider a diversity of methods [41] beside the development of novel methods (see above). This does not mean that system and normative knowledge do not demand recognition of available methods, however a stronger coherence and experience of knowledge generation is already at hand. While knowledge produced through disciplinary lenses is the necessary first step in better understanding wicked problems, it does not translate itself into the solutions to address them [42]. To effectively address wicked problems sustainability science needs to identify and clarify specific sets of methods that are considered suitable for: 1) generating different knowledge types; 2) facilitating pragmatic selection of appropriate methods to generate knowledge; and 3) allowing for increased coherency in future research.

The core criteria to identify suitable methods is that they help to approximate solutions, even if only indirectly. In addition, this needs to be integrated against research availability. It is clearly beyond the scope of this paper to provide criteria that help to identify suitable methods, given that solution attempts for wicked problems cannot be pre-defined following a simple checklist, yet this paper suggests some initial approaches. However, we suggest that mixed method approaches are hardly explored to date in the literature compared to the majority of papers that rely on single-method approaches.

Regarding the diversity of knowledge, claims for generating gains through interdisciplinary reach back decades [43], and knowledge from outside of academia is also increasingly demanded [34, 41]. Knowledge diversity as such can thus be mapped out both within academia (i.e. between different disciplines), as well as inside and outside of academia. System thinking presents a classic approach to the integration of diverse knowledge, for instance by focusing on resources [44]. In order to realize a transformational agenda, modes of governance of the systems that need sustainability solutions [45], [46], [47], [48] and interaction need to change [49], even going beyond established societal goals and transdisciplinary paradigms [50].

The question on sufficiency of results obtained from different methods can naturally not be easily answered due to the nature of wicked problems. Yet recent frameworks suggest that novel approaches may trigger our thinking of shorter pathways to solutions, while also considering the diversity of system, target, and transformational knowledge. A typical example is the leverage points approach proposed by Donella Meadows [51], which suggested the existence of different intervention types to tilt systems towards a more sustainable state. This represents in many aspects a solution orientated framework that may allow to approach typical problems from e.g. resilience theory. While resilience proposes shifts in systems from one state to another [52], leverage points may serve as “counter levers” to create solutions away from undesirable states of the system. Accordingly, we propose that such frameworks such as leverage points may demand further consolidation (e.g. transdisciplinarity [41]), while recognizing the wide array of methods at hand [34]. Yet in the context of wicked problems it is especially unclear which combination of methods may allow an approximation towards solutions.

The development of solutions to wicked problems demands methodological procedures and ethical considerations to facilitate a structured approach for researching effective solutions. It would be beneficial to integrate and learn from other disciplines while advancing sustainability procedures and research ethics. Besides procedural guidelines that clearly structure and exemplify how to select and adopt suitable methods for testing solutions to wicked problems, the experimentation with real-world transformation demands explicit ethical considerations.

Procedural guidelines for solution-oriented research in sustainability science needs to facilitate the development of methodologies that are explicitly oriented towards designing and testing solutions for wicked problems [24]. Wiek and Lang [23] outlined such a methodology with the transformational sustainability research framework. Reflecting on their proposal we distill the core features to inform procedural guidelines in sustainability science:

1) Orienting the research by defining questions and goals;

2) Designing the research: selecting frameworks and methods; and

3) Testing solution options: evaluating efficiency and effectiveness.

By dividing the orientation, design and evaluation of research into these steps, we argue that necessary ethical constraints can be made more transparent, and thus explicitly considered within research projects. We agree that more reflection on the necessary ethical rigor would be needed in future research, yet propose the suggestions mentioned by Wiek and Lang [23] as a vital starting point to integrate these approaches with the necessary reflection within the research process.

The first step ‘orienting the research’ refers to the identification of the research objectives [53]. This involves clarifying the questions the research aims to answer as well as setting its goals. In addition, one needs to identify the wicked problem that is subject to the research and explicate its wickedness and sustainability relevance. This also helps to situate the research in relation to system, normative and transformational knowledge.

The second step ‘designing the research’ refers to selection of appropriate frameworks and suitable methods as well as detailing their application. This involves reflection on the limitations and virtues of the selected framework(s) with regards to the research objectives and clarification of potential biases. However, with regard to the aforementioned need for method plurality we emphasize the need for triangulation and explicit consideration of participatory setting for collecting data.

The third step ‘testing solution options’ refers to the application of generated knowledge to the identified wicked problem. This involves an experiential approach including base-line assessment, implementation of solutions and evaluation of observed effects [37]. The use of experimental settings provide an advantage in that solutions can be tested in small-scale settings that allow for evaluation of the effectiveness of solutions without changing the entire constellation of the investigated wicked problem. However, sustainability science needs to engage in an explicit reflection of the ethical dilemmas and consequences that this might pose, also to consider resource designation within research to focus on specific problems.

Ethical considerations of solution-oriented research in sustainability science need to account for the unique features of sustainability problems [24]. To date no clear procedure exists, which gives reason for concern, as research involves not only investigation in systems including people, but also aims to transform these systems towards sustainability. There may be trade-offs, winners and losers, and unexpected impacts to both the social and physical realms of the sustainability problem. Within other branches of science—most prominently medicine—clear and strict ethical guidelines are available, and ethical committees and checklists are often part of the research process. In sustainability science, ethical considerations are most often driven by the vigor of the disciplines included in a certain research project, yet we are not aware of any procedure that documents research projects in sustainability science under ethical concerns. We argue that such guidelines are crucial in the case of wicked problems, since in these cases, solutions can only be identified in retrospect.

Sustainability science emerged out of different disciplines, which is one reason why it is so difficult to establish a coherent ethical standard. Since researchers investigate a wide cultural and societal diversity of normative dimensions, and are often even embedded into the research process, documentation and evaluation are crucial [37]. Transformation towards sustainability can follow different trajectories, which are subject to politics of change. Since wicked problems can only have one solution approximation, numerous other trajectories are potentially less effective or can create unintended negative impacts. In order to make the research process as valid as possible from an ethical standpoint, ethical guidelines would be beneficial to ensure documentation and transparency of the research.

Prominent examples for the need of ethical considerations are the currently emerging real-world labs [54]. In these ‘living labs’ complex dynamics are investigated and manipulated, however, little documented consideration is given to the ethical implications of such approaches nor does consensus exist over the guidelines for such reflections. One could consider the well-known concept of ‘informed consent’ as suitable for such approaches where the goal and procedure is clearly communicated to participants, who are asked to agree to the research process prior to participation. The concept of informed consent is however potentially controversial in sustainability science, since identification of participants and affected stakeholders is variable if not difficult [53], particularly given the multiple spatial and temporal scales that many sustainability problems span across over. Another concept well known in medical research is “double blind studies”, where both participants and scientists do not know receives a treatment and who a placebo. However, such procedures can hardly be introduced to sustainability science, the crafting of actionable knowledge relies on co-production [54]. This makes a strict planning and reviewing of research approaches necessary in sustainability science.

Another example in relation to living labs is the question of different levels of system manipulation [55]. While, for instance, some neighborhoods in a city are transformed [56], others might remain unchanged to allow for a comparison. While this certainly has merits from a statistical perspective, it may be a reason for serious concern from an ethical perspective, i.e. considering the rights of the people involved in the research. One way of overcoming this issue could be to offer to facilitate transformation after the conclusion of the research, following the three-step procedural guidance outlined above. In this way, those participants who were initially part of a control group receive the treatment once the research component is complete. On the other hand, resources are unlikely abundant enough to solve each and every single problem in sustainability science, and most definitely not every local case. Such an approach how intervention in one local system may influence a larger-scale system over a longer time period. Therefore, in order to allow for a more formalized evaluation of research processes, we call for increased attention on the ethical considerations involved in the approximation of solutions to wicked problems. Recently founded institutions such as the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (IPBES), attempt to acknowledge this concern [57, 58]. The IPBES is the intergovernmental body which assesses the state of biodiversity and of the ecosystem services it provides to society, in response to requests from decision makers (http://www.ipbes.net). The IPBES builds strongly on a transparent and coherent communication and a constant revision of their agenda. We propose that accompanying research approaches should be increasingly used by researchers and institutions, thereby allowing for an evaluation of research based on comparable ethical standards.

Rigor in the application of methods in sustainability science is challenging, since the diversity of methods is rooted in many different knowledge domains, schools of thinking, and disciplines. Sustainability science is therefore in the unique position to integrate ethical rigor based on diverse experience. While this is surely beneficial, it is vital to focus on solutions, which can be integrated with scientific rigor. In this sense, not only should our research be focused on sustainability, but it also needs to be made under sustainability standards. It is beyond the scope of this paper to discuss these standards, yet we can state that resource availability has to be balanced against the given solution approximation. While we agree that novel methods can be beneficial [1], we would also underline that a diversity of methods is already at hand, yet most methods are restricted to their given epistemological and disciplinary background and reasoning. Opening up these barriers would surely be beneficial, since diverse sampling and analysis approaches are vital, as this would theoretically increase the possibility of approximating a viable solution.

Due to the continuous nature of wicked problems, solutions may be approximated and can only be identified in retrospect, which is why research methodologies need to allow for longitudinal research and data analyses. While we acknowledge that due to their continuous nature wicked problems may not necessarily be solved and may only be dampened, we also suggest that the implementation of solutions might cause adverse effects that were not initially considered. In addition, many wicked problems are characterized by dynamics that only become visible though decade-long observation, such as rangeland management [59]. Researchers need to embrace reflexivity in order to alter and adapt the research process. This is especially challenging when it comes to mixed method approaches, as the different grain between approaches (i.e. quantitative and qualitative) is difficult to match.

Among the most advanced approaches in sustainability science is certainly systems thinking, which taps into a wide and diverse set of data and methods [15]. While to date a large array of research has already applied systems thinking, we propose that this does not often contain mixed method approaches and often focuses too narrowly on gaining increased understanding of wicked problems thereby neglecting normative and transformational knowledge. In addition, these approaches rarely employ methods that actually allow for reflexivity. It is evident that implementing acontinuous evaluation of the research process and outputs to identify the effectiveness of interventions is vital when considering wicked problems [37]. Numerous disciplinary methods are specifically designed to implement longitudinal analyses (e.g. clinical trials in medicine or greenhouse experiments in biology), while long-term approaches are also common in social sciences and urban research [60]. Matching both quantitative and qualitative methods in a mixed method framework is time consuming, and applying a wider canon of methods demands typically more resources. Yet most research projects have a much shorter funding period than what is needed to respond to sustainability problems. Projects with a backcasting approach would for instance be able to implement the approximation towards solutions of wicked problems into a sustainability science agenda. This would allow for a clear visioning process, and then enable researcher to approximate solutions, ideally based on mixed method approaches.

Wicked problems are at the heart of sustainability, as many of them prominently defy justice, since the negative impact is highest in regions that contribute less to the creation of the problem. Yet while the notion of wicked problems has existed for a long time, the rise of sustainability science in the last two decades has triggered a stronger engagement in research approaching the concept of wicked problems. We propose that by embedding the framework of wicked problems more thoroughly into sustainable science, we can create a stronger argumentation to:

1) engage a diverse set of methods, including mixed methods to guide research on wicked problems and demand procedural rigor for orienting and designing sustainability research as well as testing solution options;

2) a clear documentation of research is needed to facilitate the vital ethical considerations and to increase research transparency and legitimacy that in turn allows for retrospective analysis of wicked problems; and finally,

3) a long-term research approach is vital to enable this retrospect perspective and to make it possible to engage in the whole timeline of a wicked problem.

Sustainability science is only just starting to develop an agenda on how these three challenges can be integrated and finally solved. There is already a huge portfolio of methods available, yet many of these are not sufficiently used outside of their given knowledge domain. Also, sustainability research needs to stronger shift towards solution orientation producing normative knowledge and especially transformative knowledge. With research integration and application becoming more complex, ethical procedures need to be developed, tested and applied. Only by doing this will we be able to generate transferable knowledge and facilitate long-term research, societal changes and shifts typically span across longer time scales. Initial accounts of diverse and integrated articles [37] and textbooks become increasingly available [23], paving the road for the next generation of researchers [30].