Framework for participative reflection on the accomplishment of transdisciplinary research programs

Abstract

In response to the increasingly complex social–ecological issues facing society, there is a growing trend to conduct environmental research in large collaborative programs. This approach is described as transdisciplinary research as it transcends formal disciplinary boundaries, explicitly acknowledges that many different perspectives are relevant to the resolution of complex problems, and actively involves the users of research. This poses challenges for the evaluation of “impact” as any evaluation process must take into consideration the different expectations, values, culture, language and reward structures of the main participating groups, the funders, researchers and end users. How can these participating groups learn about the progress of a transdisciplinary research program in a way that is purposeful and structured, continues through the life of the program, and includes explicit feedback mechanisms that facilitate adaptation during the course of the program? This paper presents a framework for co-reflecting on the accomplishment of transdisciplinary research programs. The framework incorporates the perspectives of funders, researchers and users, and recognizes that while they place different emphasis on measures of achievement such as efficiency, rigor and relevance, ultimate accomplishment in terms of translating knowledge into practice requires that the needs and expectations of all three groups are adequately addressed. What emerges from the framework is the importance of early investment in processes, behaviors and relationships that foster social learning and the co-production of the knowledge and understanding that are required to ensure relevance; while maintaining emphasis in the traditional areas of formally testing evidence and mentoring young researchers to ensure rigor and build confidence and capacity in transdisciplinary approaches.

Introduction

In a world of competing proposals to tackle complex problems with limited research funds, there is growing pressure to demonstrate the contribution of research to society. The public funding of research is increasingly viewed much like any other business investment (Ziman, 1996, Etzkowitz, 2003), requiring tight accountability against a range of performance criteria. Accordingly, the quantitative evaluation of research has emerged as a central practice for determining individual performance of researchers, the quality and effectiveness of research programs and the scientific standing of whole organizations (Gibbons and Georghiou, 1987).

While research in many fields lends itself to traditional cost:benefit or return on investment analysis, (Bozeman and Melkers, 1993, Kerssens-van Drongelen and Bilderbeek, 1999, Salter and Martin, 2001, Ruegg and Feller, 2003), this is difficult in the arena of natural resource management due to long time lags between intervention and response, large spatial scales, multiple interacting drivers of change and resources held and managed as common property. The non-linear interdependencies that occur at multiple scales and the unknown thresholds of system change that are typical of most natural resource management issues lead to an inevitable level of “irreducible uncertainty” (Walker and Salt, 2006), which makes evaluation particularly challenging.

Natural resources such as water and biodiversity are essentially embedded in social systems, which are typified by a range of stakeholders with very different values, expectations and time horizons (Pahl-Wostl and Hare, 2004). In addressing the management of multi-stakeholder social–ecological systems, we are faced with problems or issues that cannot be confined to a single spatial scale or timeframe. Causes and effects tend to be connected across spatial and temporal scales in often non-linear relationships. Examples include the effects of: land use and nutrient dynamics in a catchment on the tropic status of a water body (Carpenter et al., 1999); population density of elephants on the structure of savanna vegetation (Scholes and Mennell, 2008); and deforestation and irrigation regimes on the salinity and level of a groundwater table (Walker and Salt, 2006). Such non-discreet issues are not amenable to simple solutions brought about by disciplinary research projects. Researchers may have to seek answers and integrate concepts from across natural and social science disciplines. Furthermore, to successfully address diverse stakeholder values, an acceptable solution may not be in the form of “the right answer” but rather in the form of a negotiated outcome. Research accomplishment in the broad social–ecological context may be more important than whether or not a particular project or program successfully achieved its objectives.

Accordingly, scientists are expanding their research approaches to social–ecological issues in order to increase their effectiveness in society as a whole (Lubchenco, 1998, Nowotny et al., 2001, Gallopin et al., 2001). To this end, a number of relatively new and somewhat overlapping research paradigms are emerging. These paradigms essentially ask something new of science in terms of its relationship with society. Notable examples are post-normal science, sustainability science, interdisciplinary studies and transdisciplinary studies.

In broad terms, post-normal science seeks to address issues in society associated with substantial uncertainty and high decision stakes, and to do so in close collaboration with non-scientist stakeholders (Funtowicz and Ravetz, 1993, Rogers, 2008, Francis and Goodman, 2010). Sustainability science aims to bridge divides between disciplines and between producers and users of knowledge, with a focus on human–environment relationships and the advancement of sustainable development at multiple scales (Clark and Dickson, 2003, Komiyama and Takeuchi, 2006, Burns and Weaver, 2008). Interdisciplinary studies are concerned with addressing topics or problems that are too broad and complex to be dealt with by a single discipline and in the process draw on, integrate and synthesize insights from different disciplines to produce a more comprehensive understanding or conceptual advancement (Newell, 2001, Klein, 2004, Repko, 2008). Transdisciplinary studies incorporate interdisciplinary integration and add additional research dimensions by (a) addressing problems that are user inspired and context driven; (b) embracing complexity; and (c) acknowledging and incorporating multi-stakeholder perspectives and values (Nicolescu, 2002, Lawrence and Despres, 2004, Max-Neef, 2005, Hirsch Hadorn et al., 2008). The mentioned approaches are meant to be complementary to, and not replace, disciplinary research. Although significant overlap of purpose exists between these emerging research paradigms, for this paper we use the term, and build on the concept of, transdisciplinary research.

A key characteristic of transdisciplinary research is that the domains of science, management, planning, policy and practice are interactively involved in issue framing, knowledge production and knowledge application. To achieve co-evolution of understanding, alignment of purpose and harmonized action across these domains, substantial cooperation and management effort is required at the core of the research project (Hollaender et al., 2008). If we are to seriously embrace transdisciplinary research, and in particular want to foster an effective inter-domain research partnership, how do we effectively embed domain-inclusive learning and adaptation as part of current transdisciplinary practice?

While the ultimate achievement of transdisciplinary research might be changed practice based on well tested evidence whose value to society exceeds the cost of enquiry, this cannot be assessed for several years after a particular research program has come and gone. Conventional project evaluations seem to fall short of what is required for transdisciplinary research, in part because of the latter's multi-domain nature, the imperative of (and uncertainties around) implementing new knowledge and relatively long and uncertain time scales for realizing “impact”.

In this paper we draw from our experience in diverse projects related to evaluating the impact of research. Whilst acknowledging that conventional impact evaluations have their place, we recognize a need to facilitate inter-domain learning and adaptation towards more effective practice of transdisciplinary research. To this end we propose participative reflection as a complementary activity to evaluation. We present a framework for reflecting on the broader accomplishment of transdisciplinary studies from the perspective of researchers, funders and the users of the research. The framework should be used to facilitate co-reflection during the course of a transdisciplinary research project/program in order to help shape the processes, behaviors and relationships required for achieving the ultimate goal. We discuss how our framework and an associated process of co-reflection could promote social learning and broad accountability across researcher, funder and end user domains.