The Organizer Dilemma: Outcomes from a Collaboration Exercise

Norway is a seafaring nation and has experienced throughout history many memorable maritime emergencies and crises. Recent examples are the 1980 capsizing of the semi-submersible drilling rig ALEXANDER L. KIELLAND (Norwegian Official Report 1981), the arson onboard the car and passenger ferry SCANDINAVIAN STAR in 1990 (Norwegian Official Report 1991), and the environmental damage caused by the oil leak from container vessel M/V GODAFOSS in 2011 (Accident Investigation Board Norway 2011). Despite different types of incidents, all evaluation reports have pointed to the same problem—insufficient cross-sector collaboration (Norwegian Official Report 1981, 1991; Accident Investigation Board Norway 2011). Maritime crises in certain respects can be considered more challenging than on-land incidents. For example, distance to first responders is often longer, weather conditions can impede rescue operations more significantly, aid and recovery resources are often more highly specialized, and interagency coordination in pursuit of a common goal is often difficult to achieve. Insufficient collaboration in emergencies, however, is generic issue not just a maritime problem.

The official evaluation following the 2011 terrorist attacks on Norway stated, among other conclusions, that the response had been insufficient, and that cross-sector collaboration efforts in particular had been inadequate (Norwegian Official Report 2012). In response to the attacks, in 2012 the Stoltenberg government introduced collaboration as a fourth national emergency preparedness principle together with the existing principles of responsibility, equality, and proximity. According to the White Paper (Norwegian Ministry of Justice and Public Security 2012) that introduced the principle, collaboration was to be developed through a stronger emphasis on collaboration exercises. In crisis management, there is a widespread assumption that cross-sector collaboration exercises test and develop, among others, preparedness and cross-sector team integration (Rutty and Rutty 2012). The problem, however, is that crisis collaboration exercises both on land (Berlin and Carlström 2008, 2009, 2015) and at sea (Kim 2013, 2014; Kristiansen et al. 2017; Sørensen 2017; Magnussen et al. 2018; Sørensen, Carlström et al. 2018; Sørensen, Magnussen et al. 2018) tend to produce results with limited usefulness in real crisis work. Why this perceived situation exists is debated, but cited reasons include insufficient attention to variation (Borell and Eriksson 2013), insufficient focus on learning aspects (Berlin and Carlström 2015), dominance of mechanistic behavior (Berlin and Carlström 2013), and too much dependence on standardization (Kim 2013). Despite these legitimate criticisms, a focus on the development of varied scenarios that gradually improve participants’ collaborative behavior is an important organizational task (Sørensen, Magnussen et al. 2018).

We believe, however, that lack of a research-based knowledge, together with an absent national maritime collaboration exercise framework, makes it difficult for maritime exercise organizers to achieve and facilitate adequate collaboration development for all. This is especially true when many of the participants belong to the exercise planning organization, and thus may be familiar with internal exercise planning procedures and scope traditions. To test the hypothesis that belonging to an exercise planning organization has an effect on the participant’s perceived levels of collaboration (C), learning (L), and utility (U), the purpose of this nonexperimental, survey-based study was to measure the difference in perceived levels of CLU between participants belonging to the exercise planning organizations and participants belonging to other participating groups. Although collaboration is an integrated part of many countries’ emergency frameworks, Norway is, to our knowledge, the only country in Europe that has collaboration as an official national principle. Because the amount of international literature on the perceived effects of cross-sector collaboration exercises is scare, this study can make an important theoretical contribution to an emerging field. A focus on collaboration exercises both on land and at sea has become more prominent following the 2011 terrorist attacks. An increased knowledge about the effects of these exercises will help future exercise planners design exercises in a way that leads to increased perceived levels of learning and utility. Increased knowledge about maritime collaboration exercises will help maritime exercise designers concentrate on possible maritime problem areas such as longer distances, more limited access to equipment and personnel, and the use of maritime communication channels that differ from those on land. The study also should be relevant to policymakers, since it points out the importance of designing exercises in a way that ensures optimal learning and utility value.

In crisis work, the ability to mobilize resources and to facilitate cross-sector collaboration is an important success factor (Lu and Xue 2016). Collaboration is a horizontal process where stakeholders, based on the assumptions that the benefits are higher than the costs (Andersson et al. 2014) and prestige seeking is limited (minimal desire for personal gain), engage in partnerships to resolve complex situations (Murphy et al. 2015). Organizationally, collaboration became a popular working trend in the late 1980s, following a period in which a strong competitive focus dominated (Axelsson 2000). Today collaboration is described as both democratizing and efficiency promoting (Barker 1993) and is dominant in both public and private sectors (Sullivan et al. 2012). Collaboration is also viewed as a solution to sub-optimal task distribution and regulatory fragmentation (Drucker 2007). Not engaging in cross-sector collaboration may result in reduced efficiency, flexibility, and resilience in times of crisis (Jung and Song 2015), as well as make it harder for managers to impose order and deal with hostile consequences (Salman Sawalha 2014).

This study employs a quantitative, nonexperimental, survey-based methodology. A survey design was chosen over other methodologies and designs as it was deemed most suitable for large population groups and allowed for the desired variable testing through statistical procedures (McCusker and Gunaydin 2015). Our study focuses on one exercise—SKAGERRAK CHEMICAL OILSPILL EXERCISE (SCOPE), a 2017 full-scale maritime chemical oil-spill pollution collaboration exercise based on assigned project funding from the EU Commission Directorate-General Humanitarian Aid and Civil Protection. The exercise was coordinated by the Norwegian Coastal Administration, NCA (2017). The goal was to train first responders in crisis management and collaboration skills. The exercise occurred off the coast of Norway. It included partners and participants from Norway, Denmark, Sweden, Iceland, and Germany. The exercise was chosen as a study object because it was a full-scale, cross-sector and cross-border, collaboration exercise with both national and international maritime participants. Informed consent for our study was given by the exercise planning and organizer organization. The study also was a part of an ongoing Norwegian research project on Emergency Management and Interaction during Crises—in Maritime Context, approved by the NSD—Norwegian Centre for Research Data¹ (Ref. 44815). NSD is the Data Protection Official for Research for all the Norwegian universities, university colleges, and several hospitals and research institutes. The project was carried out according to NSD standards and requirements. An electronic survey was used to collect data and measure the exercise participants’ perceived levels of collaboration, learning, and utility. The survey was designed in QuestBack and distributed jointly to all operative 600 participants through a hyperlink by the exercise planning and organizing organization. In QuestBack, the “hide identity” option was selected, which according to the QuestBack (2018) Security Statement ensures that the IP addresses, e-mail addresses, and browsers type of participants were not identified. Data were imported and analyzed in Statistical Packages for the Social Sciences (SPSS) version 24.0. The target population was all operative, exercise participants; exercise designers, the exercise directing staff, and other strategic personnel were excluded. The sample population consisted of operatives associated with the exercise organizer organization and others belonging to other public and nongovernmental emergency organizations. The data collection instrument was the “Collaboration, Learning, and Utility Scale” (CLU-scale), which is a validated Swedish-designed instrument to measure the effects of collaboration exercises (Berlin and Carlström 2015). In addition to answering questions related to perceived levels of collaboration, learning, and utility, the participants were asked to provide demographic data about their gender, age, professional affiliation, number of years of professional experience, and number of collaboration exercises attended prior to participation in SCOPE 2017.

To test whether organization affiliation had an impact on perceived levels of collaboration, learning, and utility, all the respondents, regardless of age, gender, or professional experience, were divided into two groups: (1) those who belonged to the exercise planning and organizing organization (NCA); and (2) those who did not (“others”). To test whether there was a statistical difference when it came to perceived levels of collaboration, learning, and utility values, the individual mean scores of each group on items within the collaboration, learning, and usefulness (CLU) dimensions were first calculated, and then item scores were aggregated for each dimension by averaging them within each. To test the hypothesis that there was a statistically significant mean difference between the exercise organizer group and the “others” group when it came to perceived collaboration, learning, and utility, an independent sample t test was then performed. The alpha significance level was 0.05 (Cohen 1988). The CLU-scale consists of an array of dimensions and items, which are displayed in Table 1. A five point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree) was applied to quantify the responses.

This section reviews the participant responses to the online CLU survey. An overview of sample distributions and demographics is given. Quantitative findings are reported, and the mean (M) and standard deviation (SD) values also are listed when appropriate.

The 2012 change to Norwegian emergency legislation, which added collaboration as a fourth national emergency preparedness principle, has put a new responsibility on organizers of emergency preparedness exercises. As an effect of the new legislation, an expectation has emerged that training exercises will be carried out regularly. These exercises are assumed to incorporate not only command and control, technology, emergency plans, but also are extended to include enhanced collaboration between agencies at all levels. Studies of collaboration exercises and real emergency work in Scandinavian countries have revealed a lack of collaboration efforts by differentiating the actions undertaken in specific organizational operations such as health care, firefighting, and security (Berlin and Carlström 2009, 2013, 2015; Kristiansen et al. 2017; Sørensen 2017; Magnussen et al. 2018; Sørensen, Magnussen et al. 2018). Because Norway was the first country to include collaboration as a priority in its emergency planning, it has had to pioneer a focused effort to achieve interorganizational consensus, to develop elaborate common strategies, and to use all available resources to stabilize and establish normal order (Norwegian Ministry of Justice and Public Security 2012). Despite implementation of collaboration as an official principle, the results in this study and similar maritime research (Magnussen et al. 2018; Sørensen 2017; Sørensen, Carlström et al. 2018; Sørensen, Magnussen et al. 2018) indicate that the participant’s perceived mean outcomes of CLU are similar to those found in existing international land-based exercise studies (Berlin and Carlström 2009, 2015). Studies on the effects of Norwegian land-based exercises, however, still in large part must be performed and should be considered a gap in the literature.

In accordance with the effort to improve collaboration, the results of this maritime study imply an ambition among the organizers of exercises to achieve the new legislated standard of collaboration. This ambition seems to be higher than the ability to construct an exercise focused on collaboration. The mean CLU values determined in our study show that members of the exercise planning organization reported overall lower mean scores than the participants from the participating organizations. This indicates that exercise design and implementation were not optimal to ensure equal CLU goal completion among all the participants. The overall figures showed a mean of the collaboration variable of 3.86 for the exercise organizer group and 4.04 for others, learning showed 3.58 versus 3.79 and utility 3.37 versus 3.73.

One item in the collaboration dimension—whether the exercise participants perceived that there had been sufficient forms of discussions provided during and immediately after the exercise—differed notably in means (organizers 2.78 and others 3.33). Tentatively, the figures can indicate that an intensive preparation phase resulted in exhaustion among organizers. Their ambition to participate in the discussions might have been higher than they were able to manage. The issue of whether their points of view had been regarded pointed in a similar direction. The organizer group presented a calculated mean of 4.10 and others 4.33. Obviously, the visitors were rather satisfied with the efforts of the organizers to arrange an effective exercise.

The most remarkable difference between organizers and others was in the dimension of utility. The visitor group did, to a relatively high degree, find the exercise useful and applicable to real-life events during actual emergency work (4.70), while organizers were more cautious but still positive (4.26). Compared to other recently studied Scandinavian maritime collaboration exercises, this exercise revealed a positive development. In a study of three 2016 exercises, Sørensen (2017) found a calculated CLU mean of 3.75. A study of Exercise NORD 2016 revealed an overall mean of 3.09 (Magnussen et al. 2018), and a study of a 2017 joint Norwegian-Swedish maritime Search and Rescue collaboration exercise displayed a mean of 3.66 (Sørensen, Magnussen et al. 2018). We consider this development positive, but the results still show the need for further focus on collaboration learning and development. The results in this study suggest that among organizers the ambition to accomplish collaboration was stronger than the outcome. However, the external participants found that the exercise was truly a collaboration exercise. They also found it instructive and useful when applied to real life events. In contrast to Kim’s (2013) findings, the SCOPE 2017 exercise did not seem to only focus on sector-specific exercise-script controlled elements, there were also collaboration elements included, but the results indicate that there is still room for improvement. Self-criticism is an advantageous character trait when new legislation is launched. The opposite, lack of insight into one’s own shortcomings, will not provide improvement.

This study tested whether there was a difference in perceived exercise effect among regular participants and participants belonging to the exercise planning organization. Findings indicate that levels of collaboration, learning, and utility were higher among other participants than among those belonging to the exercise planning organizations. The results of the study also indicate that new legislation can have some effect on crisis organizations, despite lacking national collaboration exercise templates and waiting procedures on how such exercises should be evaluated in terms of goal achievements. Ekman’s 2012 study revealed that participants in collaboration exercises tend to trust more readily members of organizations that they previously knew; they could identify with each other better if exercise organizers and participants focus on constructing collaboration elements in exercises. Based on the optimistic assumption that collaboration can occur, there is good reason to continue the development of collaboration exercises as well as promote collaboration during real-life crisis events in Norway.

From the perspective of maritime collaboration exercise planners, the results indicate the need for a redesign of exercise scripts in a way that facilitates a participant’s perceived levels of CLU regardless of organizational affiliation. We recommend an inclusion and facilitation of surprised unforeseen events that amplifies already known maritime challenge areas such as distance and communication, as these will force participants to communicate, share information, and improvise across sectors. We further recommend that a national maritime collaboration exercise framework should be developed and adopted. A practical implication is a recommendation to evaluate all exercises to secure the outcome regarding collaboration skills. A regularly used questionnaire such as the CLU instrument can register the long-term value of exercises. Such a procedure could be of great help in order to adjust to new regulations and improve their outcomes.

Finally, we recommend that this study’s result also form the baseline for further studies on the effects of collaboration exercises. Especially do we recommend that similar studies are performed on Norwegian land-based exercises for comparison purposes. Although the results from Norwegian maritime exercises are quite similar to those found in the existing land-based international literature, the lack of research should be considered a gap, and the maritime results should not be generalized uncritically.

This study was limited in scope because data were collected from only one maritime collaboration exercise. By not performing face-to-face data collection, our survey methodology could have missed capturing verbal and nonverbal cues and other verbal signals, as well as conducting more accurate screening. Despite three reminder messages, a third limitation was the overall low response rate (16%). Unfortunately, this seems to be an increasing trend in Norway. Already back in 2013, the Research Council of Norway (2013) noted in an article published online that the numbers of respondents willing to respond to surveys had dropped sharply, and that the average response rate had gone down by over 40% during the previous four decades. In 2013, the anticipated response rate to the national Norwegian Monitor Survey was just over 8%. Although a lack of participation may lead to sample skewness, the sample population in this study was within the range of a normal distribution and was considered acceptable (George and Mallery 2010). In this study, the definition of collaboration, learning, and utility was subject to individual interpretation. Measuring occurred based on the instrument’s items, and not a predefined definition of terms. Thus, the participants may have interpreted the meaning differently, which may have influenced their answers and can have resulted in somewhat lower term validity.