Plan evaluation is of utmost importance as a function of good governance. It provides a means to improve the institutional basis for implementing land use controls, provides an important opportunity to improve future plans to reduce risk, and improves the vision for sustainable development and management. This article provides an overview of the methods and findings of a plan evaluation project undertaken in New Zealand. The project analyzed 99 operative plans, provided in-depth analysis of ten plans, and included a capability and capacity study of councils. This is the first time all operative plans in New Zealand have had their natural hazard provisions assessed in this manner. The information provides an important baseline for future policy improvements, and a basis for future research and policy directions. The project found that, while New Zealand land use plans appear to be improving over time, there are still opportunities for improvement. These include improving linkages between objectives, policies, and rules within land use plans; and strengthening the linkages between land use and emergency management plans. The largest challenge is the accessibility, understanding of, and updating of hazard information.
1 Introduction
In 2015, three major international disaster-sustainability-climate change instruments will be coalesced: (1) the Hyogo Framework for Action on building resilience to disasters; (2) the Sustainable Development Goals; and (3) the 2015 climate agreement under the UN Framework Convention on Climate Change. The year 2015 is also the International Year of Evaluation (UN 2014). Both research and practice have demonstrated the importance of effective evaluation as a function of good governance (Behn 2003; Ehler 2003; Patton 2011).
Institutional arrangements, such as the regulatory framework of land use plans, form part of the framework for risk governance (Renn 2008). In its broadest sense, governance refers to how society makes social choices. It can also be applied to how any organization (at a national, regional, or local level) is run. According to Warburton and Yoshimura (2005), governance is about the process, and political, legal, and administrative institutions through which decisions are made; and how these processes and institutions are managed and accountable. Governance processes include and go beyond governments—systems of governance are required that combine state governments, global governance structures, local governance, civil society, and corporate activities (Warburton and Yoshimura 2005). Land use and emergency management plans are two contributors to risk governance, by providing directions for and restrictions on land use in areas susceptible to natural hazards.
Anzeige
Evaluating how natural hazard planning provisions are accommodated in land use plans is important for three reasons. First, it provides an evidence base that supports the Hyogo Framework for Action 2005–2015. In particular it supports Priority 1—to ensure that disaster risk reduction is a national and local priority, with a strong institutional basis for implementation (UNISDR 2007). Second, plans provide a vision for the future, guide and regulate urban and rural development, and provide certainty to communities and developers alike (Lyles and Stevens 2014). Finally, evaluating plans provides an important opportunity to learn how they can be improved (Berke and Godschalk 2009). In this article we present the methodology and results of a project to evaluate all operative land use plans, regional policy statements, and civil defence emergency management plans in New Zealand.
The Resource Management Act 1991 (RMA) (New Zealand Government 1991) is the primary land use planning legislation in New Zealand. Its purpose is to promote the sustainable management of natural and physical resources. Sustainable management is defined in section 5(2) of the RMA as:
[…] managing the use, development, and protection of natural and physical resources in a way, or at a rate, which enables people and communities to provide for their social, economic, and cultural well-being and for their health and safety while (a) sustaining the potential of natural and physical resources (excluding minerals) to meet the reasonably foreseeable needs of future generations; (b) safeguarding the life-supporting capacity of air, water, soil, and ecosystems; and (c) avoiding, remedying, or mitigating any adverse effects of activities on the environment. [New Zealand Government 1991, section 5(2)]
Two types of planning documents legislated by the RMA are the subject of this study:
(1)
Regional policy statements are regional-level documents that contain regional-level policy, but no rules. New Zealand is divided into 16 regions.
(2)
Land use plans focus on local-level planning and are subservient to the relevant regional policy statement. Land use plans cover a subset of the area covered by a regional policy statement. There are two types of land use plans:
District plans, covering only local-level planning matters and prepared by either a city (urban) council or district (rural) council.
Unitary plans, covering both local-level and regional-level planning matters and prepared by a unitary council (which has jurisdiction over both regional- and local-level planning matters).
Both regional policy statements and land use plans are required to include land use provisions that address the avoidance or mitigation of natural hazards.
Anzeige
Prior to this research, no comprehensive assessment of natural hazard provisions in all operative plans had been undertaken in New Zealand. While a plan evaluation assessment was completed in the 1990s (Berke et al. 1999; Ericksen et al. 2003), it was limited to a specified number of plans, and was broader in scope than looking at just natural hazard provisions. With this research, there is now a baseline that can be used to measure trends in managing natural hazards through land use plans.
We assessed all 99 operative (that is, not proposed) plans in New Zealand, specifically to address the question: what is the state of planning for natural hazards in New Zealand? This is complemented by a more in-depth analysis of the “top” ten land use plans to determine the best of current practice, and a capability and capacity survey of councils. The survey was used to assess if there is a link between plan quality and staff capability and capacity, and to ascertain key challenges when planning for natural hazards. Using a multiple stage approach (plan analysis, case studies, and a capability and capacity survey), with rigorous methods for each stage, provides for a robust assessment of plan quality.
The first section of this article provides the context for plan quality assessment in New Zealand and the second outlines the stages in project design. Following is an overview of the plan analysis, describing the methods used to evaluate the plans and the results of the evaluation. The next section provides an overview of the case study analysis of plan quality and includes the methods used and results. The following section presents the capability and capacity survey process, including methods and results. Finally, we present a discussion of the combined findings from the three stages of the project, with key conclusions.
2 Context for Plan Quality Assessment
Plan quality has been investigated in a number of ways, both nationally within New Zealand, as well as internationally (Berke et al. 1996; May et al. 1996; Berke and Godschalk 2009; Berke et al. 2012). Assessing the planning regime in New Zealand to the year 2000, Ericksen et al. (2003) reviewed the quality of plans produced by local and regional councils in New Zealand. This review assessed 16 regional policy statements, and plans from 34 district councils (Berke et al. 1999), against eight general criteria (that is, it was not specific to natural hazards).
Since this review of plans, there have been several changes to the New Zealand legislation that address natural hazards (for example, the Civil Defence Emergency Management (CDEM) Act 2002, amendments to the Resource Management Act 1991, the Local Government Act 2002, and the Building Act 2004). Further legislative changes to the RMA on the management of natural hazards are also proposed (New Zealand Ministry for the Environment 2013a, 2013b). There is also a heightened awareness of natural hazards and their consequences as a result of a number of national and international natural hazard events (for example, the 2004 Lower North Island floods, the 2004 Indian Ocean tsunami, the 2010–2011 Canterbury earthquake sequence, and the 2011 Japan tsunami).
In addition to these changes and events, the RMA requires the review and revision of land use plans and regional policy statements every 10 years to ensure the plans and statements are responding to community, environmental, and economic development needs. More than 10 years have passed since the Ericksen et al. (2003) study and many councils have developed second-generation land use plans and regional policy statements. The land use plans and regional policy statements reviewed in this study thus differ from those reviewed in Ericksen et al. (2003).
Since the development of first-generation plans, a number of resources have been developed to assist planners in New Zealand to incorporate natural hazards management into land use planning (New Zealand Ministry for the Environment 2008a, 2008b, 2008c, 2010a, 2010b; Saunders and Glassey 2007; Saunders et al. 2011; Ramsay et al. 2012; Saunders and Berryman 2012; Saunders et al. 2013). The availability of this reference material, combined with an increased awareness of natural hazards and their associated consequences, may have resulted in changes to how natural hazards are addressed within land use plans and regional policy statements.
Given the above factors, it was timely to assess how natural hazard provisions are currently incorporated into regional policy statements, land use plans (that is, district and unitary plans), and CDEM group plans. The results of the project provide baseline data that allow future comparisons to be made of how land use plans manage natural hazards.
CDEM group plans are prepared under the CDEM Act (rather than the RMA). They are included in the study because under the CDEM Act, risk reduction issues (avoidance and mitigation under the RMA) need to be consistent with regional policy statements and land use plans (Saunders et al. 2007).
3 Project Design
The project was designed as four distinct parts, shown in Fig. 1. The content analysis of plans formed Part 1 (Saunders, Beban et al. 2014a), with the purpose to assist in answering the question: What is the state of natural hazard planning provisions in New Zealand? The objective of Part 2 (case studies) was to determine best practice in local government plans in assessing and managing natural hazards to meet the purpose of the RMA. This was achieved by identifying and analyzing ten plans that are examples of good planning practice with regard to natural hazards. Part 3 involved an analysis of councils’ capability and capacity (Saunders, Beban et al. 2014b). This was to determine key challenges faced by councils when planning for natural hazards, and how a national instrument (for example, a National Policy Statement or National Environmental Standard) could be developed to meet the needs of councils (Saunders, Beban et al. 2014a). Part 4 integrated the findings into a comprehensive report that provides an overview of the state of land use planning for natural hazards (Saunders, Beban et al. 2014).
Fig. 1
Framework for the plan evaluation project, linking mandate, plan quality, capacity, capability, context, and influences. Source Adapted from Berke et al. (1999, p. 647). RMA: Resource Management Act 1991, CDEM: Civil Defence Emergency Management, RPS: Regional Policy Statement, DP: district plan
×
The design of the project was not to “name and shame” plans that were assessed to be of poor quality; rather, it was to answer the broad question: What is the state of natural hazard planning provisions in New Zealand? As part of the project, opportunities for improving practice were identified—areas where national directions may be required to improve how natural hazards are managed within plans.
4 Plan Analysis
To analyze the natural hazard provisions of the 99 plans, careful coding methods were developed and applied. A large number of questions were developed for plan interrogation, producing a wealth of data for analysis.
4.1 Coding Methods
In order to develop the method for coding the plans, a number of previous studies into the quality of natural hazard provisions in plans were reviewed—including Lyles et al. (2012), Berke et al. (2012), and University of North Carolina at Chapel Hill (2011)—as well as more general plan quality studies (Berke et al. 1999; Ericksen et al. 2003; Berke and Godschalk 2009). The content analysis methodology outlined in Krippendorff (2013) formed the basis of many of the methodological decisions.
Krippendorff offers various method designs, depending on the analysis to be undertaken. This analysis is based around many different texts—99 plans. Given this, the methodological design selected was to apply the same content analysis protocol to each plan. This allows for comparison of results.
Since the project was undertaken (during 2013–2014), Lyles and Stevens (2014) have produced a list of seven procedures that plan quality researchers should follow. These relate to the replication of existing protocols, describing the scoring system, clarifying who coded the plans and the training received, double coding, pretesting, accessibility of plans, and reliability of coding. The methods used for the plan analysis addressed each of these procedures (Saunders and Ruske 2014).
4.2 Development of Plan Evaluation Criteria and Protocols
Berke and Godschalk (2009) provide a brief review of plan evaluation criteria. They believe that two conceptual dimensions should be included in plan quality evaluation:
(1)
Internal plan quality that includes the content and format of key components of the plan needed to guide land use in the future (for example, issues and vision statement, fact base, goal and policy framework, implementation, monitoring); and
(2)
External plan quality that accounts for the relevance of the scope and coverage to reflect stakeholder values and local circumstances to maximize the plan’s use and influence.
This research has primarily focused on internal plan quality, with analysis of key components of the plan that match those by Berke and Godschalk.
When developing the questions, other studies were reviewed to evaluate opportunities to utilize existing protocols (Becker and Johnston 2000; IBHS 2001; Ericksen et al. 2003; Beban et al. 2012; Berke et al. 2012). This allows a future opportunity to do comparative studies. As a result, and reflecting the principles outlined in Berke et al. (2012), a total of 127 questions were asked of each plan, based on 11 themes:
Plan details
Legislative linkages
Sustainability and resilience
Hazard
Hazard prioritization and vulnerability
Objectives, policies, and rules
Mapping
Anticipated environmental outcomes
Monitoring and review
Risk
The four R’s (reduction, readiness, response, and recovery).
4.3 Coding of Plans
Krippendorff (2013, p. 128) states “Recording is a highly repetitive analytical task that requires strenuous attention to details. Not everyone is capable of maintaining consistency under these conditions.” As such, consistency is a key attribute of coding.
To ensure reliable data are generated, it is recommended that analysis should be undertaken by two or more coders, who code independently of each other. Rules are developed by the coding team to ensure all coders interpret the items as consistently as possible (Berke and Godschalk 2009). A percentage agreement score is then given, with a score between 70–97 % deemed acceptable (Berke et al. 2012). To ensure a higher score, pretesting the protocol is recommended, to improve the reliability of the instructions and score (Berke and Godschalk 2009). However, there are some issues with reliability scores. Percentage agreement becomes more difficult to achieve as the number of coding categories increases (Berke and Godschalk 2009). In our research, there were a total of 127 questions, which affected the reliability score.
In the study by Berke et al. (2012), 30 state hazard mitigation plans were selected and analyzed. In contrast, in this research we analyzed all 99 operative plans. Due to resourcing issues, double coding of the plans was not deemed practical. To have a consistent and reliable approach to coding, the following actions were followed:
Testing of the protocol. Testing was undertaken by three test coders, all coding the same four plans (one regional policy statement, one CDEM plan, two land use plans).
Any discrepancies were analyzed, and the protocol questions amended and further instructions provided.
One person coded all 99 plans, in order to achieve a consistent approach to the coding. This negated the need for individual reliability scores for coders.
Plans were accessed online to ensure the most recent version was analyzed. Where plans were incomplete or not online, contact was made with the council to gain access.
The coding of plans was undertaken over a short time period, from November 2013 to January 2014.
After 4 days of coding, a meeting was held with the coder to discuss any questions, seek clarification of issues, and discuss any changes required.
A number of coded plans (one regional policy statement, one CDEM plan, two land use plans) were randomly recoded by a second coder, at various times throughout the coding process. Discrepancies were less than 30 %, that is, below the percentage considered to be an issue (Berke et al. 2012).
The primary coder recoded the first two regional policy statements, two land use plans and two CDEM plans at the end of the coding process to ensure that consistency was maintained throughout the process. Discrepancies ranged from 6 to 15.6 %, which is below the 30 % outlined in Berke et al. (2012), and therefore acceptable. No further recoding was deemed to be required.
4.4 Results of Plan Analysis
The following are key findings and implications for natural hazard planning:
CDEM group plans appear to have better linkages to the legislation, monitoring provisions, and hazard and risk information than land use plans and regional policy statements. To achieve effective risk reduction, we encourage that these linkages be improved within land use plans and regional policy statements. This will ensure consistency of approach and strengthen provisions between plans. It is thus extremely important that discussions occur between regional and district policy and consent planners and the emergency management teams within councils.
A high percentage of land use plans contain all-hazard objectives, policies, and anticipated environmental outcomes.
There is a strong bias within land use plans and regional policy statements towards flooding and landslide hazards. This may be attributable to these being high likelihood hazards. Several hazards with a low likelihood and high consequences (volcanic eruptions and tsunami in particular) have very few rules.
Land use plans lead in the mapping of natural hazards, with flood extents being mapped most often. However, a wide range of scales, both ratio and ruler format, exist. Some plans do not include scales, and this undermines their accuracy and practical use.
For earthquakes, 37.7 % of land use plans had specific rules pertaining to active faults. Given the tectonic environment of New Zealand, this suggests that there is a large underrepresentation of active fault rules within land use plans.
Many plans consider only single hazard events, and not the subsequent hazards that may result from that single event. Further awareness and inclusion of cumulative hazards should be made in the future.
68.8 % of regional policy statements identified the parties responsible for implementing its provisions. If this figure were higher, there would be an increased likelihood that the outcomes identified in the regional policy statements would be achieved.
Overall, regional policy statements and land use plans rarely include provisions for monitoring hazards. Among regional councils, floods (at 50 %) are the highest ranked monitored hazard. Coastal erosion and flooding (both at 27.5 %) are the two hazards with the highest frequency of monitoring provisions within land use plans, with tsunami and volcano monitoring the least monitored hazards (both at 1.4 %). There is also a lack of clarity around who is responsible for monitoring, which could lead to some monitoring not being undertaken.
Very few land use plans (10.1 %) and no regional policy statements set out a clear process for the inclusion of new or updated hazard or risk information. This may be because the RMA sets out the process for updating plans and policy statements. Positive statements in the documents that seek to include updated information as it becomes available would likely increase the chances of this being done in a timely fashion.
While nearly all plans include the term “risk,” it is often not defined. When it is defined, there is no standard definition used, but there are common themes of likelihood (that is, probability) and consequences (that is, damage, potential effects). It is recommended that the term “risk” be defined if legislated within the RMA (as natural hazards are defined), to ensure a consistent approach to risk management.
The analysis undertaken was limited, as it was focused on the quality of the three types of documents (regional policy statements, land use plans, and CDEM plans) and comparisons between them. There is an opportunity in the future to undertake further data mining within the data collected for this project, to analyze specific natural hazard provisions within—and across—plans.
5 Case Study Analyses of Plan Quality
Ten plans were selected for a case study analysis, to investigate, in greater detail, the quality of the natural hazards provisions. Examples of good practice were sought, as well as opportunities for improving planning for natural hazards.
5.1 Case Study Selection
The findings from the content analysis were used to identify RMA plans for closer analysis as case studies of best practice. This was done by selecting 22 of the 127 questions considered to be key indicators of best practice in planning for natural hazards. A plan was considered to be demonstrating best practice if it:
Identified hazards
Accounted for uncertainty in its maps
Took an “all-hazard” approach to objectives and policies
Included hazard-specific rules
Used risk management language (consequence and likelihood)
Included hazard-specific monitoring provisions
Set out a process to update hazard information
Made linkages with to relevant legislation
The responses to the 22 questions were queried by the computer-aided analysis program SPSS, and the highest scoring plans were selected as potential case studies.
In addition to high scores in the above analysis, a number of other criteria were considered in the selection of the case studies.
First, the initial content analysis assessed only operative RMA plans, and not proposed RMA plans.1 As proposed plans may replace operative plans in time, proposed plans may demonstrate advances in best practice. It was decided to select one proposed regional policy statement and two proposed district plans, to see if these proposed plans contained examples of best practice not seen in the operative plans.
Second, as no content analysis had been undertaken on any proposed RMA plans, the three proposed plans were selected based on the authors’ and cofunder’s general knowledge of the development of particular plans, and previous work/research undertaken with particular councils.
Thirdly, the case studies selected needed to be representative of New Zealand, so as not to bias results due to any particular geographical factor. The final selection includes plans from both urban and rural areas, North Island and South Island, and plans that cover a variety of natural hazards.
Fourth, the case studies selected also needed to include regional policy statements, and both types of land use plans—district plans and unitary plans. This allows the analysis of good practice in all three types of RMA planning documents considered in this study. It also allows a comparison between unitary plans (prepared by an authority with jurisdiction over both regional and district functions), and regional policy statements and district plans (prepared by single jurisdiction authorities), to see if this different governance structure has an influence on how natural hazards are managed in land use planning documents.
In addition, one regional policy statement and one district plan needed to be selected from the same region. This allows an analysis of best practice regarding how the planning hierarchy set up as part of the governance structure under the RMA is put into effect.2
After going through the above assessment, ten plans were selected for closer analysis as case studies of best practice.
5.2 Analysis of Case Studies
Each of the ten plan case studies was analyzed in detail to determine what it could offer as guidance on best practice for land use planning for natural hazards, in much more detail than was possible during the initial analyses.
Nine categories of best practice indicators were used in the analysis. These categories were selected as indicators of general best planning practice, as well as best practice for natural hazard planning. In particular, indicators of a risk-based approach were used, as this approach is considered best practice for natural hazard planning (Quality Planning 2013).
A risk-based approach focuses on the consequences of a natural hazard event, rather than just the likelihood of a particular event occurring, in order to determine the level of risk posed (for example, tolerable or intolerable). The principles underlying the approach are that accurate hazard information should be gathered and used as the basis of decisions, development and subdivision within hazard areas should be avoided or mitigation measures used where avoidance is not possible, and in already developed areas, there should be no increase in the level of risk. For further explanation of the risk-based approach see Quality Planning (2013) and Saunders et al. (2013).
The nine categories of best practice indicators are explained in Table 1. Good examples from the case studies in each of these categories were highlighted, as well as opportunities for improvement.
Table 1
Good practice assessment categories
Category
Comment on best practice indicators
Planning framework
Effective plans need strong linkages between issues, objectives, policies, and methods. Objectives should be written as outcomes, and the policies should provide clear direction on how the objectives will be achieved and the rules implemented. The rules should implement the policies and be well supported by the policies. Maps should also be strongly linked to the planning framework, particularly rules
An “all-hazard” approach to the planning framework provides simplicity and ensures a consistent outcome for all hazards. However, a hazard-specific approach is likely to be more effective, as it provides clearer, stronger, more targeted guidance to plan users (Quality Planning 2013)
Maps
To plan for natural hazards, hazards first need to be identified. Identification of hazards on planning maps is often the basis of land use rules. Mapping of hazard extent is important, but mapping of actual risk can be a more powerful management tool. Mapping of uncertainty is also best practice, as this ensures awareness of uncertainty, and better allows for policy and rules to address uncertainty (Saunders, Beban et al. 2014)
Monitoring and evaluation
Monitoring hazards is important to ensure accurate information is used as the basis for decisions. Under the RMA, councils are required to monitor natural hazards, as well as the efficiency and effectiveness of the provisions in their plans. Hazard objectives should be written in a way that enables progress towards them to be measured (Quality Planning 2013). A plan that specifies procedures for monitoring the efficiency and effectiveness of the policies and methods sets up a transparent process and enables monitoring
Roles and responsibilities
A regional policy statement applies in multiple districts/cities and must be “given effect to” by regional and land use plans. Therefore these statements have the unique ability among RMA plans to be very directive as to the content of regional and land use plans, helping create consistency in hazard management within a region. A regional policy statement that provides clear direction to regional and land use plans is demonstrating best practice
The RMA sets up overlapping functions for the management of natural hazards between regional and district councils. Regional plans and land use plans that specify roles, and are consistent with any roles specified in a regional policy statement, help to reduce overlaps and increase efficiencies
A number of other statutes influence the management of risk associated with natural hazards.a RMA plans that refer to the roles and responsibilities of other organizations help to improve integration of hazard management
Interaction with CDEM Plans
CDEM Plans (among other things) identify and prioritize hazards within a region. To ensure integration with other hazard management activities in a region, the preparation of hazard provisions in a regional policy statement, in particular, should be linked with work being undertaken, and priorities established, as part of the CDEM Group Plan (Quality Planning 2013). Land use plans that link to CDEM Group Plans will also be demonstrating best practice
Information base/management
Planning for natural hazards needs to be based on accurate information. Setting out the roles of the district or region in collecting, managing, and disseminating hazard information within an RMA plan is a useful way to confirm responsibilities
A clear process set out in plans for the inclusion of new hazard information helps ensure the information is used in land use planning. The risk management approach used should be flexible enough for decision making to incorporate changing or new data on the nature and extent of the hazard, and how this affects risk (Quality Planning 2013)
Uncertainty
Use of the precautionary approach in situations where there is uncertainty surrounding risk is considered best practice (Quality Planning 2013; Saunders et al. 2013). Plans that address uncertainty directly can provide strong guidance and create certainty of process, if not of fact
Cumulative and cascading hazards
Cumulative hazards are multiple, unrelated natural hazards that affect the same area. This area therefore has a higher likelihood of being impacted by a natural hazard event. Cascading hazards are different types of hazards that are all triggered by the same event. When the trigger event occurs, the area will be subject to more than one hazard at the same or similar time (Beban and Saunders 2013)
Addressing cumulative and cascading hazards in a plan allows a more sophisticated approach to be taken to the management of risk
Definition of risk
Use of the term “risk” in a plan is one indication of an attempt to apply a risk-based approach. Definitions provide certainty to a plan and help achieve consistent application of the planning framework. Plans that define risk are demonstrating best practice
aFor example, the Civil Defence and Emergency Management Act 2002, the Local Government Act 2002, and the Building Act 2004
5.3 Results
The following is a summary of good practice being implemented in plans.
Planning framework: Explicitly taking a risk-based approach, using risk management language, provides a clear planning framework for the management of risks from natural hazards. Regional policy statements have a key role in implementing a risk-based approach. Basic good practice for writing plans (such as directive language, linkages between provisions, use of definitions) leads to best practice planning frameworks for managing natural hazards.
Maps: Hazard maps that incorporate information on hazard consequences provide a strong link to the planning framework.
Monitoring: Clear identification of matters to be monitored, information to be gathered, the information source, and the frequency of review set the framework for useful and informative monitoring.
Roles and responsibilities: Regional policy statements play a key role in assigning responsibilities for risk management to regional and land use plans. Using regional rules to manage land use is an important way to overcome issues associated with rebuilding in hazard areas.
Information management: Requiring hazard information to be included on land information memoranda issued by the local council, and requiring applicants to provide hazard information where uncertainty exists, are useful ways to encourage hazard information to be used in the planning process, prior to its formal inclusion in a land use plan.
This review of ten RMA planning documents highlighted opportunities for improving good practice, as well as examples of good practice. Opportunities for improvement are discussed below.
Planning framework: A number of plans reviewed retained a high degree of discretion with the decision makers. These plans generally had weak policy frameworks and commonly used discretionary activity categories. As a result, little certainty is provided to plan users on the likely outcomes of resource consent applications, and the way the policy framework is applied is likely to be inconsistent. Such a framework may be appropriate for situations where the risks associated with a hazard are very uncertain. However, where information on risk is available, such a framework should be avoided.
Maps: Consistency in the mapping of hazards in land use plans is an area for improvement. The scales used vary greatly, the amount of information contained in a map varies, and the strength of the link between maps and rules also varies.
CDEM Plans: The opportunities offered by reference to CDEM Plans are not well taken up by land use plans. Greater cross-referencing should improve efficiencies and improve the sharing of knowledge between the two disciplines.
Uncertainty: Uncertainty was not well addressed in either the regional policy statements or land use plans assessed. Specifically, guidance on how to apply a precautionary approach would improve management of uncertainty. Options may include use of a definition of the precautionary approach, or specific guidance on the use of particular activity categories in uncertain situations.
Cumulative and cascading hazards: These two concepts were poorly addressed in the plans assessed. An opportunity exists to develop guidance on how to address these issues in RMA plans. Advancement in methods of mapping cumulative hazards, in particular, and development of corresponding rules that deal with cumulative hazards in a comprehensive way, would improve their management.
Definitions: Very few definitions of risk were found in the plans assessed. This is an area for improvement, as definitions increase certainty and help ensure that the intended outcomes are actually achieved.
6 Capability and Capacity Assessment
To assess council capacity and capability, an online survey was created and circulated to all councils (that is, city, district, unitary, and regional councils). The survey was sent to resource management policy and consent managers within councils.
6.1 Survey Methods
A survey was developed based on standard social science survey methods (Parfitt 1997). Nineteen survey questions were developed (Table 2). Of these, 11 were general capacity and capability questions addressing the themes of knowledge and awareness, and staffing, and eight were demographic type questions (that is, numbers of staff, turnover, length of service), some of which are also indicators for capability. With a mix of closed “tick box,” and open questions allowing for comments, the intention was for the survey to be completed within 10–15 min—any longer and participants may not have had time. Prior to release, the survey was peer reviewed and tested to ensure the questions were easy to understand, and the results provided were what we intended.
Table 2
Capability and capacity survey questions
1. Are you aware of the risk reduction provisions in the Civil Defence Emergency Management Group Plan for your district?
2. Do you have a staff member(s) who is responsible for providing natural hazard advice to planners?
3. If you answered “yes” to Q2, how many dedicated staff do you have?
a. Staff members position(s) and description
4. If you answered “no” to Q2, is this role outsourced to external consultants?
a. If “yes” why is this role outsourced?
5. Have any of your staff with natural hazard responsibilities undertaken any training in natural hazards? For example, University paper, New Zealand Planning Institute paper, other?
a. If “yes”, which courses?
6. Are new staff made aware of natural hazards in the district/region and the implications for land use planning?
7. Is hazard information obtained by your council peer reviewed?
a. If “yes”, when and by whom?
8. When are natural hazards considered as part of the planning process?
9. How is new hazard information incorporated into planning processes?
10. Is monitoring of the council’s objectives and policies for natural hazards undertaken?
a. If “yes”, how do you do this?
11. What is the major challenge that your council faces in planning and consenting to do with natural hazards?
12. Please state the name of your council
13. What area of RMA planning are you responsible for?
14. Number of consent staff (FTE)
15. Number of policy staff (FTE)
16. Percentage of planning staff turnover last year
17. What is the average serving time for the members of your consent team?
18. What is the average serving time for the members of your policy team?
19. Comments
The survey was managed with an online survey tool. In June 2014, the survey was emailed directly to policy and consent managers at 78 councils (142 emails). If emails were returned, they were either resent to an alternative contact (if the initial person was out of office), or if they no longer worked at the council an alternative was found.
The survey did not ask for specific contact names, only the council name. Council staff had 1 week to complete the survey, and two reminders were sent. In total, 52 responses were received from 39 councils. This meant that 50 % of councils responded to the survey (though for some councils only either the policy team or the resource consent team responded, meaning that only 37 % of the total surveys that were sent out were completed).
6.2 Summary of Capability and Capacity Assessment Results
The following provides a summary of the findings from the survey. As the response rate for the survey was 50 %, the results represent only those parties who responded to the survey and cannot necessarily be extrapolated to councils that did not respond. However, the councils that did respond had a wide geographical spread and included large city councils, regional councils, unitary authorities, and rural (district) councils in both the North and South Islands.
The key findings of the responses received were:
Only 60 % of respondents were aware of the risk reduction provisions in the CDEM group plan for their district. A significant number of the respondent councils are not aware of their CDEM provisions and their associated risk reduction roles. This shows a need for CDEM staff and planners to communicate more frequently and effectively with one another.
Approximately half of respondents do not have a staff member responsible for providing natural hazards advice to planners. Of those that do, those staff members are often emergency management officers or engineers. Approximately half of those that do not have in-house resources outsourced this role to consultants. This means approximately one quarter of respondents may not be obtaining specialist natural hazard advice, which could result in inappropriate land use planning decisions.
Of the respondents whom have staff with natural hazard responsibilities, approximately half of those staff members have undertaken training in natural hazards. This training included university papers, and courses provided by the New Zealand Planning Institute and Crown Research Institutes (GNS Science and National Institute of Water and Atmospheric Research—NIWA).
83 % of the respondents make new staff aware of the natural hazards in their district/region, and the implications for land use planning. This helps to assist with the continuity of natural hazard planning and ensures that institutional knowledge is passed on when people leave.
71 % of respondents get hazard information peer reviewed. This process ensures that the hazard information used by councils is robust, has adopted the correct methodology, and includes appropriate recommendations. This ensures that councils are able to make informed land use planning decisions regarding natural hazards. However, 15 % do not peer review natural hazard information and 14 % do not know. This creates a risk that the hazard information supplied to a council may contain some inaccuracies or assumptions that are incorrect. If these inaccuracies are not identified at the land use planning stage, it could result in developments proceeding that increase the risks from natural hazards.
Half of respondents consider natural hazards when a plan change or resource consent is within an identified hazard zone. 44 % of respondents consider natural hazards as part of all plan change or resource consent applications. Ideally, natural hazards should be considered as part of all plan changes and resource consent applications. Hazards may exist outside of the existing identified zones, and if hazards are not being considered in all land use planning, it could result in developments proceeding that increase the risks from natural hazards.
45 % of respondents monitor natural hazard objectives and policies and 24 % do not monitor them, with 31 % unsure. Monitoring is undertaken is a variety of ways, from standard monitoring of the land use plan and consent conditions, through to no monitoring. However, monitoring natural hazard provisions is difficult and often provisions are not readily measurable, making measuring outcomes difficult. To overcome this, a framework could be implemented to allow for monitoring of provisions, such as the one outlined in the risk-based approach released by GNS Science in 2013 (Saunders et al. 2013).
The challenges for planners identified by respondents included lack of information; costs of obtaining information; information reliability; not enough resources in the council; historical development in hazardous areas; property rights; and guidance on defining levels of risk. Saunders, Beban et al. (2014) explore a number of potential solutions that would improve land use planning for natural hazards and would assist with addressing the challenges identified by the respondents in the survey.
The responses indicate a wide range in the percentage of staff turnover over the last year. While many respondents had no turnover, a number had a very high turnover (50–100 %). There is the risk that councils with high staff turnover could lose institutional knowledge regarding natural hazards, which could result in natural hazards not being given the attention needed in the land use planning process. This issue is potentially overcome by the high percentage (83 %) of respondents who make new staff aware of the natural hazards in their district/region, and the implications for land use planning.
Analysis of the results shows that council capability and capacity varies across the country, but, for the respondents to this survey at least, the size of the council [that is, district (rural) or city (urban)] does not appear to have an influence on the capacity and capability of staff.
7 Discussion: The State of Planning for Natural Hazards in New Zealand
So what is the state of planning for natural hazards in New Zealand? The answer is somewhat complicated—as to be expected from analyzing 99 plans. In summary, the state of planning for natural hazards in New Zealand appears to be improving.
Increasingly, planning documents are taking a risk-based approach (as opposed to the more traditional hazard-focused approach), which reflects good practice. The risk-based approach has been implicit in many plans, but more recent documents are explicitly managing risk. All regional policy statements and land use plans have provisions to manage natural hazards. A common approach is to include “all-hazard” provisions (generic provisions that relate to all hazards) alongside some hazard-specific provisions. Flood hazard is the most well-addressed hazard in planning documents. The study has shown that basic good practice for writing plans (such as directive language, linkages between provisions, use of definitions) leads to best-practice planning frameworks for managing natural hazards.
The results of this study suggest that the planning hierarchy set up as part of the governance structure under the RMA is working reasonably well. The case study analysis of a regional policy statement and district plan showed a very directive risk management framework in the regional policy statement, and a district plan that appeared to implement that framework. However, it was not possible to determine whether the rules in the district plan would manage risk in accordance with the levels set in the regional policy statement (for example, acceptable or tolerable). This suggests governance could be improved by further considering how land use rules can link more explicitly with policy direction to achieve a specified level of risk.
The study also highlighted benefits of the “combined” unitary plan governance approach to management of natural hazards, over the more common two-tiered regional and district approach. A unitary plan had a greater ability to integrate hazard management over a larger area than a district plan, and across both regional and district-level functions. It is better able to use rules to overcome restrictions on rebuilding in hazard zones, and to apply hazard rules equally to all land use activities (including those exempt from district-level rules). Unitary plans were also better able to address cumulative hazards than district plans, particularly in the coastal environment. These benefits identified in unitary plans suggest improvements that can be made to the more common two-tiered governance structure to improve hazard management across New Zealand.
The results of the study suggest other ways in which aspects of governance (plan provisions, planning practice, and capability and capacity) can be improved. The following actions could be implemented to improve the current state of natural hazards planning:
There is potential to build capability and capacity within councils for natural hazard management, particularly on managing risk. Councils should be encouraged to up-skill on risk management (including the good practice identified in the “case study” section above). This could be through providing a list of websites, guidance material, continued professional development, and/or a “checklist of understanding” of basic natural hazard and risk concepts (which could be standardized for New Zealand).
An enhanced and more integrated approach to making natural hazards information available is needed, including making information on the nature and location of natural hazards more accessible for the public. This would help overcome existing issues with information quality and dissemination, and assist people to make better individual risk management decisions.
Natural hazard provisions should be specific, measurable, achievable, relevant, and time bound. This allows for risk reduction objectives to be measured, monitored, and reviewed if improvements can be made.
Councils should implement a risk-based approach that engages with communities to determine levels of risk. This represents best practice and will lead to improved risk reduction as risk—rather than hazard—will be managed at a level appropriate for the community.
Collaboration should be encouraged between emergency managers, policy and consent planners, and with science providers. This will improve the transfer of information and experience between disciplines, and clarify understanding of roles. Similarly, sharing of ideas and examples of good practice among councils would be beneficial—examples of this exist and can be built on.
Councils need to assess the effectiveness of existing natural hazard provisions before embarking on a second-generation plan. This may involve discussions within council teams, for example, policy, consents, emergency management, building, asset engineers, and so on, as well as “ground truthing” to determine the effectiveness of policies (Smith and Dever 2014).
8 Conclusion
The three-stage plan evaluation method used in this study has provided robust data to answer the question: What is the state of planning for natural hazards in New Zealand? Overall, land use planning for natural hazards in New Zealand appears to be improving. There is a trend for second-generation plans to take a risk-based approach, which is encouraging as it allows for a more holistic consideration of natural hazards. It will be interesting in the future to assess the success of this approach as it becomes more mainstream.
The results have also highlighted ways in which the plan-making aspect of governance can be improved. Planners themselves have identified that improvements in their capability and capacity is required. It appears that programs for the up-skilling of planners, in both risk management and general good practice plan preparation, would be well received. Increased collaboration between planners and emergency managers, and a greater understanding of the role that land use plans can play in risk reduction (as required by the CDEM Act), should also result in better risk management. There is an opportunity to improve monitoring and effectiveness reviews, to encourage progressive improvements of plan provisions over time.
Results also suggest that the governance framework in which New Zealanders manage risks from natural hazards needs to be considered alongside programs to increase planner capability and capacity. Lessons are to be taken from the benefits of the combined governance approach of a unitary plan to effectively manage risks, as compared to the more common two-tiered governance approach. Improvements in the way natural hazard information is stored and accessed, as well as improvements in the relationships and engagement mechanisms between planners, scientists, and the community, are needed.
Uptake of the improvements in the plan-making aspect of governance suggested in this study should ensure the trend for risk-based, rather than hazard-based, planning for natural hazards in New Zealand, and the quality of plans, continues to improve. Looking forward, there are many opportunities to further analyze the data gathered from the plan analysis stage and cross-tabulate it with the case studies and capability survey.
Acknowledgments
The authors gratefully acknowledge project funding from the Natural Hazards Research Platform and the Ministry for the Environment of New Zealand.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
A proposed plan or policy statement is not yet operative, having been publically notified but not yet decided/beyond appeal proceedings, but does have some weight in RMA decision-making.
The planning hierarchy referred to here is the requirement in section 75(3) of the RMA, which states that a district plan must give effect to a regional policy statement.
