Risk assessment models for invasive species: uncertainty in rankings from multi-criteria analysis

Abstract

Uncertainty analysis is described in the context of risk assessment for invasive plant species, where assessment criteria can be weighted using a weight-assignment methodology based on multi-criteria decision analysis (MCDA). A description is given of the essential elements of the Victorian Weed Risk Assessment (VWRA) model that ranks weed species according to scores determined from the synthesis of expert opinion and published literature. The VWRA model uses MCDA to produce a priority ranking of risk for pest plant species by compiling complex data into components with similar themes, arranging these components into the appropriate hierarchical order and then assigning criterion weights to each component. The aim of the study was to investigate the uncertainty and statistical significance in the ranking of the invasive species produced by the model. The methodology used for the uncertainty analysis is described and employed in the evaluation of the two categories of interest, represented by the statistical factors of impact and invasiveness. The criteria contributing to the uncertainty in the predicted ranking were found to be mainly in the impact category, rather than the invasiveness category, and related to agricultural factors such as vector status, reductions in yield quantity and increasing harvest cost.

Appendix

VWRA model

At the highest level of the hierarchy in the VWRA model, the Weed Risk Score, R, for a given weed species is based on the aggregate of scores from three primary categories, i.e. invasiveness (V), distribution (D), and impact (M) as follows:

$$ R = V + D + M $$

(13)

where,

$$ V = \sum\limits_{b = 1}^{b = v} {V{}_{b}} ;D = \sum\limits_{l = 1}^{l = d} {D_{l} } ;M = \sum\limits_{q = 1}^{q = m} {M_{q} } . $$

(14)

The category scores V, D and M are combinations of a large number of branches V _b , D _l and M _q which consist of weights (expert opinion) and intensity ratings (documented information or data).

Figure 1 shows a hierarchy with the invasiveness category expanded. For an arbitrary branch, V _i is the product

$$ V_{i} = wv_{pi} \times wv_{gi} \times wv_{ci} \times rv_{ni} \quad {\text{for}}\,1 = 1,2, \ldots v $$

(15)

where, wv _pi is the primary weight for invasiveness, wv _gi is the group weight for invasiveness, wv _ci is the individual criterion weight for invasiveness, and rv _ni is the individual numerical score (intensity rating) for each invasiveness criterion (not shown in Fig. 1). More weights may be added to this product, depending on the number of levels in the hierarchy underlying the model. Each branch is a complete path from first level to the base level in the hierarchy, i.e. from primary weight to intensity rating.

In a similar manner, M _k , is the product

$$ M_{k} = wm_{pk} \times wm_{gk} \times wm_{ck} \times rm_{nk} \quad {\text{for}}\,k = 1,2, \ldots m $$

(16)

where wm _pk is the primary weight for impact, wm _gk is the group weight for impact, wm _ck is the individual criterion weight for impact, and rm _nk is the individual numerical score (intensity rating) for each impact criterion.

By convention, the so-called primary weights α, β, and δ for the three categories are the first terms in the product series for any branch in a specified category, i.e., for i, j, k = 1, wv _pi  = α; wm _pk  = β; wd _pj  = δ. The model can be expressed in a more general form, i.e. for each branch,

$$ V_{b} = \alpha \mathop \prod \limits_{i = 2}^{i = v} \left( {wv_{ib} rv_{b} } \right);D_{l} = \beta \prod\limits_{j = 2}^{j = d} {\left( {wd_{jl} rd_{l} } \right)} ;M_{q} = \delta \prod\limits_{k = 2}^{k = K} {\left( {wm_{kq} rm_{q} } \right)} $$

(17)

The full generalised VWRA model for risk prediction is determined by multiplying the final weighting of each criterion by the numerical score (intensity rating) for each question, and summing these products for each branch and each category in the hierarchy, i.e.

$$ R = \alpha \sum\limits_{b = 1}^{b = B} {\mathop \prod \limits_{i = 2}^{i = v} \left( {wv_{ib} rv_{b} } \right)} + \beta \sum\limits_{l = 1}^{l = L} {\mathop \prod \limits_{j = 2}^{j = d} \left( {wd_{jl} rd_{l} } \right)} + \delta \sum\limits_{q = 1}^{q = Q} {\mathop \prod \limits_{k = 2}^{k = K} \left( {wm{}_{kq}rm_{q} } \right)} $$

(18)

Final risk score, R, is computed for each species under consideration and subsequently ranked in priority order. The values for D (distribution) are incorporated into the VWRA by a slightly different approach, based on the analysis of spatial distributions, using present and potential distribution estimates, which are region dependent, unlike those for the impact and invasiveness categories, and also using a climate matching approach (Pheloung 1996; Weiss and McLaren 2002; DPI 2008; Bomford et al. 2009).

In a typical implementation of the model, the criterion weights are assigned values using the AHP approach and workshop sessions with subject matter experts (Saaty 2005). The output of the VWRA model is the weed risk score for a given weed species. The computed risk score for each species is then converted into a ranking for the purpose of compiling a priority list for weed control (Weiss and McLaren 2002, DPI 2008). This decision-support model has some resemblance to an intelligent expert system in that it captures judgements from experts and uses a database of information, and therefore differs conceptually from parametric statistical models, such as regression analysis or Bayesian inference.

As shown in Fig. 1, the AHP approach to weight-assignment for the VWRA model produces a hierarchy analogous to a decision tree. Scores in parentheses are weights in each section (i.e. for invasiveness, distribution and impact). The order of importance for the groups in the invasiveness category is (a) establishment (0.500)—the ability of the plant to establish an ecosystem; (b) dispersal (0.284)—its ability to disperse, (c) reproduction (0.119)—its reproduction strategy, and (d) growth/competitive ability (0.097)—its growth/competitive ability. Table A1 shows the spreadsheet format for final weights provided by the AHP, and Table A2 shows a worked example for the invasiveness component of the species Hydrocotyl ranunculoides L.f.

VWRA software

The Catchment Decision Assistant^© (CDA) software was used to prioritise criteria into a structured decision making process for the VWRA model using the AHP (Weiss and McLaren 2002; DPI 2008). A compact formalism for the AHP is summarised as follows (see Tavana 2006). The relative weight of each criterion (w _i ) and sub-criterion (w _ij ) is captured and measured using a team of experts. If c ₁, c ₂,…, c _i are the i criteria that contribute to the risk assessment, then the goal is to assess the relative importance of these criteria. Each team member compares each possible pair c _j , c _k of the criteria and provides a judgment on which criteria are more important and by what degree. The AHP quantifies these judgments and represents them in an i × i matrix, A = (a _jk ), where j, k = 1,2,…, i.

When c _j is judged as equal in importance as c _k , then a _jk  = 1. When c _j is judged as more important than c _k , then a _jk  > 1. When c _j is judged as less important than c _k , then a _jk  < 1. Note that a _jk  = 1/a _kj and a _jk  ≠ 0. Since the entry a _jk is the inverse of the entry a _kj , the matrix A is a reciprocal matrix. Thus, A _jk reflects the relative importance of c _j compared with criterion c _k . For example, if a ₁₂ = 3 then c ₁ is 3 times as important as c ₂. The vector w representing the relative weights of each of the i criteria can be found by computing the normalised eigen vector corresponding to the maximum eigen value of matrix A. An eigen value of A is defined as λ which satisfies the matrix equation A w = λw , where λ is a constant associated with the given eigen vector w. The best estimate of w is the one associated with the maximum eigen value (λ _max) of the matrix A. Since the sum of the weights should be equal to 1, the normalised eigen vector is used (Saaty 1994a).

The AHP requires team members to be consistent in their pair-wise comparisons and this is checked by an associated measure of consistency. When the judgments are perfectly consistent, the maximum eigen value is λ_max = i, which is the number of criteria compared. In general, the responses are not perfectly consistent, and λ_max > n. Larger values of λ_max imply greater degrees of inconsistency. The consistency index is defined as CI = (λ_max − i)/(i − 1). The consistency ratio is the ratio of CI to RI for the same order matrix, where RI is based on a simulation of a large number of randomly generated weights, i.e. CR = CI/RI, where a value of 0.10 or less is considered acceptable (see Saaty 1994a, 2005; Tavana 2006). If the CR is unacceptable, the team member is notified that the pair-wise comparisons are logically inconsistent and is encouraged to revise them in a second round.

Implementation of the AHP process is described in more detail by Saaty (1994a, b, 2005), and demonstrated in recent applications by Tavana (2006), and Rahman and Saha (2008). The steps in formulating the VWRA model, using the AHP, and using the CDA software, are described by Weiss and McLaren (2002).