Skip to main content
main-content

Tipp

Weitere Artikel dieser Ausgabe durch Wischen aufrufen

01.11.2011 | Ausgabe 14/2011 Open Access

Water Resources Management 14/2011

Multi-criteria Decision Analysis: A Strategic Planning Tool for Water Loss Management

Zeitschrift:
Water Resources Management > Ausgabe 14/2011
Autoren:
Harrison E. Mutikanga, Saroj K. Sharma, Kalanithy Vairavamoorthy

1 Introduction

One of the main challenges facing water utilities worldwide is the high levels of water losses in the distribution networks. According to the World Bank study (Kingdom et al. 2006), about 32 billion m 3 of treated water is lost annually as leakage from urban water distribution systems (WDSs) around the world, while 16 billion m 3 is used but not paid for. They further estimate that these losses cost water utilities as much as US$ 14 billion per year, with a third of it occurring in the developing countries. In light of global pressures (climate change, urbanization, growing populations, increasing water demand and scarcity etc.), water utilities particularly in the developing countries need to operate more efficiently to provide sustainable water services.
Water loss management (WLM) has become an important decision issue in meeting utility’s strategic goals. Whereas strategic planning (SP) has proven to be a valuable tool for sustainable urban water management (Malmqvist et al. 2006), water utilities in developing countries often lack the necessary capabilities to carryout SP (Mugabi et al. 2007). SP is about setting a long-term direction based on sound predictions, analysis of options and key decisions about the future of an organisation. The Uganda National Water and Sewerage Corporation (NWSC) corporate plan (2009–2012) broadly categorises water loss reduction among other sub-goals under the main goal of revenue maximization (NWSC 2009a). This traditional way of evaluating water loss reduction strategies based on a single criterion of maximising revenue is unrealistic as water services management takes place in a multiple-criteria environment. Clearly, a more structured approach for SP is required to safeguard against asset stripping by a utility driven mainly to maximize profits.
Multi-criteria decision analysis (MCDA) is a tool developed in the field of decision theory for resolving operational research problems with a finite number of decision options amongst which decision-makers (DMs) have to evaluate and rank based on the weights of a finite set of evaluation criteria (EC) (Figueira et al. 2005; Lu et al. 2007; Simonovic 2009). The discrete MCDA techniques take into account a range of quantitative and qualitative criteria beyond the financial criterion and can deal with long-time horizons and uncertainties during the optimal decision-making process (NAMS 2004). They have the potential to provide well structured, rational, consistent, transparent and objective solutions to complex decision problems in water resources management and planning (Lai et al. 2008). A recent review of MCDA for water resource planning and management has shown that MCDA is mostly used for water policy evaluation, strategic planning and infrastructure selection (Hajkowicz and Collins 2007). The same review indicates that the most commonly applied methods were fuzzy set analysis, compromise programming (CP), analytic hierarchy process (AHP), elimination et choice translating reality (ELECTRE) and preference ranking and organization method for enrichment evaluation (PROMETHEE) respectively.
Many researchers have recently solved water resources management and planning problems by application of various MCDA techniques such as PROMETHEE (Morais and Almeida 2007; Silva et al. 2010), ELECTRE II (Bouchard et al. 2010), fuzzy TOPSIS (Afshar et al. 2011), and fuzzy AHP (Wang et al. 2011). The inexact two-stage fuzz-stochastic programming method developed for water resources management under uncertainty (Lu et al. 2008), could be a valuable tool for improving WDS efficiency (leakage reduction) and supply reliability by application of the flexible penalty policies incase water service providers fail to deliver the agreed upon amount of water to their customers. Of particular interest, is the leakage management study in Brazil (Morais and Almeida 2007). Although Morais and Almeida ( 2007) addressed the leakage strategy in WDSs, they did not tackle water loss in totality as the apparent water loss component that is often significant in developing countries (Mutikanga et al. 2009) was not considered. In addition, they did not carry out a water balance to identify whether the problem was leakage or apparent losses. They instead indicate a water loss of 60% for the case study. Use of percentage as a water loss indicator can be misleading as it is heavily influenced by consumption and has nothing to do with water loss control (Fanner et al. 2007).
The application of decision theory to real-world problems in practice is by far not trivial and could be one of the hardest parts of the decision-making process. According to Hajkowicz and Higgins ( 2008), the main challenge is not in developing more sophisticated MCDA methodologies but to develop more systematic frameworks to support the initial structuring of the decision problem in terms of selecting criteria and decision options. Clearly, there remains a gap between developed decision theories and applications. The knowledge gap is even bigger in developing countries where these methods and tools are hardly applied and therefore not very well understood and validated.
In this paper, an integrated MCDA framework for strategic water loss management planning (SWLMP) is proposed to aid urban water utility DMs in selecting and prioritizing water loss reduction strategies. The framework methodology is applied to a typical developing country city of Kampala, Uganda in East Africa. For the case study WDS, seven evaluation criteria under five main non-commensurate and conflicting objectives that include supply reliability, cost efficiency, water conservation, affordability and public health protection were identified and examined. The SWLMP framework includes the PROMETHEE outranking method of the MCDA family and its new computer software tool Decision Sights (D-Sight). The methodology explicitly includes the actors’ preferences in the evaluation of water loss reduction strategies and as such is able to support the DMs in the decision-making process. The D-Sight’s in-built sensitivity analysis tools were used to examine the influence of actor preference weighting on the final solution ranking. The SWLMP framework has shown to be a valuable tool for facilitating the decision process of prioritizing water loss reduction options.
This paper is organized as follows. Section  2 outlines the concepts and frameworks of decision making, group decision making, MCDA and the outranking PROMETHEE method of MCDA. Section  3 presents the integrated MCDA framework for SWLMP. In section  4, the application of the MCDA framework is presented together with the results and sensitivity analysis. The results discussion and some lessons learned are presented in section  5. Finally, conclusions are drawn in section  6.

2 Multi-criteria Decision Analysis

2.1 The Decision Making Process

Decision making has been defined differently by various researchers. Harris ( 1998) formally defines decision making as “ the study of identifying and choosing alternatives based on the values and preferences of the decision makers”. According to Lu et al. ( 2007), decision making is “ the cognitive process leading to the selection of a course of action among alternatives”. It can be deduced from these two definitions that decision making involves DMs, options and selecting a final solution in a clear and transparent manner.
In real-world, most decision environments are complex demanding multiple perspectives from different people, thus decisions are often made via group decision making (GDM). A group’s decision is usually understood to be the reduction of different individual preferences, among objectives in a given set, to a single collective preference (Ghanbarpour et al. 2005). The question that arises is how best to aggregate the DM’s preference structures. Most MCDA methods have been extended to GDM as group decision support methods (GDSM) such as PROMETHEE GDSM (Macharis et al. 1998) and fuzzy multi-criteria GDSM (Lu et al. 2007). The GDSM methods provide DMs with the freedom of having their own criteria, weights, and preference structure. The DMs generate their own individual solutions which are merged in the end and analyzed collectively in the evaluation matrix table to establish a group decision result. During aggregation, each member is considered as a separate criterion and could be given a weight corresponding to his influence and/or importance in the group. However, it is usually advised to assign equal weights to all DMs to indicate that their different points of view are of equal importance (Macharis et al. 2010). Lu et al. ( 2007) suggests other GDM methods based on rules such as authority rule, majority rule, negative minority rule, ranking rule and consensus rule. The processes of GDM have been reported to improve by using the aforementioned rules with techniques such as the Delphi method (Ghanbarpour et al. 2005; Lu et al. 2007). The Delphi method aims at building an interdisciplinary consensus about an opinion, without necessarily having people to meet face to face, such as through surveys, questionnaires and emails. It is more suitable in emotional decision environments, where strong factions exist and when dealing with a pre-selected group of experts.
Some studies have also evaluated the alternatives using the group median weight value for criteria as representative of the group preference (Hokkanen and Salminen 1994; Kodikara 2008). They argue that the median agrees with the majority of the group. In their recent application of the multi-actor multi-criteria analysis (MAMCA) methodology, Macharis et al. ( 2010) used the average value of individual respondents as representative of the points of view of each stakeholder group in evaluating the impact of policy measures on challenges affecting the Flanders in Action Process. The advantage of using the median value is that it is not as sensitive to extreme values as the mean value (Keller and Warrack 2003).

2.2 Multi-criteria Methods

MCDA is a structured framework for analyzing decision problems characterized by complex multiple objectives and criteria. Guitouni and Martel ( 1998) described the MCDA methodology as a non-linear recursive process made up of four steps: (i) structuring the decision problem, (ii) articulating and modeling the preferences, (iii) aggregating the alternative evaluations (preferences) and (iv) making recommendations. It is generally used as an analytical tool but it can also be applied as an integrated framework by coupling with appropriate problem structuring methods. Mathematically, a typical MCDA problem can be modeled as follows (Brans and Mareschal 2005):
$$ { \max }\{ {{\text{g}}_{{1}}}\left( {\text{a}} \right),{{\text{g}}_{{2}}}\left( {\text{a}} \right), \ldots, {{\text{g}}_{\text{j}}}\left( {\text{a}} \right), \ldots, {{\text{g}}_{\text{k}}}\left( {\text{a}} \right)|{\text{a}} \in {\text{A}}\}, $$
(1)
where A is a finite set of possible alternatives {a 1,a 2,…,a i,…, a n} and {g 1(.), g 2(.),…, g j(.),…g k(.)} a set of evaluation criteria.
The basic data of a multi-criteria problem (Eq.  1) consists of an evaluation matrix (EM) shown in Table  1. The EM may contain a mix of ordinal and cardinal data. An MCDA model (Table  1) requires at least two non dominated decision options and discriminating criteria (Hajkowicz and Higgins 2008). Combining preference information (thresholds and weights) with performance data in the EM table provides the results.
Table 1
Multi-criteria evaluation matrix
a
g 1 (.)
g 2 (.)
g j (.)
g k (.)
a 1
g 1 (a 1 )
g 2 (a 1 )
g j (a 1 )
g K (a 1 )
a 2
g 1 (a 2 )
g 2 (a 2 )
g j (a 2 )
f K (a 2 )
 
a i
g 1 (a i )
g 2 (a i )
g j (a i )
g K (a i )
 
a n
g 1 (a n )
g 2 (a n )
g j (a n )
g K (a n )
There are various methods of MCDA. Lai et al. ( 2008) in their review paper on MCDA categorized these methods as follows:
1.
Elementary methods—are the simplest form of MCDA and are hardly applied in water resources management due to inadequacy of their simple preference models (e.g. weighted sum, conjunctive and disjunctive methods etc).
 
2.
Single synthesizing criterion approach—they reduce all criteria to a single criterion for comparison. They include multi-attribute utility theory (MAUT) methods, technique for order preference by similarity of ideal solution (TOPSIS), simple multi-attribute rating technique (SMART), Fuzzy (weighted sum and maxmin), AHP etc. These methods belong to the American school of thinking (Roy and Vanderpooten 1996).
 
3.
Outranking methods—use pairwise relations to compare actions, identifying preferences for one over the other and preference aggregation. They include novel approach to imprecise assessment and decision environments (NAIDE), ELECTRE, PROMETHEE, ORESTE, REGIME etc. These methods belong to the European school of thinking (Roy and Vanderpooten 1996).
 
4.
Goal or reference point method—identifies decision options that are closest to the ideal and furthest from the anti-deal. They include goal and compromise programming.
 
5.
Fuzzy set theory—the fuzzy set approach uses imprecise and uncertain information that provides a rigorous and flexible approach to complex resource management problems. Fuzzy set is used as a tool that can be applied to any MCDA methodologies rather than as a specific MCDA methodology itself.
 
MCDA methods are ideal for water resources management and planning and are usually preferred to other methods for various reasons including transparency and accountability to decision procedures, ability to resolve conflicts and use of formal axioms of decision theory to inform choice (Hajkowicz and Collins 2007). However, critics of MCDA say that the method is prone to manipulation, is very technocratic and provides a false sense of accuracy (Janssen 2001).

2.3 PROMETHEE II

The family of PROMETHEE methods were developed by Brans and Vincke ( 1985) to help a decision-maker rank partially (PROMETHEE I) or completely (PROMETHEE II) a finite number of options which are evaluated on a common set of non-commensurable multiple criteria. The method uses the outranking principle to rank options. The preference structure is based on pairwise comparisons. The method requires identification of different options and objectives of the decision problem. The objectives are expressed in form of evaluation criteria (EC) associated to the performance functions and weights. Once the actors have defined and weighted the EC and selected the preference thresholds, the next step is to complete the evaluation matrix (EM) table. Once these requisites are in place, then the PROMETHEE procedure can be applied (Brans and Mareschal 2005).
The preference structure of PROMETHEE is based on pairwise comparisons which is usually the deviation between evaluations of two options based on a particular criterion. The larger the deviation, the more the preference. The preference function ( P j ) translates the difference between the evaluations (i.e. scores) obtained by the two option strategies ( S n and S m ) in terms of a particular criterion, into a preference degree ranging from 0 to 1. This means that for each criterion the DM has in mind a function
$$ {P_j}({S_n},{S_m}) = {g_j}[{f_j}({S_n}) - {f_j}({S_m})] $$
(2)
and for which
$$ 0 \leqslant {P_j}({S_n},{S_m}) \leqslant 1 $$
(3)
And where g j is a non-decreasing function of the observed deviation ( d) between f j ( S n ) and f j ( S m ). The pair { g j (.), P j (S n , S m )} is called the generalized criterion associated to criterion g j (.). Such generalised criterion has to be defined for each criterion. In PROMETHEE, there are six generalized criteria functions to choose from: the usual criterion (Type 1), U-shape criterion (Type 2), V-shape criterion (Type 3), level criterion (Type 4), V-shape with indifference criterion (Type 5), and Gaussian criterion (Type 6) (Brans and Mareschal 2005). Some functions require the definition of preference and indifference thresholds, p and q respectively. For working with ordinal scale criteria in PROMETHEE, De Keyser and Peeters ( 1996) recommend cardinalisation of evaluations in combination with the use of type I preference functions.
The principle is the construction and exploitation of the outranking preference index (π). The aggregated preference index over all criteria is defined as:
$$ \pi ({S_n},{S_m}) = \sum\limits_{{J = 1}}^K {{P_J}} ({S_n},{S_m}){w_j} $$
(4)
where, w j is a weight, measuring the relative importance allocated to criterion j. The weights are supposed to be normalized, so that
$$ \sum\limits_{{J = 1}}^K {{w_j} = 1} $$
(5)
As each option faces (n–1) other options, the following positive (Ф +) and negative (Ф ) dominance flows express the dominating (strength) and the dominated (weakness) character of a strategy S n , respectively, over all the other strategies.
$$ {\Phi^{ + }}({S_n}) = \frac{1}{{n - 1}}\sum\limits_{{m = 1}}^N {\pi ({S_n},{S_m})} $$
(6)
$$ {\Phi^{ - }}({S_n}) = \frac{1}{{n - 1}}\sum\limits_{{m = 1}}^N {\pi ({S_m},{S_n})} $$
(7)
It is then possible to determine the overall score (net flow) for each option by calculating the balance of the dominance flow using Eq.  8.
$$ \Phi ({S_n}) = {\Phi^{ + }}({S_n}) - {\Phi^{ - }}({S_n}). $$
(8)
The latter gives a quality measure of the strategy. The higher the net flow (Ф) value, the better the strategy ( S n ). Accordingly, PROMETHEE II ranks the strategies in the order of their decreasing net flows. The associated D-Sight software allows sensitivity analysis to be carried out using additional tools such as “walking weights” and the decision stick. The graphical investigation of the conflicts between the criteria based on principal components analysis is enabled by means of the geometrical analysis for interactive assistance (GAIA) plane. This provides valuable visual insights into the trade-offs that need to be made by DMs.
The PROMETHEE method is easy to comprehend compared to other outranking methods and is widely applied in practice. A recent literature review identifies 217 scholarly papers based on the PROMETHEE method and about 90% were categorized as application papers solving real-life problems in water resources, environmental management, logistics and transport, forestry, chemistry, finance etc. (Behzadian et al. 2010). These methods are highly appreciated by end-users because they are easy to use, intuitive, auditable, and transparent with several graphical and interactive tools. Like all methods, PROMETHEE has some drawbacks that users must be aware of including lack of guidelines on how to determine criteria weights and it suffers from rank reversal on addition or removal of options particularly PROMETHEE II (De Keyser and Peeters 1996; Macharis et al. 2004; Mareschal et al. 2008). To avoid “black box” effects in PROMETHEE, Macharis et al. ( 2004) suggests use of a more manageable figure of seven EC and seven alternatives. However, this could be another weakness of the PROMETHEE by limiting complex MCDA problems to a few options and criteria. This weakness could be improved by hybrid methods that combine two or more methods for operational synergies such as PROMETHEE-AHP (Macharis et al. 2004), Fuzzy-AHP (Wang et al. 2011), and fuzzy multi-objective/criteria decision-making methods (Lu et al. 2007).

3 The Proposed Decision Framework

In this section we describe an integrated MCDA group decision-aiding framework for SWLMP in water utilities. The proposed decision-aiding framework is shown in Fig.  1.
Integrative framework is the problem structuring method which provides guidance for stakeholder engagement, criteria selection and alternative development (Lai et al. 2008).The method aids water utility DMs in selecting and prioritizing water loss reduction strategy options. The framework explicitly includes sustainability dimensions of WLM such as economic, technical, environmental and social. The prioritizing is enabled by PROMETHE II outranking method of the MCDA family. The framework method comprises of seven phases: (1) problem structuring phase; (2) design phase; (3) choice phase; (4) group decision phase; (5) testing phase; (6) implementation phase; and (7) monitoring phase. The application of the seven-step framework methodology in formulating water loss reduction priorities is briefly described.

3.1 Problem Structuring Phase

The aim at this stage is to come up with a clear and concise problem statement. To be able to do so requires identifying root causes, understanding the organizational context information, managerial assumptions and any related initial and desired conditions. Identification of actors and their participation in the decision process starts at this stage. Actors often referred to as stakeholders are people who have an interest, financial or otherwise, in the consequences of any decision taken (Macharis et al. 2010). In this study, actors refer to people other than the analyst involved in the decision process (DMs, experts and stakeholders). The analyst (in this case researcher) is responsible for the decision-aid process as a facilitator ensuring that the responsibility for the decision and its outcomes is of the DMs. The ultimate goal of this step is to come up with a clear problem statement agreed upon by all actors.

3.2 Design Phase

In this phase, the following decision tasks are accomplished:
Establishment of Objectives and Goals
Good decisions need clear objectives and goals. Goals are broad statements of intent and desirable long-term plans. The goals and objectives are derived from the utility’s vision and mission statements. The goals and objectives should include sustainability dimensions such as economic, environmental and social aspects. In practice, objectives are often conflicting and may be realized over a short, medium and long-term period.
Identification of Options or Alternatives
At this stage options that may contribute to the achievement of the objectives are identified. Water loss reduction options are selected after carrying out a water balance/audit. The water balance reveals the nature and magnitude of the decision problem and provides guidance on which strategy options to adopt. The strategy options can then be selected from a rich menu developed by the International Water Association (IWA) and the American Water Works Association (AWWA) based on many years of research (Alegre et al. 2006; AWWA 2009; Fanner et al. 2007). Each option should be clearly described as to how it will aid in solving the decision problem and how each differs from others.
Identification of Evaluation Criteria (EC)
In this step, criteria for evaluating performance of options are defined. The choice of the EC is based on the established objectives and the purpose of options considered. Criteria should be able to discriminate among options, complete, non-redundant, operational, measurable and minimal for sound decision making (Georgopoulou et al. 1998). Criteria should not be too many or too few. This means the selected criteria should be meaningful, relevant and completely cover all aspects of the objectives. Although large infrastructure projects may require over a hundred criteria, a more manageable typical range is six to twenty (UKDCLG 2009). Generally, actors should be engaged in developing criteria but in most cases analysts do develop the criteria to avoid influences from powerful DMs on other actors and when actors are not technically competent to generate criteria. The developed criteria should be able to measure the sustainability dimensions and are usually categorized in a similar manner as options into economic, environmental, technical and social aspects—usually in the AHP hierarchical structure.
Model Formulation
This step basically involves building the evaluation matrix (EM). Each entry in the EM represents the evaluation of an option according to its performance based on a criterion. Scoring of options is a technical activity, performed by a competent team especially when real outcomes of the decision options are often uncertain (Flug et al. 2000). The team may be composed of the analyst and experts on the subject matter. It requires experience and at times literature research especially in planning studies where outcomes are hard to predict with certainty. The EM could include quantitative (cardinal) data, qualitative (ordinal) data or both depending on the context information as most MCDA methods can work with both data. It has been urged that the presence of qualitative information in real-world problems involving environment, socio-economic and physical planning aspects, is a rule rather than an exception (Munda et al. 1994). Working with qualitative data usually requires scaling techniques of manipulating linguistic terms into interval scales. Linguistic terms reflect uncertainty, inaccuracy and fuzziness in the decision making process. Fuzzy sets coupled with MCDA could be explored to handle complex water resources planning problems—fuzzy multi-criteria methods (Lu et al. 2007).
Preference Elicitation
In this step, weights and preference functions for the EC and objectives reflecting their relative importance are defined by the actors. The actors must assign preference functions to each criterion using appropriate tools such as the generalized criteria functions in PROMETHEE (Brans and Mareschal 2005).
Weighting of criteria is subjective and has direct influence on the results of prioritizing strategy options. It is therefore critical that criteria weights are determined rationally and truthfully. Generally, there are two methods for weighting of criteria: the equal weights and the rank-order method. The equal weights method (w i = 1/n, i = 1, 2,…, n) is the most widely applied due to its simplicity. It requires minimal knowledge of the DM’s priorities and minimal input from the DM. The problem with this method is that it assumes equal importance for all criteria which is unrealistic and often not the case in practice. The rank-order method (w 1 ≥ w 2 ≥ … ≥ w n ≥ 0 and sum of all criteria weights must equal to one).
Wang et al. ( 2009), classifies the rank-order method into three categories: subjective weighting method, objective weighting method and combination weighting method. The subjective methods determine criteria weights based on the preferences of the DMs. They explain the elicitation process more clearly and are the most used for MCDA in water resources management. They include SMART, AHP, SIMOS and the Delphi method. The objective weights are obtained by mathematical methods based on the analysis of initial data. The objective weight procedure is not very clear and includes methods such as least mean square (LMS), minmax deviation, entropy, TOPSIS and multi-objective optimization. The combination or optimal weighting methods are a hybrid of methods that include multiplication and additive synthesis.
In this study, we apply the SIMOS procedure for weighting criteria (Figueira and Roy 2002). The main innovation of this procedure is relating a “playing card” to each criterion. The procedure can be summarized into four main steps as follows:
1.
Each DM is given n colored cards (or n criteria). Each card has the criterion name and its objective inscribed on it and objective of the criterion. A number of white cards (blank cards) are also provided.
 
2.
The DM is then asked to rank the cards from the least important to the most important. If certain criteria are perceived to be of equal importance (same weighting), the cards are grouped together (same rank position).
 
3.
The DMs are asked to insert the white cards between two successively ranked colored cards (or group of cards) in order to express their strong preference between criteria. The number of white cards is proportional to the difference between the importance of the considered criteria.
 
4.
The DM is finally asked to answer the question “how many times more important the first ranked criterion (or group of criteria) is, relative to the last ranked criterion (or group of criteria)?”
 
The ranking of criteria are then transformed using an appropriate algorithm that attributes a numerical value to the weights of each criterion (Figueira and Roy 2002). The main advantage of the SIMOS weighting method lies in its simplicity but yet intuitive procedure. The fact that the DM handles the cards in order to rank them, inserting white ones, allows indirect understanding of the aim of the procedure. In addition, DMs express the relative importance of criteria using ordinal preferences, allowing determining indirectly numerical values for weights. The respondents often find it easier to express their weightings on an ordinal scale rather than on a numerical scale. This procedure has been applied by other researchers in Greece (Georgopoulou et al. 1998), Austria (Madlener et al. 2007) and Australia (Kodikara 2008) and was found to be very well accepted by DMs.

3.2.1 Selecting the Decision-making Method

There are many MCDA methods for solving a decision problem. Munda et al. ( 1994) categorizes them as: (i) utility based models, (ii) outranking methods, (iii) the lexicographic model, and (iv) ideal point approaches and aspiration level models. Selecting an appropriate MCDA method based on criteria in literature and observing present institutional constraints, the selection of MCDA techniques is itself a multi-criteria problem (Abrishamchi et al. 2005). The guidelines on how to chose an appropriate MCDA method can be found in Guitouni and Martel ( 1998). Generally, selecting an appropriate MCDA method depends on the decision problem at hand, expertise, experience and the preference of the analyst or DM. Preferably, the simpler method should be chosen for the decision analysis.
There is no single MCDA method that can claim to be a superior method for all decision making problems. Different researchers have provided different views on this issue. Guitouni and Martel ( 1998) argue that different methods will yield different recommendations while Hajkowicz and Higgins ( 2008) argue that the ranking of decision options is unlikely to change markedly by using a different MCDA method provided ordinal and cardinal data are handled appropriately.

3.3 The Choice Phase

At this stage, the EM is complete with performance data of each option against each criterion. The options are then evaluated against objectives using the defined criteria, weightings of criteria, preference functions and thresholds by running the selected MCDA model. The results may be the ranking of options or a selected subset of the most promising options depending on the decision problem. The evaluation is usually of technical nature and must be executed by the analyst or any other experienced expert. Because the assignment of criteria weights is subjective and subject to errors during elicitation and in the aggregation process, it is important to verify the stability of the results by performing a sensitivity analysis to verify whether the result changes when the weights are modified. The results obtained in this phase must be recommended by the analyst to the actors for a final group decision.

3.4 Group Decision Phase

This stage is concerned with conflict resolution and reaching consensus. The analyst presents the results to the group with aid of visualization tools for clarity. Individual actor results as well as group decision results are presented including sensitivity analysis. In case of disagreements among the actors, the group could be asked to assign new weights to criteria and the evaluation repeated. If disagreements still arise, then new options and criteria could be selected and the evaluation repeated. Generally, the whole process is iterative and it is up to the analyst to determine the termination criteria in case of continued disagreements.

3.5 Testing Phase

Once the decision is made, pilot testing follows to validate whether the chosen option truly solves the identified decision problem based on real field measurements. This phase is very important in planning studies particularly when new strategy options have not been applied before in the organization and the MCDA results are based on uncertain performance predictions. In addition, implementation of water loss reduction strategies is very costly and must be based on accurate and reliable data. Depending on the findings, the earlier results of prioritized strategies could change before implementation. Once the solution is validated, it is recommended to DMs for consideration and onward implementation.

3.6 Implementation Phase

This final stage is to implement the final decision via well-scheduled action plans and available budgets. The implementation of prioritized strategy options could change along the way as more insight into the impact of different options on different components of water losses become available through the water balance. This provides flexibility during implementation. In practice, there is always some degree of overlap, where a mixture of strategies is applied concurrently to maximize the level of water loss reductions and cost optimization. For example pressure management could first be done at a macro level and later at a micro level after network zoning and District Meter Area (DMA) establishment.

3.7 Monitoring Phase

The final monitoring phase provides a continuous feedback loop on performance of each strategy based on actual field measurements and data collection to update the decision-making process. The whole MCDA framework methodology is actually iterative (Fig.  1) and should be considered as an exercise, which can always be revised and refined in the light of better data and information to support design of new options.

4 Application of the Integrated Framework Methodology

The methodology was applied to the NWSC’s Kampala Water Distribution System (KWDS) in Uganda’s capital city of Kampala. The KWDS service area encompasses an area of about 350 km 2 with an estimated population of 1.5 million and 73% water coverage. Water supply has not kept pace with population growth with consequences of irregular supply in most parts of the city. The development of the KWDS commenced in the year 1928 and water production has since been gradually increasing up to the current average level of about 148,000 m 3/day, serving about 150,000 service connections. The condition of the network has deteriorated over the years, due to poor operating practices and inadequate strategic asset management (Mutikanga et al. 2009). The major customer complaints are related to supply interruptions and low pressure as a result of frequent mains bursts and service line leakages. Apparent losses are also high with rampant illegal use of water, high metering inaccuracies and meter failure. The average number of failures reported during the year 2010 was 1175 breaks/100 km/year. This is rather very high compared to figures reported in the Netherlands of 8/100 km/year (Vloerbergh and Blokker 2009). Pipeline systems having an average annual pipe break ratio per 100 km of less than 40 are considered to be in an acceptable state (Pelletier et al. 2003). Non-revenue water averages about 43% of system input volume or 22 million m 3 per year (NWSC 2009b).

4.1 Problem Formulation for the KWDS

Based on the case study information the agreed upon problem statement was documented as “ identify and prioritize strategies to reduce water losses in KWDS”.

4.1.1 Requirements

The problem definition dictated the following key requirements for the decision problem:
1.
Strategy options should address both real and apparent losses.
 
2.
Cost of implementing a set of selected strategies should not exceed €3.6 million per year.
 
3.
Strategies should lead to a water loss reduction of at least 12 million m 3 per year.
 
4.
Implementation period to achieve water loss reduction target should not exceed 10 years.
 
Actors
The actors were proposed by the analyst and approved by the General Manger of KWDS. They were all selected from within the utility apart from one actor. Three actor groups were identified to represent utility DMs, water users and environmentalists. In total eight DMs were selected for the preference elicitation process. In this study, actors are referred to as the eight DMs (DM1 to DM8). The senior manager in-charge of the Kampala Finance and Accounts Department together with three senior managers from the Kampala Water Supply Department represented utility interests (DM1, DM2, DM3, and DM4). One Branch Manager from the Commercial and Customer Care Department together with the GTZ Technical Adviser to NWSC represented customer’s interests (DM5, DM6). The NWSC’s Water Quality Manager and the Kampala Water Production Manager represented the environmental interests (DM7 and DM8). The environmental group was a hypothetical group due to time constraints to mobilize real environmental officials from relevant government departments.
Establishing Goals and Objectives
The goals and objectives of the study were derived from the utility’s mission of “ providing safe and reliable water services to customers at a fair price and in an environmentally friendly manner”. The goal of this study was to reduce water losses in the KWDS. The goals were viewed in the broader national water sector policy of utility financial viability, environmental protection, public health protection, technically acceptable level of service and socio-economic aspects.
In light of the utility’s mission and problem statement, five main objectives were established:
1.
Maximize revenues and minimize costs.
 
2.
Maximize water savings.
 
3.
Maximize good quality water.
 
4.
Maximize water supply reliability.
 
5.
Maximize affordability of water.
 
Generating Options
In order to generate appropriate water loss reduction strategy options, a water balance was established by the utility water loss unit using the IWA/AWWA water balance methodology and the proposed methodology for assessing the apparent loss component (Mutikanga et al. 2010). The water balance for KWDS is shown in Table  2. Tabesh et al. ( 2009) applied the IWA/AWWA water balance methodology in their model for evaluating the WDS for an Iranian town. Although the methodology proposed by Tabesh et al. ( 2009) to evaluate water losses concurs with the IWA/AWWA water balance methodology, it had the following shortcomings:
Table 2
KWDS water balance for year 2010
Parameter
Water loss components
Unit
Quantity
Error Margin (95% confidence level)
System input volume
 
m 3
52,499,787
±7.0%
Revenue water
 
m 3
30,891,487
 
Non-revenue water
 
m 3
21,608,300
±15.3%
Water losses
 
m 3
21,319,631
±7.9%
 
Real losses
m 3
11,863,566
±7.5%
 
Apparent losses
m 3
9,456,065
±2.5%
 
Customer metering errors
m 3
8,726,065
±1.0%
 
Unauthorised consumption
m 3
730,000
±2.3%
  • In assessing legitimate night use during the hour of minimum night flow (MNF) they used default values (6% active population at night with 10 L/head/h) suggested by McKenzie ( 1999) for South African Cities, which are not likely to be valid for Iranian towns;
  • They introduced new terminologies of “operational error” and “management error” in assessment of apparent losses that are not part of the IWA/AWWA water balance methodology and are likely to cause confusion;
  • In assessing meter inaccuracies they did not consider user demand profiles yet meter accuracy is a function of water used at different flow rates (Arregui et al. 2011). In addition, they did not indicate which meter testing standards were used as different standards are likely to produce different meter accuracy results.
Based on the KWDS water balance (Table  2), the following seven strategy options were proposed by the analyst and accepted by the DMs.
1.
Meter replacement (S1).
 
2.
Illegal use control (S2).
 
3.
Improved speed and quality of repairs (S3).
 
4.
Selective mains and service line replacements (S4).
 
5.
Network zoning and establishing District Meter Areas (DMAs) (S5).
 
6.
Pressure management (S6).
 
7.
Active leakage control (S7).
 
Determining Evaluation Criteria
The seven EC shown in Table  3 were generated through a brainstorming session and agreed upon by all DMs as suitable for performance evaluation of the strategy options with aim of realizing the set objectives and goals.
Table 3
Evaluation criteria
Code
Criteria
Description
EC1
Revenue generation
The ability of option to improve revenue. The higher the potential, the most preferable the option
EC2
Investment cost
The cost needed to implement the option. The lower the cost, the most preferable the option
EC3
Operation & maintenance costs
The costs associated with adopting the option. The lower the cost, the most preferable the option
EC4
Water saved
The ability of option to reduce leakage. The higher the potential, the most preferable the option
EC5
Water quality
The ability of option to improve water quality. The higher the potential, the most preferable the option
EC6
Supply reliability
The ability of option to minimize supply interruptions. The fewer the frequency of bursts and leaks, the most preferable the option
EC7
Affordability
The impact of option on water tariff. The lower the impact on tariff, the most preferable the option
Predicting Performance
The evaluation matrix (EM) for the case study is shown in Table  4. Each entry in the EM represents the evaluation of an option according to its performance based on a criterion. Scoring of options was based on experience and literature research as real outcomes of the decision options will be realized in future. In addition there was lack of reliable quantitative data for some options that are currently in use due to institutional challenges coupled with database limitations. To ensure accurate and objective evaluations, the evaluation matrix was completed by a team of the utility personnel in the water loss control unit who are experts on this subject matter. Since the criteria were qualitative in nature, the strategy options were evaluated using criteria measured on a Likert Scale ranging from 1 (poor performance) to 5 (very good performance) as shown in Table  5. This being an interval scale, the intervals between statements are meaningful but scale scores have no meaning.
Table 4
Evaluation matrix
Objective
Criteria
Direction
Strategy options
S1
S2
S3
S4
S5
S6
S7
Finacial-economic
EC1 Revenue
Maximise
5
3
3
3
1
2
1
EC2 Investment Cost
Minimise
2
1
1
5
3
3
2
EC3 O & M Costs
Minimise
1
4
5
1
2
2
4
Environmental
EC4 Water Saved
Maximise
1
1
4
5
2
4
3
Public Health
EC5 Water Quality
Maximise
1
1
2
5
2
4
3
Technical (Level of Service)
EC6 Supply Reliability
Maximise
1
1
3
5
2
4
3
Socio-economic
EC7 Affordability
Maximise
5
4
1
2
2
3
2
Table 5
Transformation of linguistic terms to interval scale
Linguistic terms
Interval scale
Very poor (very low)
1
Poor
2
Fair
3
Good
4
Very good (very high)
5
Selecting the Multi-criteria Method
The MCDA method used was the PROMETHEE II and its D-Sight software tool.
Preference Modeling
The preference elicitation process in the case study comprised of an interviewer-assisted questionnaire survey to derive preference functions (PFs) and weights for the evaluation criteria (EC) and objectives. A survey was conducted on eight DMs and weights were assigned for each criterion and objective to reflect their relative importance to the decision. As the criteria were qualitative, the PF applied in this study was the type I (usual criterion) of the six generalized criteria as defined by the authors of the PROMETHEE method (Brans and Mareschal 2005). The preference thresholds can be chosen by means of the D-Sight software. In this way, a lot of flexibility is provided to represent the preferences of DMs.
Determining Criteria Weights
The weights were derived using the “Revised Simos” Procedure (Figueira and Roy 2002) and the criteria weight values are presented in Table  6. For the group decision, the median was considered as the representative value since it agrees with the majority view of the group.
Table 6
Evaluation criteria weights assigned by each DM
Criteria
Weight values
DM1
DM2
DM3
DM4
DM5
DM6
DM7
DM8
Mean
Median
C1
19
12
6
18
0
4
15
11
11
12
C2
5
11
7
5
9
0
2
9
6
6
C3
10
8
7
10
4
0
9
9
7
9
C4
26
25
14
33
7
15
12
14
18
15
C5
13
14
16
7
26
15
15
14
15
15
C6
19
20
32
20
35
38
40
24
29
28
C7
7
10
8
7
19
27
8
19
13
9

4.1.2 Evaluating Options

The water loss reduction strategy options were evaluated and prioritized with the D-Sight Software tool, which uses the PROMETHEE algorithm. The PROMETHEE II individual decision and group decision rankings are shown in Tables  7 and 8. The prioritized strategies for the group decision are graphically depicted in Fig.  2. The evaluation of options was done by the analyst as it is a task of technical nature that needs expertise.
Table 7
PROMETHEE II rankings for individual DMs and group scenario
Rank
Individual decision maker
Group decision
DM1
DM2
DM3
DM4
DM5
DM6
DM7
DM8
Rank1
S4
S4
S4
S4
S4
S4
S4
S4
S4
Rank2
S6
S6
S6
S6
S6
S6
S6
S6
S6
Rank3
S3
S3
S3
S3
S7
S7
S3
S1
S3
Rank4
S1
S7
S7
S1
S3
S3
S7
S3
S7
Rank5
S7
S1
S5
S7
S5
S1
S1
S7
S1
Rank6
S5
S2
S1
S5
S1
S5
S5
S2
S5
Rank7
S2
S5
S2
S2
S2
S2
S2
S5
S2
Table 8
Global ranking of water loss reduction strategy options
Rank
1
2
3
4
5
6
7
Strategy
S4
S6
S3
S7
S1
S5
S2
Net Flow (Φ)
0.64
0.36
0.01
−0.07
−0.2
−0.33
−0.40
Sensitivity Analysis
The sensitivity of the results was analyzed using the capabilities of the D-Sight Software in-built tools. The results are presented in Table  9. The stability intervals indicate the range in which the weight of a criterion can be changed without affecting the ranking. For example, the technical criterion with an initial normalized weight of 30.4% may be weighted between 19.7% and 44.7% (stability interval of 25%) without affecting the ranking, provided all other factors remain constant. The sensitivity of the ranking can be considered as marginal with respect to the criteria weight values assigned.
Table 9
Weight sensitivity analysis of group on strategy ranking
Criteria
Min. weight
Value
Max. weight
Stability interval
Financial-economic
19.5%
28.3%
35.4%
15.9%
Environmental
2.7%
15.8%
33.0%
30.3%
Public health
6.3%
15.8%
27.1%
20.8%
Technical
19.7%
30.4%
44.7%
25.0%
Social-economic
0.2%
9.8%
15.4%
15.2%
Group Decision-making Phase
The PROMETHEE and GAIA plane provide both descriptive and prescriptive tools and were exploited during a group meeting organized to discuss the PROMETHEE II ranking of options, sensitivity of weighting criteria and agree on a compromise solution. During the discussions, most DMs were surprised that the current strategies (S1 and S2) being implemented by the utility were ranked among the last. Although the DMs were surprised by the outcome, consensus was reached that the global prioritized ranking was a group satisfactory solution that fully represented their preferences and met the set objectives and goals. Reaching consensus was rather easy as there was no dispute on the first two best options (S4 and S6) and minor disagreements on the third ranked option (S3) and last option (S2) as shown in Table  6. As a way forward, the DMs also unanimously agreed that since scoring of options was based on ordinal data and literature research, a testing phase (Fig.  1) was warranted to validate and quantify more precisely the impact of each strategy option before the implementation phase. This will provide information required to re-construct Fig.  2 on scale. This pilot testing approach has been used to justify implementation of pressure and leakage management strategies on the Gold Coast, Australia (Girard and Stewart 2007).

5 Results Discussion

The PROMETHEE net flows of strategy options S1 to S7 are as shown in Table  8. The strategy option S4 (mains and service lines replacement) is the best and strategy option S2 (illegal use control) is the least preferred. Whereas option S2 is the most preferred in terms of investment cost (Table  4), it is the least favorable in terms of performance. All DMs ranked option S4 as the best strategy (Table  7) and six DMs out of eight, ranked S2 last. This could be explained by the criteria weights in Table  6, where the most important criterion was water supply reliability. During the group discussion, the consensus was that reliable water supply must first be available before it is stolen and measured accurately. The current strategies (S1 and S2) being applied by NWSC-Kampala, are valid based on the traditional cost-benefit analysis (CBA) method rather than on the MCDA method that includes sustainability measures and incorporate other stakeholder views such as customers.
From Table  6, the most important criteria according to the DMs preferences are supply reliability, water savings and water quality respectively. According to Christodoulou et al. ( 2010a), “ the driving forces behind pipe-replacement capital improvement projects have primarily been the mandate to safeguard the health of urban populations, the need to increase the reliability of the pipe networks and the service provided to people, as well as socioeconomic factors in relation to the cost of operations and maintenance of pipe networks”. This explanation coupled with the DMs’ preferences on EC clearly justifies why the most preferred options were S4 and S6 (pressure management) respectively. Since PROMETHEE II is a decision support tool, one could probably argue that since pressure management (PM) is less costly compared to option S4, it could be the best alternative, should there be budget constraints during implementation. Although PM is hardly applied in most utilities of the developing countries, it is now widely acknowledged that PM is the most cost effective and efficient tool for leakage reduction once pipes have been laid (Araujo et al. 2006; Nazif et al. 2010). Recent studies and research have shown that the general benefits of PM are three fold: reduction of background leakage, reduction in main break frequency and reduction of leakage flow rate (Charalambous 2008; Girard and Stewart 2007; Lambert and Thornton 2011).
The challenge likely to be faced by utility managers in implementing the best strategy option S4, is the question of which pipes to replace, when and where? The motivation to provide answers to this question has recently attracted a lot of research (Alvisi and Franchini 2009; Christodoulou et al. 2008; Christodoulou and Deligianni 2010). Although, the multi-objective procedure for optimal pipeline rehabilitation and leakage detection scheduling proposed by Alvisi and Franchini ( 2009) is a valuable tool, its application in developing countries with poor financial record keeping and no clear separation of proactive and reactive budget allocations for WDSs is doubtful. Christodoulou et al. ( 2008) and Christodoulou and Deligianni ( 2010) have developed a neurofuzzy decision-support system (DSS) that integrates risk analysis and asset management based on application of analytical and numerical methods, geographic information systems (GIS) and artificial neural networks (ANN). The DSS predicts risk of pipe failure and prioritizes which pipes to replace and where using GIS techniques to provide better visualization. Whereas the DSS has been successfully evaluated using 5-year period data sets for New York city (USA) and Limassol city (Cyprus), its application in developing countries is likely to be limited by the huge amount of reliable data required for training and re-training of the ANNs.
The third ranked strategy of improved speed and quality of repairs (S3) is more of a reactive strategy after failure has occurred. Although the first two ranked strategies are very good they don’t actually repair leaks and main breaks in practice. Strategy S3 is the only one that repairs actual failures and will always be used to supplement other strategies. The duration of real losses can be conceptually divided into awareness, location and repair times (AWWA 2009). The run time of a leak is a major factor contributing to the overall real loss volume and early detection and repair is essential. Online artificial intelligence systems have been reported to be effective tools for early detection of bursts according to a recent study carried out in the UK (Mounce et al. 2010). Improved supervision at the operational level coupled with online detection techniques offers hope for minimizing leakage in Kampala city among other options.
The fourth ranked strategy of active leakage control (S7) emerged low probably due to the fact that there is no sense in putting too much effort in detecting invisible underground leaks in a city like Kampala where the utility is overwhelmed by visible leaks everywhere. However, this is a very valuable strategy that enables utilities to address leakage proactively, in order to attain economic and sustainable levels of leakage. The sixth ranked strategy of network zoning and DMA establishment (S4) was considered of less importance probably because it is a complementary strategy to other strategies that allow leakage managers to divide the network into small discrete zones that are easy for monitoring flow and pressure measurements, to assess and quantify leakage and to enable leak detection. DMAs also require costly on-line hydraulic measurements (real-time) using wireless sensors, telemetry (e.g. SCADA) and other communication infrastructure for effective leakage management (Christodoulou et al. 2010b). Most DMs were not aware of these state-of-the-art techniques and underrated the usefulness of DMAs in WLM.
Although not determined scientifically but based on many years of experience, the general guidance provided to utilities in selecting the order of implementing real loss reduction strategies (Trow and Farley 2004), is in agreement with the developed prioritized water loss reduction strategy framework for Kampala city (Fig.  2). However, Fanner et al. ( 2007) suggest that mains replacement should always be carried out last as they require high investment costs. The decision on when to replace pipes should be based on local conditions, consideration of sustainability dimensions and preference of all stakeholders. According to Christodoulou et al. ( 2010a), proactive replacement of small-diameter pipes takes precedence over replacement of large diameter pipes. This seems to make much sense especially for Kampala city where most leaks occur on service lines and quick gains in terms of water savings are likely to be realized by prioritizing replacement of customer service lines and small distribution pipes.

5.1 Challenges and Lessons Learned

The main challenges and lessons learned during application of the SWLMP multi-criteria decision-aiding framework were as follows:
1.
The PROMETHEE MCDA outranking method is a powerful tool for solving decision problems under uncertain environment. It is very helpful in handling trade-offs between multiple criteria and conflicting decisions and is likely to break barriers hindering sustainable water loss reduction in utilities of the developing countries.
 
2.
The decision process exposed information gaps and has compelled the utility to establish a knowledge base to support future decision-making processes.
 
3.
Although the PROMETHEE method is a powerful tool, it does not provide specific guidelines for determining criteria weights and PROMETHEE II is likely to suffer from rank reversals (Macharis et al. 2004). In addition, defining the generalized criteria was not all that trivial for most participants.
 
4.
The foundation of the decision-making process is the problem structuring and design phases and their efficient execution is decisive for success.
 
5.
The D-Sight software is simple to use and its GAIA graphical descriptive interfaces were very valuable for the interactive group discussions.
 
6.
Public participation is not institutionalized in the utility which does not suit the MCDA consensus seeking-approach. This denies customers and other stakeholders chance to influence the decision process. For example road users are often inconvenienced whenever there is a pipe failure and their views could probably have added more social costs criteria that could have probably reversed the ranking.
 
7.
Although participation of more stakeholders makes the decision processes more credible, the process can be resource consuming if not well designed and carefully planned. In addition mobilizing DMs for brainstorming sessions away from their routine work was always a difficult task.
 
8.
The major drawback of the MCDA method was reducing each option to a single abstract value during data aggregation. Trade-offs between good performance on one criterion and poor performance on some other criterion often results into loss of important information. For example poor performance on water quality could be compensated with good performance on investment cost. The underlying value judgments of the aggregation procedure are therefore debatable and probably not acceptable from the public health and regulatory point of view. It is therefore critical that an appropriate non-compensatory aggregation method is selected for solving such decision problems.
 

6 Conclusions

This study presents an integrated strategic water loss management planning framework with emphasis on structuring the decision problem including careful selection of criteria and decision options. The proposed integrated MCDA framework explicitly considers sustainability dimensions of economic, environmental, technical and social objectives. The framework is capable of handling qualitative information and as such is useful for water loss reduction planning where numerical data is often lacking and imprecise. The framework includes the PROMETHEE outranking method with its D-Sight software tool in solving the decision problem.
The multi-criteria decision-aiding framework was illustrated on a real-developing world case study clearly highlighting the step-by-step group decision-making process in practice. For Kampala city, the prioritized options for water loss reduction were mains and service lines replacement followed by pressure management and improved speed and quality of repairs. The results show that the most preferred options are those that enhance water supply reliability, public health and water conservation measures. In addition, the results demonstrate that the cheapest option is not necessarily the best when multiple-criteria are considered in an explicit way. The D-Sight software proved to be a valuable tool for handling the complex water loss decision planning problem.
It can be concluded that the developed multi-criteria decision-aiding framework (SWLMP) is a valuable tool envisaged to help water utilities in evaluating and prioritizing water loss reduction strategies in urban water distribution systems particularly in the developing countries where the problem is more prominent.

Acknowledgements

This work was funded under the Netherlands Fellowship Program and by National Water and Sewerage Corporation, Uganda. The authors are grateful to all the NWSC staff and the GTZ technical adviser to NWSC who participated in the questionnaire survey and brainstorming sessions. The authors are also grateful to Cathy Macharis (Free University of Brussels), Yves De Smet and Quinton Hayes (Université Libre de Bruxelles) for their support in acquiring the D-Sight software. Finally, the authors are very grateful to the anonymous reviewers for their insightful critique of the manuscript that greatly improved the article.

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Open AccessThis is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License ( https://​creativecommons.​org/​licenses/​by-nc/​2.​0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Unsere Produktempfehlungen

Premium-Abo der Gesellschaft für Informatik

Sie erhalten uneingeschränkten Vollzugriff auf alle acht Fachgebiete von Springer Professional und damit auf über 45.000 Fachbücher und ca. 300 Fachzeitschriften.

Literatur
Über diesen Artikel

Weitere Artikel der Ausgabe 14/2011

Water Resources Management 14/2011 Zur Ausgabe