Can ISO-Defined Urban Sustainability Indicators Be Derived from Remote Sensing: An Expert Weighting Approach

Abstract

In the context of the United Nations’ “Agenda 2030 for Sustainable Development” and the presented Sustainable Development Goals (SDGs), the process of developing and agreeing on indicators to monitor the SDGs implementation becomes fundamental. In this paper, we identify indicators for the sustainable development of cities that have the greatest potential for their underlying data to be measured by means of remote sensing. We first identified existing indicators, which are derived from the International Standard ISO 37120, “Indicators for city services and quality of life”, as being partly or fully measured by the use of remote sensing, and then presented these indicators to remote sensing experts in an assessment procedure. We then investigated Multi-Criteria Decision-Making (MCDM) weighting methods to identify the most relevant quality of life indicators that can be captured by means of remote sensing techniques. We assess the remote sensing experts’ knowledge in the context of Decision Support Systems (DSS), and by means of both a questionnaire-based approach and a pairwise comparison approach. The approaches are compared with each other regarding their complexity, their potentials and limitations, and the respectively identified remote sensing based indicators. We identified three indicators related to surface characteristics as having the highest remote sensing potential. When contrasted to the results of the pairwise comparison, the questionnaire-based approach revealed high usability and confirmability. In the end, this approach enables cities’ administrations to decide which indicators they want to cover by means of remote sensing, depending on the capacities of their departments.

1. Introduction

In recent years, there has been an increasing demand for guidance on assessing possible future threats and supporting proactive decision-making in the context of (urban) sustainable development. The initialization of the plan of action “Agenda 2030 for Sustainable Development”, set off by the United Nations Organization (UN), and the provision of 17 Sustainable Development Goals (SDGs) and 169 targets, reflects the seriousness of the global situation. The introduction of sustainable development as a global normative development concept adds a challenging level of complexity. Thus, the demand for tools to assess future risks and support precautionary decision-making for sustainability is growing day by day.

In general, there are three approaches to investigating the interdependency of activities within a city and its functions and the principle of sustainability, which are in fact two different areas of research. One approach is to derive indicators and parameters from data that are used to monitor, measure and describe sustainability. The second one is striving to achieve sustainability for a city and the according implementation of measures. The third approach is to attribute a sustainable character or a sustaining effect to certain objects, groups or phenomena.

Public authorities willing to measure the sustainability of their city often encounter limitations regarding the in-situ measurements. Depending on the economic, political or topographical conditions within a city, in-situ measurements can be challenging due to financial limitations, time requirements, or lack of resources such as missing measurement devices or members of staff. Until now, it has been difficult for planning authorities to select from the cornucopia of indicators, since existing indicators are often not standardized, consistent, or comparable over time or across cities [1]. Indicators often have a strong spatial reference, and the required data are not necessarily directly available. Moreover, after selecting appropriate indicators, planning authorities still have to face the problem of how to measure the required data and how to determine the appropriate methods for the acquisition of the data that are needed for the particular indicators.

In this paper, we are dealing with the acquisition of the input data for the indicators used to measure the sustainable development of cities and municipalities. Using MDCM methods, we demonstrate how remote sensing can contribute to monitoring cities’ services and quality of life. The aim is to identify indicators pertaining to the sustainable development of cities and municipalities that can be measured using remote sensing data; namely, those indicators with the greatest remote sensing relevance.

The overall objective is to present urban planners with existing indicators that are derived from the International Standard ISO 37120, “Indicators for city services and quality of life”, and that are identified as being partly or fully measured by means of remote sensing. We do not offer new indicators or measure national, urban, green area or building sustainability by means of case studies and indicators; rather, we identify indicators for the sustainable development of communities that can be partly or fully measured by use of remote sensing.

The present study provides information regarding indicators that are best suited for assessment by remote sensing, according to the results of a remote sensing expert survey. To the end, this approach enables cities’ administrations to decide which indicators they want to cover by means of remote sensing, depending on the capacities of their department(s). In a wider context, the identification of such indicators by means of an assessment procedure involving remote sensing experts represents, in fact, one of the first steps of reorganizing sustainability monitoring procedures in general. As will be outlined in the subsequent sections, the provided information regarding indicators that are best suited for assessment by remote sensing shall lead towards an actual application case. The achieved insights shall support the development of a particular methodology, which will allow the sustainable development of urban areas to be monitored in the future.

2. Theoretical Background

2.1. Sustainable Urban Development

Sustainability refers to the adoption of practices in relation to environmental use and management that provide a satisfactory standard of living for today’s population and that do not impair the capacity of the environment to provide for and support the needs of future generations. Alternatively, sustainable development is that which meets the needs of society today without foreclosing the needs or options of the future [2,3,4]. According to Trinder, 2008 [4], the principle of sustainability regarding the use of the environment’s resources includes the notion that the outputs derived, whether they are from land, water, or air, can be produced continuously over time. It is inherent that a balance can be achieved between the rate of economic growth, the use of resources, and environmental quality, thus minimizing the risk of long-term environmental degradation. Masser, 2001 [5] stated that sustainable urban development at the local level is the process of bringing economic, community and ecological development into balance with one another. Based on the cited literature, one may recognize that sustainable urban development is seen as an active process that has to be continuously overseen by the responsible people and institutions.

2.2. Indicators for the Sustainable Development of Communities

One of the main research activities in the field of urban sustainability has been the search for measures, indicators and tools that enable sustainable urban development to be monitored. Trinder in [6] gave a good overview of indicators and measures that have been presented throughout the last two decades. The number of indicators ranges from 1 to 188 [6]. The sheer number of indicators and different approaches impede a potential operational use for city administrations that want to measure the sustainability of their respective city. For developing countries, especially, consequent monitoring and evaluation of urban change is a major issue in urban planning and management [5]. In this context, the ISO (the International Organization for Standardization) and the responsible committee ISO/TC 268: Sustainable development in communities developed a document that outlines indicators for city services and quality of life, and provides a uniform approach to what is measured and how that measurement is to be undertaken [1]. The mentioned standard defines and establishes methodologies for a set of indicators to steer and measure the performance of city services and quality of life. The indicators provided through the ISO document can be used to track and monitor the progress of cities’ performance. In total, 163 countries are members of the ISO through their national standards bodies, and the standards provided by the ISO committee generally have a high degree of popularity throughout different areas. These facts increase the likelihood of the ISO 37120:2014 actually being applied and put to operational use in the future. Thus, the indicators provided through the ISO 37120:2014 hold the opportunity to foster urban remote sensing activity within municipalities because they were developed for a holistic and integrated approach. These indicators are the starting point for the present study since they were selected to make reporting as simple and inexpensive as possible, recognizing the differences in resources and capabilities of cities worldwide [1].

2.3. Remote Sensing and Sustainability

Depending on the size of the city, the monitoring of (sustainable) urban development has to cover large areas. Data for large-scale spatial and temporal monitoring can be provided by remote sensing techniques [6]. Further, such techniques provide up-to-date and area-wide data on different aspects of the environment [7], and Weng, 2016 [6] stated that remote sensing is emerging as an essential geospatial tool in sustainability. According to Rahman, 2007 [8], satellite remote sensing offers excellent possibilities for mapping, monitoring, and managing various features of the urban environment. However, in this study, we did not focus on a particular remote sensing platform or instrument. Whether available remote sensing data consist of orthophotos, satellite images, LiDAR data, terrestrial laser scan data, radar data, or in-situ image data such as Google Street view images, they may all be a possible source for monitoring the sustainable development of cities. Additionally, in combination with ancillary data, remotely sensed data enable decision-makers to steer and measure the performance of city services and quality of life [1]. Kadhim et al., 2016 [9] provided an overview of applications of remote sensing regarding urban sustainability when integrated with the available environmental, economic, and social data.

2.4. Indicators for Sustainable Development of Communities and Remote Sensing

“Urban greenbelt (or open space) mapping, urban encroachment, growth of slums on vacant lands, urban housing, urban utilities and infrastructure, solid waste management, urban transportation and traffic planning, urban hydrology, urban cadastral and real estate, urban ecological hazards, and urban census data all can be mapped, monitored, and analyzed using remote sensing” [8].

The indicators presented in this study represent a selection of indicators from the ISO 37120:2014 standard that were identified as having remote sensing potential (

Table 1. Overview of the selected 13 indicators (CI = Core indicator; SI = Supporting indicator; PI = Profile indicator).

N°	Type	Category	Indicator	References
1	CI	Energy	Energy consumption of public buildings per year (kWh/m2)	[20,21]
2	SI	Energy	Total electrical energy use per capita (kWh/year)	[22]
3	SI	Environment	Noise pollution	[23]
4	SI	Recreation	Square meters of public outdoor recreation space per capita
5	CI	Shelter	Percentage of city population living in slums	[24,25,26,27]
6	SI	Shelter	Percentage of households that exist without registered legal titles	[28,29]
7	SI	Solid waste	Percentage of the city’s solid waste that is disposed of in a sanitary landfill	[30]
8	CI	Transportation	Kilometers of high capacity public transport system per 100,000 population	[31]
9	CI	Transportation	Number of personal automobiles per capita	[32,33,34]
10	CI	Urban Planning	Green area (hectares) per 100,000 population	[7,35]
11	SI	Urban Planning	Areal size of informal settlements as a per cent of city area	[36]
12	PI	Geography and climate	Land area (Square kilometers)
13	PI	Geography and climate	Percentage of non-residential area (square kilometers)	[37,38]

). Within the respective ISO document, the indicators are structured around themes. Besides this, the overall set of indicators is divided into three types: “core” (46), “supporting” (55) and “profile” (39) indicators. The total number of indicators is 140.

2.5. Gathering the Remote Sensing Experts’ Knowledge

The knowledge of remote sensing experts regarding remote sensing and sustainability is substantial. Several publications [4,5,6,7,9,10,11] document the well-founded basic-principles regarding the role of remote sensing in the context of monitoring (sustainable) urban development. However, these and other publications, as well as the introduction of numerous different indicators, also demonstrate that there is still a gap of knowledge between remote sensing and sustainable urban development and a need for further research in this field. In this study, we verified whether expert knowledge can be used to identify remote sensing relevant indicators for monitoring cities’ services and quality of life (QoL) and whether these indicators can be ranked according to their relevance of being remotely sensed. Experts—independent from their field of research—often act in an advisory function to municipalities or stakeholders in general. The procedure for decision-making that includes all stakeholders in a development process has been presented by Azapagic and Perdan, 2005a [12], 2005b [13]. This decision-making process is built on the concept of multiple criteria decision analysis, which is described in some detail [4].

2.6. Multi-Criteria Decision-Making

Multi-Criteria Decision-Making (MCDM) and Multi-Criteria Evaluation (MCE) have represented well-investigated study area for decades. MCDM techniques refer to the research field of Decision Support Systems (DSS) and incorporate a set of alternatives that are evaluated based on conflicting criteria. As noted by Malczewski (1999) [14], MCDM problems incorporate the following six components:

• A goal or a set of goals the decision maker (interest group) attempts to achieve;
• The decision maker or group of decision makers involved in the decision-making process along with their preferences with respect to evaluation criteria;
• A set of evaluation criteria (objectives and/or attributes) based on which the decision makers evaluate alternative courses of action;
• The set of decision alternatives, that is, the decision or action variables;
• The set of uncontrollable variables or states of nature (decision environment); and
• The set of outcomes or consequences associated with each alternative attribute pair [15,16].

The workflow of a Multi-Criteria Decision Analysis (MCDA) comprises three phases that interact with each other: (1) Intelligence Phase; (2) Design Phase; and (3) Choice Phase (Figure 1).

(1)

The Intelligence Phase focuses on defining the problem definition that incorporates the evaluation criteria and constraints (restrictions). MCDM problems are hierarchically structured and consist of an overall object and several sub-objectives. Each sub-objective can be further divided into additional sub-objectives and finally their criteria at the bottom level. The structure of the problem definition always depends on the research question and the application domain. The selection of evaluation criteria can be realized by examining relevant literature, incorporating domain experts, and conducting analytical studies or surveys.

(2)

Within the scope of the Design Phase, the decision matrix can be generated for each hierarchical level that includes the criteria, the alternatives, and the criterion weights. In general, weights express the preferences of the decision makers for the criteria that are incorporated in the decision model and represent the relative importance of each criterion with respect to others. High weight values indicate a strong influence of the particular criterion. Mathematical methods to retrieve the set of criterion weights are ranking-, rating-, pairwise comparison-, trade-off analysis- or entropy-based criterion weight methods, which can be further divided into global- and local methods [14,17,18].

(3)

In regard to the Choice Phase, the estimated criterion weights are used as input to apply specific decision rules to rank the alternatives with respect to their performance. Additionally, sensitivity analysis methods have to be performed to verify the robustness and stability of the decision model outputs. The result of the decision-making process is a recommendation for future action.

During this research, MCDM weighting methods were investigated to identify the most relevant QoL indicators that can be captured by means of remote sensing techniques. A detailed description of adapted MCDM weight methods is outlined in Section 3.

3. Methodology

The objective of this study was to gather and access the comprehensive knowledge of remote sensing experts on a global scale. Referring to the list of the 13 preselected indicators (cp. Indicators for sustainable development of communities and remote sensing), we aimed to identify the indicator with the highest remote sensing potential. Additionally, we sought to rate each indicator regarding its remote sensing relevance, as given by the remote sensing experts. We gathered comprehensive expert knowledge in two ways (Figure 2):

• Questionnaire: online survey
• Analytical Hierarchy Process (AHP) after Saaty, 1980 [19].

It is worth mentioning that, within the online survey, we used a combination of the ranking and rating methods [14,18] using the scale of the AHP introduced by Saaty, 1980 [19]. The reason for this and the actual implementation are described in the following sections. Data handling and preparation are presented in Section 3.2.1. The received answers to both the questionnaire-based approach (online questionnaire) and the pairwise comparison approach (analytical hierarchy process) were analyzed. After the evaluation, the results of both approaches were compared with each other.

3.1. Selection of the QoL Indicators with Remote Sensing Potential

We selected 13 indicators from all three types (core indicator, supporting indicator, and profile indicator). One may argue that such preselection regarding the remote sensing potential and the remote sensing feasibility strongly depends on the personal experience and opinion of the particular remote sensing expert. However, the presented indicators are detailed in various publications, and most have been part of scientific research in the field of remote sensing in other respects (Table 1).

The following list provides the indicators and their respective description [1]:

• Energy consumption of public buildings per year (kWh/m²): Total use of electricity at final consumption stage by public buildings (kWh) within a city divided by total floor space of these buildings in square meters (m²).
• Total electrical energy-use per capita (kWh/year): Total electrical usage of a city in kilowatt-hours including residential and non-residential use divided by the total population of the city.
• Noise pollution: mapping the noise level $L_{den}$ (day-evening-night) likely to cause annoyance as given in ISO 1996-2:1987, identifying the areas of the city where $L_{den}$ is greater than 55 dB(A), and estimating the population of those areas as a percentage of the total city population.
• Square meters of public outdoor recreation space per capita: The square meters of indoor public recreation space divided by the population of the city.
• Percentage of city population living in slums: The number of people living in slums divided by the city population.
• Percentage of households that exist without registered legal titles: The number of households that exist without registered legal titles divided by the total number of households.
• The amount of the city’s solid waste that is disposed of in a sanitary landfill in tons divided by the total amount of solid waste produced in the city in tons.
• Kilometers of high capacity public transport system per 100,000 population: The kilometers of high capacity public transport systems operating within the city divided by 1/100,000th of the city’s total population.
• Number of personal automobiles per capita: Total number of registered personal automobiles in a city divided by the total city population.
• Green area (hectares) per 100,000 population: Total area (in hectares) of green in the city divided by 1/100,000th of the city’s total population.
• Areal size of informal settlements as a percent of city area: The area of informal settlements in square kilometers (numerator) divided by the city area in square kilometers.
• Land area (Square kilometers).
• Percentage of non-residential area (square kilometers).

3.2. Questionnaire-Based Approach: The Online Survey

This section outlines the concept and implementation of the questionnaire-based approach (Appendix A, Figure A1, Figure A2, Figure A3, Figure A4, Figure A5, Figure A6 and Figure A7) to identifying the most relevant QoL indicators with remote sensing potential. MCDM weighting techniques such as ranking and rating methods were adapted to achieve this goal. A detailed explanation is provided in the following subsections.

3.2.1. Concept Development

The result of the proposed workflow (Figure 3) represents a collection of weight samples that allows the indicators to be prioritized according to the average value of all weight samples. Furthermore, it allows the weight range for each remote sensing indicator to be determined, which refers to the uncertainty of the experts’ preferences.

Figure 3 illustrates the calculation of the weight values for the 13 remote sensing indicators. This collection of indicators has to be ranked by each expert according to their preferences and relevance. Afterwards, the most important indicator will be chosen for rating the remaining indicators with respect to the scale of Saaty (1980) [19]. Saaty [19] developed the AHP that is based on the pairwise comparison of the criteria for each hierarchical level in MCDM models. The advantage of this method is that just one comparison at a time has to be performed by the experts. Nevertheless, for a higher number of criteria or indicators, the number of comparisons increases dramatically, which makes this procedure imperative, as the consistency of the judgments is difficult to maintain. The number of pairwise comparisons ($n_{pwc}$) is given in Equation (1), where $n$ represents the number of indicators.

$$n_{pwc}=\frac{n\left(n-1\right)}{2}$$

(1)

Therefore, the number of comparisons is reduced to $n-1$ ratings according to the predetermined scale of 1–9. The weight-assessing workflow uses the reciprocal scale values for comparing the indicators and performs the ratio estimation procedure, where each reciprocal indicator value ($1/x_k$) is normalized by the sum of all reciprocal indicator values incorporating the self-comparison of the most important indicator (Equation (2)).

$$w_k= \frac{\frac{1}{x_k}}{{\sum }_{k=1}^n\frac{1}{x_k}}$$

(2)

The computed values are the weights of the remote sensing indicators expressed as percentages that refer to the experts’ preferences. Subsequently, the experts have the opportunity to change the weight percentages of the indicators. If no changes are required, then the computed weight sample will be added to the collection incorporating all weight samples. Otherwise, the weight changes conducted by the experts provoke further calculation and validation procedures. For the recalculation of the unchanged weight values, the number of changes $n_c$ and the summed weight percentage value $s_{mwd}$ (sum of modified weight differences, see Equation (3)) regarding the difference of the weight values $w_k$ from the modified weight values ${w\prime }_k$ has to be determined.

$$s_{mwd}= {\displaystyle \sum }_{k=1}^{nc} w_k - {w\prime }_k$$

(3)

The modified weight values ${w\prime }_k$ of the unchanged indicators are computed according to Equation (4). The proportional change concerning the sum of modified weight differences is added to the kth initial weight value of the respective indicator.

$$w{\prime }_k = w_k + \frac{w_k}{{\sum }_{k=1}^{n-nc} w_k}*s_{mwd}$$

(4)

3.2.2. Analysis of the Responses from the Online Questionnaire

Each weight sample will be added to the collection of weight samples and afterwards the statistics of the weight collection will be calculated. The average weight values can be used for conducting the aggregation by using a decision rule, such as simple additive weighting, ordered weighting averaging, ideal point or analytic hierarchy process. The weight ranges for each indicator can be used to perform the sensitivity analysis to verify the robustness of MCDM models. Finally, relevant remote sensing indicators have to be selected by the project expert team.

Initially, the collected responses from the participants are evaluated regarding their completeness. In the first step, only duly completed questionnaires are processed and validated with respect to their plausibility.

For the remaining 33 responses, ranked indicators are compared with the weighting of the highest ranked indicator within each response using Kendall’s tau-b statistic [39]. Kendall’s tau is a rank correlation statistic, that is, a particular case of the general correlation coefficient based on the ranks of the data. Therefore, it is suitable for measuring the strength of an ordinal association between two variables. While multiple rank correlation statistics exist, Kendall’s tau has been shown to yield a comparably small gross-error sensitivity and small asymptotic variance, entailing greater robustness and efficiency than comparable statistics [40]. In addition to the estimation of this rank-based measure of association, we test for the significance of Kendall’s tau-b on a two-sided test of the null hypothesis that ranking and rating are independent [41]. All responses that either yield a negative rank correlation coefficient or whose null hypothesis is not rejected (at a significance level of 0.1) are excluded from the successional analysis to foster the consistency within the responses.

From the ranking of the indicators, and from the rating scores given by the participants, we concluded on the weightings. According to the methodology presented in Section 3.2.1, all weights are added to the collection of all weight samples. The statistics regarding the list of all weight samples are calculated. The overall ranking is calculated as a sum of the individual rankings from each participant. The frequency of occurrence is calculated for each indicator based on its position within the ranking. The lower the rank, the higher the remote sensing relevance; e.g., the indicator “Land Area” was ranked the most relevant indicator by participant ID109; its position within the ranking is (1). At this point, it is worth mentioning that a certain rank does not indicate a quantitative indication of its importance rather than a qualitative one. In the next step, the average weighting values and the ranges are calculated. The weight ranges are defined by the minimum and the maximum values of the corresponding indicator weight as well as the standard deviation.

3.3. Pairwise Comparison Approach: AHP

This section focuses on the concept and implementation of the pairwise comparison approach for identifying the most relevant QoL indicators with remote sensing potential. A detailed description is provided in the following subsections.

3.3.1. Concept Development for the AHP Matrix Calculations

Our second approach is to apply the AHP after [19] for assessing the remote sensing experts’ knowledge. The 13 selected QoL indicators are used to conduct the pairwise comparison procedure. Additional to the weight computation, validation and correction of the pairwise comparison results according to [42] are used to ensure the consistency within the scope of the AHP (Figure 4).

The AHP approach incorporates: (i) the development of the pairwise comparison matrix; (ii) the calculation of the criterion weight values; and (iii) the computation of the consistency ratio, which represents a quality measure regarding the consistency of the pairwise comparisons assessment by the experts.

(i)

For each hierarchical level of the MCDM model, a squared matrix is generated that incorporates n² elements, which represent all possible pairwise combinations for the criteria of a certain hierarchical level. The diagonal of the matrix and all elements below the diagonal are excluded for retrieving the pairwise comparison values from the involved experts. Therefore, the number of comparisons is reduced to $n\left(n-1\right)/2$ actual pairwise comparisons. The experts have to rate their preferences according to the scale of [19], where the values range from 1 (same importance) to 9 (extreme importance). The diagonal of the matrix will be filled with ones, because self-comparisons must be equally rated. Finally, the matrix below the diagonal are filled with the reciprocal values of the expert preferences by changing the values of the row and column indices of the matrix.

(ii)

Concerning the calculation of the weights, the initial pairwise comparison matrix is multiplied by itself. For each row, the sum is calculated and normalized. The normalized values represent the calculated eigenvector from the pairwise comparisons of relative importance. This procedure is performed for several iterations until no changes of the criterion weights can be identified anymore. The number of iterations depends on the desired accuracy (the number decimal place). An approximation approach for calculating the criterion weights can be found in Malczewski, 1999 [14].

(iii)

This step illustrates the consistency check of the pairwise comparisons. Each column of the initial pairwise comparison matrix is multiplied with the corresponding element of the weighted sum vector (Step ii). Afterwards, for each row, the sum is computed (weighted sum vector). During the next task, the consistency vector has to be determined by dividing the weighted sum vector by the criterion weights (Step ii). The average value $\lambda$ of the consistency vector is greater or equal (full consistency) to the number of criteria and will be normalized by the following formula, representing the Consistency Index (CI):

$$CI=\frac{\lambda -n}{n-1}$$

(5)

Finally, the Consistency Ratio CR (2) can be computed by dividing the Consistency Index by the Random Index (RI) that represents the consistency index of a randomly generated pairwise comparison matrix [14].

$$CR=\frac{CI}{RI}$$

(6)

The calculated consistency ratio has to be lower than 0.1; otherwise, it indicates an unacceptable level of inconsistency in the pairwise comparison judgments.

The advantage of the AHP is that only one comparison is regarded at a time, which facilitates the assessments of the experts. One disadvantage is represented by an increasing number of criteria; the higher the number of criteria, the higher the number of pairwise comparisons. For example, a decision problem incorporating 13 criteria requires 78 pairwise comparisons, which is not practical anymore and the probability of making mistakes during the pairwise comparison process increases. Furthermore, the scale from 1 to 9 does not allow a subtle distinction, because of the narrow scale range in comparison to other weighting methods. However, according to Malczewski and Rinner, 2015 [18], this weighting procedure is the most often applied method for calculating the criterion weights in the domain of MCDM. Malczewski and Rinner, 2015 [18] provided an overview of MCDM-based applications incorporating the pairwise comparison weighting strategy.

3.3.2. Analysis of the Responses from the Pairwise Comparisons (Analytical Hierarchy Process)

The probability of committing errors during the pairwise comparison procedure tends to rise with an increasing number of indicators. For this research, 13 indicators were used to perform the pairwise comparison procedure, incorporating 78 comparisons. The consistency ratio is a measure of quality concerning the judgments of the experts. Davoodi, 2009 [42] proposed a strategy to analytically compute a consistent AHP matrix from an inconsistent one. According to Davoodi, 2009 [42], the adjustment of an inconsistent AHP matrix is represented as follows:

(1)

For each row of the inconsistent AHP matrix, a consistent AHP matrix will be generated.

(a)

The row vector according to the row index i is copied to the consistent AHP matrix.

(b)

The reciprocals of the row vector are inserted into the column vector by changing the order of the row i and column index j.

(c)

The values of the diagonal w[i, j] are equal to one (indices i and j are equal).

(d)

The remaining elements of the consistent AHP matrix are calculated as follows w[i, j] = w[i, k] × w[k, j], where k represents the index of the initial row (step a).

(2)

All intermediate matrices, 13 in our case, are combined with the help of the geometric mean. The final matrix represents a consistent AHP matrix. For all intermediate matrices and the final matrix, the consistency is equal to zero, and all eigenvalues are equal to the number of indicators.

For each expert evaluation, a consistent AHP matrix has to be generated, and the geometric mean of those matrices has to be computed in the context of group decision-making. The resulting weights of the indicators indicate the relative importance incorporating all judgments of the experts. Five experts are asked to perform the pairwise comparison of the 13 preselected indicators. Every participating remote sensing expert has to conduct all 78 comparisons by using the scale of [19] to obtain the overall remote sensing potential for each QoL indicator. Based on the results obtained from these pairwise comparisons, the original AHP matrix and the consistent AHP matrix according to [42] are established, and the weights as well as the consistency ratio are derived.

Finally, we investigated the consistency between the online questionnaire and the AHP approach. This was done by performing a rank correlation test for Kendall’s tau-b as well as a Wilcoxon rank sum test on the questionnaire responses and the AHP responses. The Wilcoxon rank sum test is a nonparametric test for testing the null hypothesis that the underlying distributions of two independent samples have the same distribution [43].

4. Results

The results of both the questionnaire-based approach and the pairwise comparison approach are initially analyzed separately—following the presented methodologies—and then compared with each other.

4.1. Evaluation of the Results of the Online Questionnaire

In total, 83 remote sensing experts contributed to the online questionnaire: 33 remote sensing experts duly completed the survey, and 50 participants did not duly complete the survey. In total, more than 400 years of professional experience of remote sensing experts were gathered by the survey. As mentioned in Section 3.2.2, answers of duly completed questionnaires (33) that do not meet the requirements established via Kendall’s tau-b rank correlation statistic and the corresponding hypothesis test are dropped (Figure 5), leaving a remainder of 23 answers.

According to the validation procedure for the rating values of the 13 indicators, only 23 samples were considered for the final weight computation. The analysis of all remaining duly completed online questionnaires (23) yielded the following result based on the average weighting for all indicators: The indicator considered to be the third most relevant one is the supporting indicator (type) “Square meters of public outdoor recreation space per capita”. The remote sensing experts ranked the supporting indicator “Land area (square kilometers)” as the second most relevant quality of life indicator. The indicator “Green area (hectares) per 100,000 population” was ranked the most relevant indicator. This means that the core indicator “Green area (hectares) per 100,000 population” was considered the most relevant quality of life indicator that can be captured by means of remote sensing techniques. The average weighting for the 13 indicators based on all duly completed questionnaires (23) is presented in Figure 6. They are also compared with each other regarding the variations in weightings for the same respective indicators by the different experts.

4.2. Evaluation of the Results from the Pairwise Comparison (AHP Approach)

The second approach to obtaining the weight values was implemented by using the AHP procedure, which was conducted by five remote sensing experts. The AHP procedure—incorporating 78 pairwise comparisons—is a highly time-consuming weighting method and the probability of committing errors rises with an increasing number of comparisons. Therefore, all original AHP matrices were adjusted by generating consistent AHP matrices following the method of [42]. Two original AHP matrices had a consistency ratio lower than 0.1, and two original AHP matrices represented a slightly higher consistency ratio.

Table 2. The computed weight values for the 13 indicators in respect to the questionnaire-based approach and the AHP procedure.

ID	Indicators	Weights of Questionnaires	Weights of AHP Procedure
1	Energy consumption of public buildings per year (kWh/m2)	4.75	2.09
2	Total electrical energy use per capita (kWh/year)	5.39	1.88
3	Noise pollution	5.55	1.47
4	Square meters of public outdoor recreation space per capita	11.28	12.72
5	Percentage of city population living in slums	6.44	5.78
6	Percentage of households that exist without registered legal titles	3.76	4.60
7	Percentage of the city’s solid waste that is disposed of in a sanitary landfill	4.89	3.28
8	Kilometers of high capacity public transport system per 100,000 population	7.53	4.16
9	Number of personal automobiles per capita	6.01	2.20
10	Green area (hectares) per 100,000 population	14.51	18.73
11	Areal size of informal settlements as a per cent of the city area	9.14	9.58
12	Land area (square kilometers)	13.04	21.82
13	Percentage of non-residential area (square kilometers)	7.71	11.70

and Figure 7 illustrate the weights of the indicators for both methods, the questionnaire-based approach and the AHP procedure. The higher the weight values, the higher the remote sensing relevance for a specific indicator. The third column of Table 2 shows the weight values for the questionnaire-based approach, and designates that indicators “Green area (hectares) per 100,000 population” (core indicator), “Land area (square kilometers)” (profile indicator) and “Square meters of public outdoor recreation space per capita” (supporting indicator) outperform the remaining indicators.

The results of the AHP procedure provide few differences compared with the questionnaire-based approach. Particularly, the outperforming indicators “Land area (square kilometers)” and “Green area (hectares) per 100,000 population” differentiate stronger from the remaining indicators. Additionally, the indicator “Percentage of non-residential area” was rated as remote sensing relevant.

5. Discussion

In the last three decades, only a small number of remote sensing expert surveys have been conducted [44,45,46,47]. Referring to the number of duly completed online questionnaires, this research shows that it is challenging to motivate experts to participate in online surveys and surveys in general. The authors are aware that the research question, its implementation within an online survey, and the final presentation to the participants is difficult and has a strong influence on the received (number of) answers. However, the evaluation shows that most participants who did not complete the survey already abandoned it when confronted with the ranking of the preselected indicators, or even earlier, right after the welcome page, when confronted with the question about their professional background and their professional experience. In addition, the implementation of a pairwise comparison including a large quantity of pairs within a user-friendly and adequate timeframe is challenging. In this context, the contribution of the remote sensing experts to this survey and the evaluation of their duly completed answers represent a unique opportunity for the whole remote sensing community.

The remote sensing experts that conducted the remaining duly completed surveys (23) come from different professional fields, namely:

• (14) Academic university: i.a. PhD students, university professors
• Academic—extra-faculty (non-university) research institutes
• Engineering office/Engineering consultant: i.a. one software developer
• Public service

5.1. Proposals for Supplementary Indicators (Online Questionnaire Approach)

We tended to raise the user’s acceptance through a comprehensive explanation of the task(s) and the objective of the study. We enabled the possibility to do (multiple) modifications of the calculated weights; only three remote sensing experts used this possibility.

Beyond that, the participating remote sensing experts also had the opportunity to state the absence of an indicator and thus provide indicators of their own (

Table 3. Indicators that were also considered to be remote sensing relevant mentioned during the online questionnaire and their ISO 37120 equivalents.

Indicators Provided by the Remote Sensing Experts	Adequate Equivalent Indicators Found in the ISO 37120
Public transport capability (using street network/building blocks as proxy)	Kilometers of light passenger public transport system per 100,000 population (core indicator)
	Annual number of public transport trips per capita (core indicator)
Population density per km2 estimated by the size/heights of the buildings	Population density (per square kilometer) (profile indicator)
Potential for solar energy on roofs within the city	The percentage of total energy derived from renewable sources, as a share of the city’s total energy consumption (core indicator)
3D city models for sustainable planning of green/smarter cities	no matches found
Urban ventilation corridors (km2)	Fine particulate matter (PM2.5) concentration (core indicator)
Water pollution	Percentage of the city’s wastewater that has received no treatment
	Percentage of city population with potable water supply service
Potential of building roofs to be used for solar energy production (roof’s area, aspect, slope, shading situation due to neighboring buildings, vegetation, terrain)	The percentage of total energy derived from renewable sources, as a share of the city’s total energy consumption (core indicator)
Population would be necessary for many parameters, land use/zoning would be useful	no matches found
Average height of construction	no matches found

). Occasionally, the remote sensing experts also included or indicated the method of measuring these indicators within such a statement. Although the indicators that were provided in this way were not included for the remaining survey, we consider this information to be valuable, since the remote sensing experts could additionally give insights into their expertise and might have provided indicators that could be considered in the future. These indicators were evaluated regarding contextual overlaps with indicators provided by the ISO 37120 document.

From feedback regarding the questionnaire that we received prior to conducting the investigation, we knew that (remote sensing) experts would often prefer to give their expertise more directly and work with methods and indicators they have already applied rather than ranking indicators that they did not choose themselves. We strove for a user-friendly survey by enabling the possibility of making comments or remarks in two different places within the online survey. It would go beyond the scope of this research to discuss all indicators that were specifically provided by the remote sensing experts. Most have an adequate equivalent within the ISO document, as shown in Table 3. We found it of particular interest that “potential of solar energy” was suggested twice as an additional indicator; i.e., that this indicator was stated to be absent in the survey by two independent experts. In fact, this indicator as such is not mentioned in the ISO document. The reason for its absence may be related to the fact that not all cities globally are suitable or favored for the usage of solar energy, as it is only one renewable source of energy. Cities that are unsuitable due to factors that are beyond their control, such as the city’s location (degree of latitude), the predominant terrain or the prevailing climate, would simply be discriminated against.

5.2. Identification of the Indicators That Are Best Suited for Assessment by Remote Sensing

Some scatterplots show an assignment of discontinuous values by the participants, i.e., that certain jumps appear within one plot (Figure 5: e.g., IDs 80, 93, 95, 112). As mentioned before, these responses could only partly be included in the analysis. We assume that those remote sensing experts who assigned the values in such differentiated manner were challenged by the pairwise comparison procedure. The use of both the ranking and the rating methodology allows for evaluation of either an inconsistent ranking result or an implausible rating. The combined use of these methodologies facilitated conclusions regarding such conspicuous responses to be drawn.

Other scatterplots show an inverted rating, i.e., that in contrast to the ranking, high values were used for the comparisons at the beginning, and lower values at the end (Figure 5: IDs 39, 41, 43). Such a rating indicates either that the whole comparison task was misunderstood or that the stated values (1–9) were inverted, indicating the overall relevance or the importance of an indicator, respectively. For the sake of supplementary evaluation purposes, we produced a heatmap of the expert assessments that highlights the number of ratings for each indicator and rank (Figure 8).

For indicator 13, “Percentage of non-residential area (square kilometers)”, this heatmap shows that many experts similarly assessed this indicator to rank 3. However, because no rating exists on rank 1 or 2 for this indicator, it was finally outperformed by indicator 4, “Square meters of public outdoor recreation space per capita”, which landed on the 3rd rank.

This study reveals the indicators for the sustainable development of communities (ISO 37120 document) that hold the highest potential for their underlying data to be acquired by means of remote sensing. According to remote sensing experts from different professional fields and different regions of the world, the following most relevant indicators could be identified (

Table 4. Indicators for monitoring city services and quality of life that have the highest remote sensing potential according to the remote sensing experts.

Rank	Identified Indicators by Means of the Online Questionnaire Approach	Identified Indicators by Means of the Pairwise Comparison Approach (AHP)
1	Green area (hectares) per 100,000 population	Land area (square kilometers)
2	Land area (square kilometers)	Green area (hectares) per 100,000 population
3	Square meters of public outdoor recreation space per capita	Square meters of public outdoor recreation space per capita

).

5.3. Pros and Cons of the Assessment Approaches

The chosen assessment approaches (AHP and questionnaire-based online survey) generally led to similar results. Nevertheless, the time and effort required for both approaches varied. In

Table 5. Advantages and disadvantages of the online questionnaire approach and the pairwise comparison approach.

	Online Questionnaire Approach	Pairwise Comparison Approach (AHP)
Advantages	Fewer comparisons Thus, less time-consuming than the standard AHP Less prone to errors Correlation and slope of the resulting scatterplots indicate the plausibility of the results and whether questions might have been misunderstood by the participant More flexibility because of the possibility of making changes to the weights during the survey	Each indicator is directly compared with all other indicators The consistency can be determined Errors that occur during the pairwise comparison could be immediately corrected
Disadvantages	Requesting the ranking and the rating is time-consuming Thus, higher withdrawal rate Lack of confirmability for some (remote sensing) experts—the task might still be too difficult Only one indicator is compared with all other indicators (reduced number of comparisons)	Too many comparisons for 13 indicators (78 in total) Time-consuming Error-prone Low user-friendliness resulting in few responses

, the advantages and disadvantages of the chosen approaches are critically discussed.

Alternatively, a grouping of particular indicators would have been possible according to the typology (e.g., core-, supporting- and profile indicator) or their correspondence to certain thematic groups (e.g., environment, energy, transport, etc.). However, this procedure also has some disadvantages and, in the case of a standard pairwise comparison, the number of comparisons remains high. Moreover, a grouping of already very conceptional indicators may also additionally impede comparisons of such.

5.4. Comparison of the Approaches (Online Questionnaire vs. AHP)

The results for both the online questionnaire and the AHP approach were demonstrated to be plausible. Considering the disadvantages of the AHP, such as duration or error-proneness in relation to an increasing number of comparisons, we tested the consistency of the results derived by the online questionnaire and the AHP (Figure 9).

The experts who conducted the AHP procedure were more homogeneous in their ratings than experts involved in the questionnaire-based approach. The results of Kendall’s tau test clearly indicate that the two results are not independent, i.e., that there is a significant rank correlation between the results of the online questionnaire and the AHP approach. In addition, the null hypothesis of a Wilcoxon rank sum test performed on the two samples could not be rejected, indicating that there is not sufficient evidence against the hypothesis that the underlying populations have the same distribution. The usability and the confirmability of the online questionnaire (results) are thus demonstrated.

5.5. Potential Challenges in Applying of the Indicators that Are Best Suited for Assessment by Remote Sensing

The successful identification of indicators with high remote sensing potential represents the basis for the elaboration of a suitable methodology in an actual application case. However, the indicators Green Area, Land Area, and Public Outdoor Recreation Space strongly differ in their degree of complexity and demand specific approaches regarding the measurement of their underlying data. While Green Area and overall Land Area may be well distinguished by remote sensing, identifying Recreation Space and distinguishing it from other uses of, e.g., Green Area, and within non-green Land Area solely based on remote sensing data would be—without a doubt—a challenging task. In this section, each indicator is discussed in terms of its actual application and related literature is presented.

The study of green areas and their actual size within cities has a long tradition within the field of urban remote sensing [7,35,48] and urban ecology [49]. In popular science publications (e.g., periodicals) as well as in scientific publications, urban green space is recognized as one of the most important indicators for the quality of life within a city and strongly associated with citizens well-being in general. Even urban administrations usually promote their city with its amount of green space. Although there is an upcoming discussion about the different qualities of urban green space regarding accessibility, visibility and impact on health and environment of urban green spaces [23,49,50,51,52], the general positive influence of green spaces on citizens is mostly undisputed [51]. Several studies [7,35,48,53] have successfully investigated mapping and measuring urban green space in the field of remote sensing.

The size, extent and physical growth of cities have been widely discussed in the respective literature [31,54,55,56,57,58]. Since the boundary between the city and the countryside often appears to be lifted, questions have been raised such as where does the city begin and where does rural land end. Several studies [58,59,60,61,62,63] developed methodologies to detect and observe the boundary between urban and rural areas and to describe urban forms in general, thus helping to illustrate the urban–rural dichotomy, as discussed by the United Nations Statistics Division, and to contribute to the study of urban form and sprawl. Therefore, it can be stated that the identified indicator Land Area relates to well-studied remote sensing methodologies.

In contrast to the indicators mentioned before, the monitoring of public outdoor recreation space or recreation space in general has hardly been conducted within the remote sensing community [64]. Nevertheless, in a wider context, remote sensing techniques have been used to investigate visitors’ perceptions of urban parks [65]; or remotely sensed data have been used for to estimate of the social benefit of urban recreational spaces and green spaces [66]. The fact that the indicator Public Outdoor Recreation Space addresses the legal status of space, the use of such space, and its accessibility distinguishes the indicator Public Outdoor Recreation Space from the indicator Green Area, the analysis of green space and its detection by means of remote sensing. The identification and monitoring of recreation space usually requires on-the-ground observation, at least for calibrating the data from a remote sensing platform. Further, the validation of the remotely sensed data often requires ancillary data as reference for validating these data. In addition to this, the difficulties of distinguishing land use and land cover have been a tender subject throughout the last decades. Several approaches, such as the Local Climate Zones (LCZs) [67] or the Urban Structure Types (USTs) [68,69] approach addressed these challenges by presenting certain classification schemes that mostly rely on remotely sensed land cover data. Thus, these challenges would have to be tackled as soon as the indicator Public Outdoor Recreation Space is applied.

6. Conclusions

In the context of the International Standard ISO 37120, “Indicators for city services and quality of life”, this research provides information regarding indicators that are best suited for the assessment by remote sensing. Remote sensing experts with different levels of experience and from different professional fields contributed to this research. Their expertise was used to identify the indicators with a high remote sensing potential. Two approaches were used to achieve the objective of identifying the indicators for measuring city services and quality of life that have the highest remote sensing potential: “Green area (hectares) per 100,000 population”, “Land area (square kilometers)” and “Square meters of public outdoor recreation space per capita”. Those indicators that are not suited for analysis by means of remote sensing could also be identified; e.g., “Percentage of households that exist without registered legal titles” or “Percentage of the city’s solid waste that is disposed of in a sanitary landfill”. Using MCDM weighting methods and statistical tests, we demonstrated how remote sensing can contribute to monitoring city services and quality of life. Based on the expertise of remote sensing experts and the results of this study, it is now possible to present urban planners with existing indicators from the International Standard ISO 37120, “Indicators for city services and quality of life”, that were identified as being possibly partly or fully measured by means of remote sensing.

In addition, a new method was introduced using specific elements of the AHP after Saaty, 1980 [19] by means of an online questionnaire [70]. The test statistics revealed the usability and the confirmability of the presented online questionnaire approach. This approach enables researchers to reach more experts for a potential survey and demonstrates the significance of therein obtained results.

7. Outlook

This research represents the basis for further research activities such as comparing cities or districts within a city (alternatives) concerning the QoL by means of MCDM- and Spatial-MCDM (S-MCDM) models incorporating the ISO 37120:2014 indicators. Therefore, the next possible step could be the application of the identified indicators by means of remote sensing. A methodology has to be developed that enables the extraction of specific indicators from remotely sensed data in a transferable and consistent manner. This methodology should consider that the monitoring of such indicators at a city scale presumably requires very high spatial resolution data; thus, the time and effort necessary for conducting such monitoring should be outlined and cost–benefit considerations should be addressed. Although the mere use of remote sensing data may facilitate such monitoring process, it cannot principally be ruled out to additionally use ancillary data, e.g., vector data in the form land use plans, cadasters, etc., for the identification of, e.g., public outdoor recreation space within a city.

In S-MCDM, alternatives can be represented as locations (raster-based approach) or geographical features [14,18,71,72,73] such as points, polylines or polygons (vector-based approach). In raster-based approaches, each criterion can be expressed as a map that involves the decision alternatives (locations), whereas, for vector-based approaches, attribute fields and columns are used to represent the set of criteria for geographical features. There is a variety of different application domains that addresses S-MCDM, such as landscape assessment, environmental protection, utility management, sustainable and regional development, water resource management, land use planning, site planning, natural hazard risk assessment, waste management and network infrastructure planning [74,75,76,77,78,79,80,81,82,83,84]. Those application domains incorporate various mathematical models such as weighted linear combination [17], ordered weighted averaging [85,86,87], ideal point [60,88] or analytic hierarchy process [19] to compute the performance of the alternatives (e.g., cities or districts of a city). Additionally, this research project pointed out that there is a recognizable uncertainty in respect to the expert’s preferences. Especially in group decision-making scenarios, incorporating different interest representatives and competing objectives, experts often argue for disparate indicator weights. Those differences can be expressed in weight ranges, which indicate uncertainties concerning the expert preferences [77]. Examples of spatial uncertainty and sensitivity analysis can be found in [88,89,90,91].