City Assessment Tool to Measure the Impact of Public Policies on Smart and Sustainable Cities. The Case Study of the Municipality of Alcobendas (Spain) Compared with Similar European Cities
verfasst von
:
Raffaele Sisto, Javier García López, Julio Lumbreras Martín, Carlos Mataix Aldeanueva, Linos Ramos Ferreiro
Data analytics is a key resource to analyze cities and to find their strengths and weaknesses to define long-term sustainable strategies. On the one hand, urban planning is geared to adapting cities’ strategies towards a qualitative, intelligent, and sustainable growth. On the other hand, institutions are geared towards open governance and collaborative administration models. In this context, sustainability has become a global concern for urban development, and the sustainable development goals (SDGs), defined by United Nations, are the framework to be followed to define the new city goals and to measure the advances of the policies implemented over recent years. The main objective of this research is to explain the methods and results of the application of a city assessment tool for measuring the impact of public policies on the socioeconomic and environmental structure of a city. It addresses the case study of the evaluation of the strategic plan “Diseña 2020” of the municipality of Alcobendas (Madrid, Spain, with 116.037 inhabitants), the document used to communicate the actions needed to achieve the city goals during the planning exercise. A selection of urban indicators has been aligned with the SDGs defined in the Agenda 2030 to develop a tool for the measurement of the impacts of policies in economic, social, and ecological terms. Through this set of indicators, the tool is able to quantify the impact of the policies on the city and the SDGs and to support the decision-making processes of the administration. The set of urban indicators is divided into five areas: economic development and employment, sustainable development, open government, social responsibility, and quality of life. The data evolution, across the recent years 2012–2018, is used to monitor and benchmark the effects of the applied policies. In addition, Alcobendas can be compared with other Spanish and European cities with similar characteristics; it makes possible assessing the achievement of the city’s strategic areas, incorporating the current trends and fostering the SDGs. Thanks to the quantitative comparable results and the objective approach, this research shows a methodology based on indicators that could be applied and scaled to other cities to generate a common framework for measuring the impact of public policies on cities.
1 Introduction
Sustainability has become a global concern (Keivani 2010) for urban development, and the sustainable development goals (SDGs) defined by United Nations represent the framework to be followed as a combination of three global problems: ecological, economic, and social issues. The SDGs define new guidelines to be included in a holistic framework and offer quantitative goals to reach a global model of sustainable development (Sachs 2012). In this regard, cities have a fundamental role in the advancement of SDGs (OECD 2016), and European local administrations are beginning to establish local goals to design new strategies and policies that follow SDGs. Monitoring the SDGs is an important challenge and a strategic opportunity for stakeholders and beneficiaries involved in the Agenda 2030 at all levels (Sachs 2012), and this requires a method to measure SDGs for defining and eventually achieving them (Lu et al. 2015; Schmidt et al. 2015). In this context, data analytics for evaluation and policy-making is the key priorities for the future of smart and sustainable cities (Kamp et al. 2003), and public administration, at the national, regional, and local level, has acquired a new commitment towards horizontal strategies and performances called Open Government (Lu et al. 2015). Public administration generates and collects a vast quantity of data in many domains (Janssen et al. 2012), and data supports policy-makers in addressing complex problems (Arzberger et al. 2004); therefore, indicators are being increasingly recognized as a useful tool in policy analysis and public communication (Melchiorri and Siragusa 2018).
Over recent years various organizations have put in great efforts to establish a standard or a methodology of indicators to measure SDGs at the national and local level. Sustainable development solutions Network (SDSN) has developed the SDG Index (Biggeri et al. 2019; Sachs et al. 2017) to monitor progress across a diverse set of goals with multiple targets and indicators and to track overall progress. Several similar studies have been undertaken at the national level by various chapters of SDSN, and the first report that measured SDGs in cities in an European context was the SDG Spanish Index Report (Sánchez de Madariaga et al. 2018). The application of the tool is based on the SDGs framework following, as a starting point, the indicators defined in the SDG Spanish Index Report (Sánchez de Madariaga et al. 2018). The report assesses 100 Spanish cities with more than 80,000 inhabitants and all the provincial capitals, according to 85 indicators regarding the 17 Sustainable Development Goals (SDGs) defined by United Nations in 2015, and it is the primary official reference to measure SDGs at the local level in Spain.
Anzeige
The main objective of this research is to describe in detail the methods and results of the application of the city assessment tool, called the Smart&City SDG evaluation tool, for measuring the impact of public policies in the socioeconomic and environmental structure of a city in the framework of the SDGs. This research shows the methodology used to build the tool based on urban indicators to define valid metrics to evaluate the strategic plan of an urban area.
The city selected to apply the tool was Alcobendas, Spain (with 120.000 inhabitants), in the framework of the evaluation of its strategic plan “Diseña 2020.” It is intended to obtain a final report, as objective as possible, that reflects the results achieved of the strategies of the public policies carried out. In addition, Alcobendas is compared with European cities with similar characteristics that enable positioning the city in the achievement of each of its strategic goals, incorporating the current trends, and the SDGs.
The goal is to give the administration a quantitative result of its strategy and to obtain results to scale and apply the tool to the Spanish and European city context to have a standard tool to evaluate the real impact on of the cities’ strategies in the SDGs.
2 Method
The main procedure used for the construction of the tool is based upon the methods applied by SDSN (Sachs et al. 2017), by the Organization for Economic Cooperation and Development (OECD 2008) and by Sánchez de Madariaga et al. (2018) in the SDG Spanish Index Report. The first step in designing the tool is the selection of indicators followed by the SDGs alignment, standardization, normalization, and aggregation. The authors have carried out a quantitative analysis using time series from 2012 to 2018. With the selected set of indicators, and adapting it to the case of study, the tool is able to quantify the impact of the policies in the city and is also able to support the decision-making processes of the administration. A time-series data evolution is used to monitor and benchmark the effects of the applied policies at the local level and on the SDGs. The tool was developed using a set of Spanish and European cities with similar characteristics to those of Alcobendas, the city that was the object of the study (Fig. 1).
Fig. 1
Construction Scheme of the Smart&City SDGs evaluation tool case of application: the strategic plan of Alcobendas
×
Anzeige
A strategic plan is a set of policies and/or projects that determine the general development of an urban area (Gubaidullina et al. 2018) and that, due to its transversal and general nature, must be analyzed by pragmatically considering the 17 SDGs. At present, the planning is mainly oriented towards adapting the strategy of the cities to the changing times, reorienting its activity towards a qualitative, intelligent and sustainable growth, and reorienting institutions towards a model of open government and collaborative administration. In this context of change, the Alcobendas City Council defined a new vision for the city with which municipal actions would be aligned. The choice of this city is due to three main reasons: It is the city with the greatest availability of data in Spain; it is the first city in Spain that developed a long-term strategic plan untied to the duration of the legislature; and the city had the need to objectively evaluate the results of the strategies applied during the last eight years. The assessment of the strategic plan was carried out through the analysis and comparison of values of various urban indicators and their contrast with:
the degree of achievement of the actions and objectives of the plan
the degree of compliance regarding the SDGs
a comparison with similar cities (“twin” cities).
Quantitative analyses were carried out among the various levels of disaggregation of approaches (Sisto et al. 2018). The objective was to identify which sets of urban areas or domains are relevant and what level of association should be reached with the indicators. Based on this analysis, paths and orientations were designed to be followed through five major strategic goals for the development of the city:
Goal 1. Promotion of the city, economic development, innovation, education, and employment
Goal 2. Sustainable development: smart growth
Goal 3. Good governance, open, and responsible management
Goal 4. Social responsibility
Goal 5. Quality of life: culture, leisure, and sport.
The plan collects in detail the five strategic goals, 16 general objectives, 38 projects, and 206 actions. To measure the degree of fulfillment of the objectives and, therefore, of the strategic goals, a series of monitoring indicators was defined by means of which the advances in the implementation of the agreed strategies and its effects on the city have been measured.
2.1 Data Availability and Selection of Indicators
The first step of the procedure is an exploratory analysis of the available data at the local level and the selection of the known and official databases (local, regional, national, European). The starting point of the dataset is the one defined in the SDG Index Spanish report (Sánchez de Madariaga et al. 2018), and due to its special relevance, the following reference documents and databases were reviewed to complete the indicators:
Global indicator framework for the SDG and targets of the 2030 Agenda.
ISO 37120. Sustainable development communities. Indicators for city services and quality of life.
CASBEE for CITIES 2015. Environmental Performance Assessment Tool for Municipalities.
ITU-T Y.4903/L.1603. KPI for smart sustainable cities to assess the achievement of SDG.
ICR iCityRate. Classifica delle città intelligenti italiane. Citta e Comunita Sostenibili.
The databases used to obtain data at the local level for the municipality of Alcobendas and the others cities selected (see Sect. 2.5) were: Statistics Portal of Alcobendas (Observatorio de Alcobendas), Statistics Portal of Community of Madrid (ALMUDENA), National Statistics Institute of Spain (INE) and Eurostat.
2.2 Characteristics of the Indicators
Once the urban domains have been identified and based on the experience and knowledge of the official databases available, indicators are screened by criteria such as quality, availability, and reliability. For the sake of future replicability of the evaluation, the compliance criteria of five key characteristics of the SMART scheme have been used: specific, measurable, attainable, realistic, and timely (Sisto et al. 2018). According to the reliability criteria, only official and/or prestigious sources were considered, with open data, grouped and consolidated. As far as possible, databases of international data according to replicability criteria have been prioritized. Likewise, the selection criteria of indicators were adapted to the official indicators proposed by the Inter-agency and Expert Group on SDG Indicators.
In turn, within the available indicators, three types are distinguished according to the approximation they intend to measure:
Control indicator: It measures the execution of projects and politics; it is a subjective measure defined by the local administration.
Effort indicator: It measures resources aimed to reach a particular goal (such as budget, number of policies); it is used to measure whether the resources allocated to the project have actually been implemented.
Outcome indicator: It measures a magnitude directly related to the quality of life of citizens, (gdp, pm10, employment, etc.).
The tool mainly encompasses outcome indicators aligned with the SDGs. To the extent possible, outcome indicators should always be chosen for the evaluation of the elements of the plan because the effort is assessed according to the achievement percentages of the various projects. The outcome indicators reflect the true impact of the policies on the well-being of citizens, although there is a risk of confusing the real effect with the situation or the actions of other agents. Therefore, it is key to introduce comparisons between the subject of study and similar entities.
On the other hand, the value of the indicators is highly dependent on the contextual aspects. The composition of indicators and the creation of indexes is the technique that makes possible the mitigation of this effect and the monitoring of the evolution of an indicator among the various comparison elements. For the composition of indicators, a “matrix” with long-term context variables and more short-term evolution variables is established. The purpose of this diagnosis is to have continuity in the evaluation phase.
2.3 Alignment with SDGs
The next most relevant indicators for urban ecosystems have been identified from a sustainable development point of view. Indicators have been grouped according to the concepts or keywords described in each SDG; the most frequent and redundant have been preselected. From this first preliminary selection of indicators, consultations with experts from each sector have been made to validate and/or discard some of them; the associated databases have been identified and each indicator was related to one or more specific target of the SDGs. Some goals have not been considered due to one or more of the following reasons:
the goal has already been reached;
the goal is focused on the regional or national level;
the goal is not susceptible to urban scale measurement.
As a result of the previous process, a definitive list of quantitative indicators (see the Appendix), considered valid only data at the municipal level was associated with the appropriate goals.
2.4 Comparability
The comparability is key to the objectiveness of the project because it will enables evaluating the real situation regarding its objectives and expectations. This means that, as far as possible, the proposal of indicators is put into context, on one hand, with the values of the entire environment and the city, and, on the other hand, with the values of similar cities to Alcobendas. The comparability is established in two dimensions; on the one hand, official public indicators must be calculable beyond the city of application itself. Therefore, the desirability of opting for open, public, and official databases is reiterated; on the other hand, benchmarking cities that, due to their similar characteristics, make possible a constructive comparison between them should be selected. Comparison cities were chosen at three geographical levels: regional, national, and European. The selection criteria, defined in consultation with sectorial experts, of these cities were:
Population similar to Alcobendas—between 110,000 and 130,000 inhabitants
Urban typology—cities in a metropolitan area
Unemployment rate close to the country structural unemployment
Gross domestic product per capita.
The cities selected were the following: Bruges (Belgium), Saint-Denis and Argenteuil (both, France), Bremerhaven, Offenbach, Recklinghausen (all, Germany), Zoemeter (the Netherlands), Bergamo and Monza (both, Italy), Pozuelo de Alarcón, Majadahonda, Las Rozas de Madrid, San Sebastián de Los Reyes, Rivas-Vaciamadrid, Sant Cugat del Vallès, Santa Coloma de Gramanet, Reus, Barakaldo and Getxo (all, Spain), Winterhur (Switzerland), and Bolton and Blackburn (the UK).
2.5 Normalization Procedure
After the identification of appropriate indicators, the next step was the normalization of data. The collected values for the indicator are composed of different units of measurement, which makes it difficult to proceed to the aggregation of data and the obtention of a total score. To compare indicators with different metrics within the objectives and projects groupings defined in the plan, a normalization of the values obtained was carried out, based on the definition of maximums and minimums for each metric according to the relative objectives to the current state. The standardization method applied is an escalation to a value in scale 0–1 using the Min-Max method (Sachs et al. 2017). This technique normalizes the indicators to attain an identical range (0–1) by subtracting the indicator value from the minimum value and dividing the difference by the range of the indicator values (Sachs et al. 2017; OECD 2008). The formula for the Min-Max normalization method is:
where X′ = normalized value of the indicator (0–1), X = value of the indicator, Xmin = the minimum value of the indicator, and Xmax = the maximum value of the indicator.
Each indicator has been normalized using a scale 0–1, 1 being the best possible score and 0 the worst; but this equation is only applicable to values with a positive direction (the higher, the better). As the purpose of the normalization is to create a uniform set of values for the indicators in which all have a positive direction, and, to apply the equation also to the values with a negative direction (the lower, the better), a further normalization method was employed (1 − X). At least, to differentiate the results between negative and positive, the minimum value (0) was transformed in −100 and the positive value (1) was transformed in +100.
2.6 Definition of Maximum and Minimum and Aggregation
The standardization of the obtained values was carried out, based on the definition of maximums and minimums for each metric. To define the optimal value, the following criteria were sequentially used: global goals as defined from recognized reports; national or regional goals, calculated from the values of the comparison cities (SDSN methodology) (Sachs et al. 2017); and local goals. For most indicators, the worst value has been defined as the 2.5 percentile of the distribution of the cities selected following the methods established in the SDSN methodology (Sachs et al. 2017). This standardization enables building an index per project by adding all its associated indicators and assessing its growth with an annual average of the normalized values of each indicator and the difference between the 2014 value and the 2018 value (Fig. 2).
Fig. 2
Interpretation of the visual tool
×
Equal weights were given to each indicator with regard to its values in each aggregated index. Equal weighting is a viable alternative due to the fact that all the indicators involved are priority policy concerns. Thus, in light of the measurement of the projects, all indicators are equally considered and none is more important than the others. In terms of the aggregation method, the linear additive technique was adapted. This involves adding all the normalized values arithmetically:
where score = aggregated result of the subindices analyzed (total score, strategic goals, objectives, projects), X = value of the subindices and n = number of subindices.
The aggregation of several motor projects for the evaluation of a general goal is done in the same way (Fig. 3), calculating the mean of the resulting values per year of each project. More detailed information about the upper and lower limits of each indicator are shown in the appendix.
Fig. 3
Aggregation of values at various levels
×
3 Results
The results obtained from the analysis are summarized in the following time graph.
The actions that make up the Design Alcobendas 2020 Plan have so far been 83% executed, a very high percentage for such a plan. The 64 selected urban indicators reflect a positive evolution in all the strategic goals. Since the implementation of the strategic plan in 2014, the city has improved by +23 points (in a scale from −100 to +100) regarding the selected urban indicators. This positive evolution occurs in all strategic goals and is consistent with the compliance percentages achieved, according to the internal evaluations of its follow-up. In particular, strategic goal 1 (Eje 1 in Fig. 4) has improved by +36 points, strategic goal 2 has improved by +16 points, strategic goal 3 has improved by +12 points, strategic goal 4 has improved by +20 points and, finally, strategic goal 5 has improved by +23 points. In comparison with the rest of the cities, Alcobendas has also experienced a positive evolution above average. The most outstanding evaluation results are presented next.
Fig. 4
Historical evolution of the results of the strategic goals of the plan (Eje)
×
4 Discussion
Strategic goal 1 highlights Project 1, “Employment Promotion,” with a positive evolution of 48 points (Fig. 5). This is because the unemployment rate (7.1% in 2018) follows a trend of improvement well above the average of the Community of Madrid (11%) and that of Spain (14%).
Fig. 5
Time series results of the evolution of the unemployment rate in Alcobendas
×
In strategic goal 2, Project 14 citizen security stands out with a positive evolution of 53 points. The crime rate has continued its downward trend of improvement throughout the duration of the plan. However, a similar trend is observed in all the cities analyzed in the Madrid region (Fig. 6). This may point to the fact that some supramunicipal policy has caused the increase of citizen security.
Fig. 6
Time series results of the evolution of the crime rate in Alcobendas
×
Strategic goal 3 highlights Project 15 transparency, open government, and administration, whose level of excellence and demand has been preserved throughout the development of the plan. In this case, it should be noted that although Alcobendas is the best city in this respect (basically it cannot improve), it does not score positively, and the value is 0 (Fig. 7). This serves to highlight that it is necessary to maintain the level of excellence in this respect, but maybe the city council should also devote attention to other issues or problems in future strategic plans.
Fig. 7
Time series results of the evolution of the transparency index
×
It can be reasonably deduced that the designed strategy has been largely implemented and that it has had, so far, a positive impact on the public policies that were pursued. In view of future strategic plans and upcoming evaluation cycles, it is recommended:
To realize a previous alignment of projects to the SDGs in their various applications in different contexts. This will allow comparability and result orientation with an international projection.
To specify the actions in quantifiable objectives to be achieved based on a previous diagnosis. Thus, a more precise compliance can be monitored and the relevant corrective measures can be evaluated.
To carry out a study of the analysis of results periodically during the implementation of the plan.
5 Discussion
The advantages of the tool are linked to the possibility of obtaining an objective picture of the strategic areas that work better or need more attention in the development of new policies. The tool shows and quantifies immediately which areas have performed better and which strategies need adjustments. The structure of the tool enables an in-depth analysis by using the scores attributed to each sub-index, which serves to better identify each element. The selected indicators were adequate as they accurately reflect the factors that most influenced the quality of life and the SDG achievement in urban areas. In addition, since the results are expressed in numeric values, the tool significantly reduces the subjectivity in the evaluation of the plan and enables comparability of the various topics.
Another strength is the time series used. It is essential to monitor the scores over time to show the tendencies, weaknesses, and opportunities that the public administration should consider when designing its policies and planning its actions.
This research also aims to explore and study the limits founded. The authors decided to apply the same weight to all indicators, but some topics should play more important roles in the urban context. The tool partially solves this problem, favoring the use of internationally recognized indicators in the measurement of each topic without using other specific variables that could distort the results. However, the introduction of a different weight for each topic, based on recognized criteria, is needed. Another important challenge is the limited availability of data: In some case, the indicators with available and comparable data were not the best for the measurement of policies or a topic. In this regard, much progress is needed in the availability of data and the homogenization of databases.
6 Conclusion
The value proposition of this tool consists of:
Measuring the impact of the strategy deployed in the territory where it is applied.
Being able to translate the global impact into a single aggregate indicator for stakeholders.
Measuring the situation of the city compared with other similar cities in achieving its goals in strategic areas.
Comparing the situation of the city against the SDGs of the 2030 Agenda.
Identifying best practices within a strategy and serving as an input for the next planning cycles.
It should be noticed that, thanks to the quantitative comparable results and the objective approach, the Smart&City SDGs evaluation tool does not reflect SDGs and the achievement of local goals per se, but rather achievements in the identified indicators in comparison with other similar cities. Although the values do not necessarily imply that a city is smart or sustainable or not, they can help assess the performance of a city and how it fared in comparison with the achievement of other cities. This helps quantify the impact of the adopted policies, identify best practices within its strategy, and improve future policies for the next strategic plan. Furthermore, the subindices help identify the weaknesses and strengths of the city, which provides insights on how to improve their performance. The visualization of the urban economic, environmental, and social impacts goes beyond the support of the decision-making processes and can also be used as a political argument to promote and foster long-term planning (Waas et al. 2014) and urban sustainable development in European cities. Finally, the selection of urban indicators is proposed as a part of a standardization and scalable process which can be replicated in other cities.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
2. Promote creation of companies, entrepreneurial culture, and its associated talent
Number of companies by main activity
X1000 inh.
INE
9
3. Implementation of innovative economic activities
Number of patents/number of companies
Ratio
INE
9
4. Improvement of competitiveness of business and industry
Employment vulnerability index (index built by the number of companies by main activities)
Index
Eurostat
9
GDP growth rate/employee growth rate
Ratio
ALMUDENA
8
5. Promotion of commercial activities
Number of commercial activities
X1000 inh
ALMUDENA
8, 12
Number of commercial activities/total of productivity units
Index
ALMUDENA
8, 12
6. Coordination and improvements in training and employment policies
Municipal budget settled for education spending policy
€ cápita
INE
4
Survival rate of pupils in non-compulsory education
%
ALMUDENA
4
7. Language knowledge
Number of language teaching places offered/number of students
Ratio
ALMUDENA
4
8. Smart City
Smart economy index (based on number of companies by activity; municipal debt)
Index
INE
9, 16
Smart governance index (based on electoral participation; transparency index)
Index
INE, DYNTRA
16
Smart environment index (based on indicators of “Project 10—environment and energy sustainability")
Index
Eurostat
7, 11
Smart people index (based on resources for cultural and educational infrastructure per 1,000 hab.; budget in education/total number of non-university students; education level of population ISCED 5–8)
Index
Observatorio
ALMUDENA
Eurostat
4, 10
11
Smart mobility index (based on indicators of “Project 9—New urban developments and public spaces” and “Project 13—sustainable mobility and transport”)
Index
INE
ALMUDENA
11
Smart living index (based on indicators of “Project 12—Facilitate access to housing and promoting rental housing”)
Index
INE
ALMUDENA
10, 11
9. New urban developments and public spaces
Area of urban plots/total urban land
m2t/m2s
ALMUDENA
11
Rate of the budget settled for public lighting spending policy
City Assessment Tool to Measure the Impact of Public Policies on Smart and Sustainable Cities. The Case Study of the Municipality of Alcobendas (Spain) Compared with Similar European Cities
verfasst von
Raffaele Sisto Javier García López Julio Lumbreras Martín Carlos Mataix Aldeanueva Linos Ramos Ferreiro
