The Inclusive Sustainable Transformation Index

The modernization of agriculture and sustainable industrialization are essential features of the structural transformation process. Productivity increases in agriculture provide food, labor, and savings that fuel the process of urbanization and industrialization (Timmer and Akkus 2008). The development of a competitive industrial sector yields an even higher payoff. Economists have established at least since the early 1960s that manufacturing has always played a significant role in the total output of more affluent countries, and that countries with higher incomes are typically those with substantially larger transport and machinery sectors (McMillan and Rodrik 2011). In fact, only in singular circumstances such as an extraordinary abundance of land or resources have countries succeeded in developing without industrializing. Industrialization also promotes inclusive development by expanding the fiscal space for social investments.

Within the industrial sector, manufacturing in particular has transformed the dynamics of the world economy. The globalization of manufacturing is driven by many factors, including profound changes in geopolitical relations among world nations, the widespread growth of digital information, the decline of transportation costs, the development of physical and financial infrastructure, computerized manufacturing technologies, and the proliferation of bilateral and multilateral trade agreements. These developments have permitted the decentralization of supply chains into independent but coherent global networks that allow transnational firms to locate different parts of their businesses around the world. The creative design of products, the sourcing of materials and components, and the manufacturing of products can now be done more cheaply and more efficiently from virtually any region of the planet while final goods and services are customized and packaged to satisfy the needs of customers in faraway markets. The globalization of manufacturing has thus allowed developed economies to benefit from lower wages in developing countries such as China, India, Bangladesh, Costa Rica, Mexico, and Brazil while creating job and learning opportunities in these formerly poor nations.

In recent decades, innovation, technological developments and new sources of economic growth have led some economists to question whether manufacturing still matters. Manufacturing’s share of global value added has steadily declined over the past 30 years as the global value added of services has grown.² However, these trends are mainly observed in high-income countries and can be explained by several factors. First, productivity increases and rising standards of living in advanced economies have pushed up wages and forced many industries to delocalize their production to lower-cost nations. Second, increasing levels of efficiency in the world economy have reduced the relative prices of consumer goods while at the same time the demand for services such as healthcare, security, and transportation has increased. Finally, and perhaps even more importantly, manufacturing jobs have a multiplier effect on employment in services, as the development of industries everywhere automatically generates a wide variety of economic activities, from transportation to housing, from hospitality to entertainment.³

Concerns about the future of manufacturing as a viable source of economic growth have been investigated empirically by Hausmann et al. (2011). They found over 70% of the income variations among nations can be explained by differences in manufactured product export data alone. The analysis of the composition and scale of a nation’s manufacturing sector revealed that sophisticated economies export a large variety of ‘exclusive’ goods that few other countries can produce. These economies have typically accumulated productive knowledge and developed manufacturing capabilities that others do not have. It therefore appears that national income and economic sophistication (economic complexity) rise in tandem.

Even basic manufacturing expertise can gradually generate new knowledge and lead to new, more advanced products, provided that the right strategic and business decisions are made on industrial and technological upgrading. In the words of Hausmann and Hidalgo (2012, p. 13), economic development is ‘a social learning process, but one that is rife with pitfalls and dangers. Countries accumulate productive knowledge by developing the capacity to make a larger variety of products of increasing complexity. This process involves trial and error. It is a risky journey in search of the possible. Entrepreneurs, investors, and policy-makers play a fundamental role in this economic exploration. Manufacturing, however, provides a ladder in which the rungs are more conveniently placed, making progress potentially easier.’ In sum, manufacturing still generates economies of scale, sparks industrial and technological upgrading, fosters innovation, and has significant multiplier effects.

There is no shortage of composite indexes to track economic development over time and across countries. In fact, there are so many of them that it has become almost impossible for policymakers to make sense of the stories they tell and to identify the specific, actionable policy levers that yield clear economic and social gains. In his critical review of some popular composite indexes of development, Ravallion (2011, p. 2–3) categorized them into two broad types. First, indexes such as the gross domestic product (GDP), for which the choice of the component series and the aggregate function ‘are informed and constrained by a body of theory and practice from the literature.’ Second, indexes such as the Human Development Index that are based on ‘a set of indicators that are assumed to reflect various dimensions of some unobserved (theoretical) concept.’ He saw the former as more appropriate indexes, while the latter lacked the necessary analytical legitimacy: ‘neither the menu of the primary series nor the aggregation function is pre-determined from theory and practice, but are […] key decision variables that the analyst is free to choose, largely unconstrained by economic or other theories intended to inform measurement practice’ (p. 3). To illustrate his point, Ravallion (2011) contrasts an index where the variables and weights are based on a regression model calibrated with survey data with an index where they are set by an analyst who has some concept of economic welfare in mind. Ravallion (2011) refers to the latter as a ‘mashup’ index.

Such a distinction may seem like an elegant conceptualization of the problem at hand, but it is an artificial one. First and foremost, the contention that GDP should be the model index, one legitimized by ‘pure’ theoretical reasoning and rigorous analytical modeling, is invalidated by the body of academic research that has highlighted its many shortcomings (beyond being a randomly aggregated set of variables that form a series of accounting identities). GDP as an index is not beyond suspicion. Calling it a ‘capitalist conspiracy’ like Venezuelan President Hugo Chavez did, may have been an extreme form of criticism. Yet, as a model index, GDP carries many shortcomings and paradoxes (see Coyle 2014). While GDP measures product, it ignores central facts such as quality, costs, sustainability, or purpose (Stiglitz et al. 2010).⁴

Moreover, the expectation that economists and other social scientists can construct development indicators that pass the test of ‘pure theories’ simply because such indicators would be ‘based on a regression model calibrated to survey data’ is unrealistic. It is well known that regression models are based on a host of assumptions; without them, legitimate inferences cannot be drawn from the model. While there are statistical procedures for testing some of these assumptions, the tests often cannot detect substantial failures. As pointed out by Freedman (2010, p. 14), ‘model testing may become circular; breakdowns in assumptions are detected, and the model is redefined to accommodate. In short, hiding the problems can become a major goal of model building.’

However appealing it may be at face value, the dichotomy between ‘credible’ indexes based on some theory or calibrated from regression analyses, and ‘mashup’ ones singled out as ‘randomly elaborated,’ is problematic because what constitutes an acceptable theoretical basis is always debatable. Such a distinction assumes the existence of a rationally-neutral analyst who can observe and monitor performance with distance, detachment, and balance. Philosophers have long provided good arguments about the impossibility of this type of rational actor. From Darwin to Marx, Nietzsche, Freud, Wittgenstein or Heidegger, there is an accumulated body of evidence that the so-called sovereign rational subject—the detached observer imagined by Kant—actually does not exist.⁵ It follows that no development economist is intellectually autonomous, self-transparent and capable of identifying causal relationships and causal mechanisms in a detached manner. It is impossible to deny the role of the pre-conceptual and non-conceptual at the very core of the rational. The notion that subject and object can be separated from one another—the supreme dogma of empiricism—is merely an illusion.

Therefore, any economic theory or model, especially one built on regression analyses, should acknowledge the limits of its generated knowledge. Any index out there reflects an explicit or implicit theoretical analysis of the dynamics of economic development. The real criteria for assessing pertinence and effectiveness should be whether an index provides useful information to strengthen intellectual and policy arguments and whether it helps focus the attention on social and economic goals deemed to be of importance to society.

The first step in the construction of the IST index is the selection of the indicators of inclusive and sustainable development, which will be outlined in detail in the next section. The remainder of this section discusses the subsequent steps, namely the way in which these indicators are rescaled, weighed and aggregated, as well as how the level of development is taken into account and how missing observations are handled.

Before the indicators can be combined, they first have to be put them on a comparable footing. There are different ways of transforming these variables, many of which are described in the OECD handbook on composite indicators (OECD-JRC 2008). The distribution of values of the indicators in the IST dataset can differ markedly between indicators and different levels of development. This argues against using more common transformations like z-scores or Min–Max as done for example in the Quality of Growth index (Mlachila et al. 2014). Instead, we use the (empirical) cumulative density function (CDF), which expresses the probability of finding a country with a lower score.⁶ The main advantage of the CDF is that it provides a clear interpretation: a CDF of zero means that the country performs worse than all other countries with the same level of development, while a CDF of one indicates the opposite. Any score in between can be interpreted as the fraction of countries that scores worse. Moreover, as we will see below, the CDF also enables us to account for the level of development in a straightforward fashion, without discontinuity problems.

The estimation of the CDF is done year-by-year, using only the information on the distribution of the indicator available at that point in time. This allows the frame of reference to change over time, as the older patterns in the distribution of the indicators left out. As a result, the transformed scores represent a country’s position relative to its peers in that year. The CDF will not change if all countries improve (or deteriorate) at the same rate. Similarly, a country’s score will decrease if it remains unchanged while its peers improve. In this way, the index underlines the idea that inclusive and sustainable industrial development is a continuous process of improvement for all countries, rather than a fixed path with a clearly defined end goal. An additional benefit of this relative approach is that it undermines what Ravallion (2011) termed rank-seeking behavior: increasing your score until it just exceeds a benchmark. Unless a government is willing to decrease its level of development deliberately, each aspect of the index will need continuous improvement for the country to keep up with its peers.

After transformation, the indicators are combined into the IST index using a simple average. This means that each component receives equal weight in the final index. Moreover, it imposes perfect substitutability between all goals: a high score on manufacturing compensates perfectly for a lower environmental score. However, in the robustness section of this paper (Sect. 5.1), we abandon the assumption of perfect substitutability and instead use a geometric average, which penalizes countries for imbalances in their scores.⁷ However, as will be shown in the robustness section, the effect of using a geometric average on the index values is small.

As the weights sum up to one, the IST index has the same range as the transformed indicators: an IST score of one means that the country outperforms all of its peers, the opposite holds true for zero. Anything in between can be interpreted as the average fraction of countries that perform worse.

A central tenet of New Structural Economics is that the structural characteristics of countries are not a one-size-fits-all (Lin 2012a). Instead, the economic structure that best helps growth will change as the country develops and new characteristics become feasible. Simply put, one cannot expect relatively developing countries such as India, Burundi or Ethiopia to have the same environmental, institutional and economic characteristics as rich countries like Denmark or Japan. Rather than relying on the individual to consider this when using the IST index, we want to embed this thinking directly into the index. It should indicate the performance of a country relative to countries with a similar level of development.

By way of illustration, Panel a of Fig. 1 plots the investment in Research and Development in percent of GDP (R&D) versus the level of development (dev) as measured by the log of the Gross National Income (GNI) per capita in 2012. It shows that the distribution of investment in R&D depends strongly on the level of development. The higher dev, the more the distribution shifts towards the maximum values of R&D.

An intuitive way of taking development into account is to group countries by their development level and only compare their characteristics within these groups. In line with NSE theory, PPP converted per capita income can be used as an indicator of the level of development and the capacity of an economy.⁸ For example, the income classification employed by the World Bank identifies four different income levels: low-income, lower-middle and upper-middle income and high-income countries.⁹ Panel b shows the joint distribution of y and dev and Panel c illustrates how this translates into the conditional CDF for the four income-levels. In general, countries with a high investment in R&D will get a higher score. However, for a given percentage of investment in R&D, a low-income country will receive a higher score than a middle or high-income country.

The use of discrete income groups to differentiate levels of development has one important drawback: it creates discontinuities for countries lying on the border. A small change in the level of development can change the group a country belongs to, which can have significant consequences for those variables that strongly depend on it. This can lead to a situation where a small improvement nevertheless leads to a decrease in the index value, only because the country is now compared to an entirely different set of countries. It also biases the comparison between countries that lie on either side of the cut-off point. This problem can be avoided by using a continuous way of controlling for the level of development such as the conditional cumulative density function. The conditional CDF gives us the probability of finding a country with a lower score that has the same level of development: \(F_{y|dev} \left( a \right) = p(y \le a|dev)\). As illustrated in Panel d, the transformed values using the conditional CDF are very similar to those using fixed thresholds, but without the discontinuity problems for countries whose level of development is close to the thresholds.

The conditional cumulative density \(F_{y|dev}\) is estimated using a multivariate kernel density estimator. An essential feature of this type of estimator is that it assigns a higher weight to information in the vicinity of the point of interest, both in terms of the variable and the level of development. The bandwidth of the estimator determines what counts as the vicinity: the larger the bandwidth is, the more the performance of dissimilar countries taken into account. By estimating the bandwidth, we can adjust it to match each indicator.¹⁰ If an indicator is strongly dependent on the level of development, only the information of countries with a very similar level of development is taken into account. However, if the level of development does not affect y, the increase in the bandwidth ensures that more information is used, allowing us to estimate the CDF with greater certainty.

Finally, we ensured that a higher score is always an improvement in outcomes. If an indicator measures something positive (e.g., the share of renewable energy), the transformed indicator \(\hat{y}_{i}\) indicates the probability of finding countries with a similar level of development (\(dev_{i}\)) that score lower: \(\hat{y}_{i} = F_{y|dev} \left( {y_{i} |dev_{i} } \right).\) Conversely, for indicators that measure something negative (e.g., CO₂ emissions) \(\hat{y}_{i}\) shows the probability of finding a country that scores higher: \(\hat{y}_{i} = 1 - F_{y|dev} \left( {y_{i} |dev_{i} } \right).\)

The final issue that needs to be addressed before the index can be computed is how to deal with missing values. The IST index comprises a large number of indicators, and as can be seen in Table 1, the coverage of those indicators can be markedly different. When we limit the dataset to the period 1990–2016, slightly more than half of all observations are missing. A closer examination of the pattern in the missing data reveals that this cannot be easily solved. Figure 2 illustrates this by mapping the data availability using black (available) and white (missing) rectangles. Each row in this table corresponds to one of the 27 variables. The columns, in turn, show the different combinations in which the indicators are available, with the width of each column signaling how often this combination occurs in the dataset. The columns are sorted in decreasing order of occurrence. The first column tells us that the most common combination is that only the national electrification rate (O7.1) is available. The second most prevalent combination (4% of the data) is the total absence of any information. In contrast, there are no observations that are covered by all indicators as there are always at least two indicators missing.

Our first step in tackling the missing data problem is to limit the number of countries for which we compute the index. Specifically, we exclude those countries for which the average data availability over the entire period is less than 25%. This removes mostly small island nations and city-states like Gibraltar, Nauru, or Vatican City, leaving 198 countries in the dataset. However, as Fig. 2 already suggested, limiting the dataset in this way will not solve the missing data problem, as there is no simple pattern in the missing observations. While removing the countries with the worst coverage does reduce the number of missing observations with one quarter, the total fraction of missing values remains high (43%).

In a similar vein, we could leave out the variables that have the lowest availability. For example, by excluding the participation in global value chains (O2.3), gender equality (O5.2) and the wage gap (O5.3), municipal waste (O.6.4), waste recovery (O.65), the Ocean Health index (O8.2) and the protection of terrestrial and marine protected areas (O8.4), the number of missing observations can be cut by half. However, changing the variables also affects the meaning of the index, as it amounts to imposing a zero weight on a quarter of the indicators in our dataset. Moreover, all but the most extensive reductions in the scope and span of the index would still leave differences in availability, making it hard to tell whether changes in the index are not due to differences in availability of the underlying indicators.

Rather than drastically limiting the scope of the IST index, we opt instead to solve the missing data problem using multiple imputation, i.e., filling in the gaps with the most likely value of each indicator. Unlike simple imputation (e.g., linear interpolation), multiple imputation draws many different possible values for each missing observation. This gives us a large number of imputed datasets, each of which yields a different index. The final result is computed as the average of these indexes. The variation of the index over different imputations also gives us an indication of the reliability of the results. The more data is missing and the worse the available data is at filling in the gaps, the greater the dispersion in the imputed values and the wider the confidence intervals of the index will be. In summary, multiple imputation allows us to compute the IST index despite the significant data availability problems without having to reduce the scope of the index or omit variables ex-ante.

To estimate the most likely value of the missing observations, we use a method similar to the Multivariate Imputation by Chained Equations method (MICE) of Buuren and Groothuis-Oudshoor (2011). MICE uses the values of the available indicators to predict what the missing values could have been. For example, it will use the information on the number of patents applied to estimate what the level of investment in research and development could have been. While this works well in a cross-section analysis of countries, MICE ignores time-patterns in the data. As we have a panel dataset where many of the indicators depend strongly on their previous values, we expand the MICE method to a state-space model to also take this time-dependency into account. The so-called Multiple Imputation by State Space models (MISS) improves the quality of the imputations thereby significantly decreasing the size of the confidence intervals. Technical details on the imputation and a comparison of the two techniques can be found in “Multiple Imputation Using State-Space Model (MISS)” of Appendix.

To test the robustness of our imputation algorithm, we also ran the model on a reduced dataset where the data availability of each country was at least 50%. This reduced the total number of countries by half compared to our baseline model (which only imposed 25% availability). However, the imputation of the missing values remained virtually unaffected. For all but one variable, the correlation between the mean imputed values was in excess of 0.99. The change in the sample did affect the size of the confidence intervals, but not in a uniform way. One-third of the sample had confidence intervals that were at least 10% smaller; a third had larger confidence intervals, while the remainder had confidence intervals that were more or less the same.¹¹

In this final section, we determine the IST index’ sensitivity to our modeling choices. First, we look at how the results change when an alternative measure of the level of development is used. To that end, we use the UN’s Human Development Index (HDI), a composite index that combines GNI per capita with life expectancy and education level. With the HDI as our measure of development, the individual indicators from Table 1 are transformed and the results are combined into a second index: \(IST^{HDI}\). While there are some differences between our baseline index and \(IST^{HDI}\), their overall correlation is high (Fig. 6, panel a). The development category where the most prominent changes take place is in the lower-middle income group. While the holistic nature of the HDI might make it a more appealing choice as a measure of development, there are two reasons why we use GNI per capita instead. First of all, there is an overlap between HDI and the indicator of human capital included in the index (O4.3). Secondly and most importantly, the HDI is only available every five years between 1990 and 2010.¹⁷

As a second robustness check, we consider the effect of using a geometric average to combine the indicators. Unlike the arithmetic average, which imposes perfect substitutability between the components, the geometric mean penalizes countries with asymmetric component scores. The effect on the ranking by the index is minimal, as the correlation between \({\text{IST}}^{\text{GEO}}\) and the baseline IST is 0.94. Nevertheless, as panel b of Fig. 6 shows, the scores are lower using the geometric average. Some countries find their score significantly decreased, like Libya which sees a 44% decrease (from 0.32 to 0.18). While the distribution of IST is negatively skewed, \({\text{IST}}^{\text{GEO}}\) has a more symmetric distribution. On the other hand, it also has much wider confidence intervals. Given that the results are so similar, we opt for the arithmetic average as it provides a more straightforward interpretation (i.e., the average fraction of countries that score better).

We also check how the index changes when variables are omitted. In this regard, the index is recomputed 26 times using all but one of the indicators. Regardless of which variable is left out, the index and its standard deviations are almost identical: the correlation of both exceeds 0.98 each time (Fig. 6, panel c). Finally, as some categories contain more variables than others, we also check how the results change when each category, rather than each indicator, receives equal weight in the final index. Similar to the leave-one-out estimations, the results are practically identical.