11. Measuring Development Cooperation and the Quality of Aid

It is clear from other chapters of this book, in particular that of Gerardo Bracho, that the mechanisms for effective development cooperation between countries—either through the Global Partnership for Effective Development Cooperation (GPEDC), the Organisation for Economic Co-operation and Development (OECD), or the United Nations (UN)—are currently uncertain, or contested. But progress can be made on measuring what countries are doing in different areas of cooperation. One of the major constraints Bracho highlights to deeper collaboration relates to whether all countries have “the same responsibilities as traditional donors” in the effectiveness agenda and whether each should be “subject to practically the same scrutiny” (Bracho 2021, Chapter 17). However, even before we consider respective countries’ responsibilities and scrutiny, we can at least try and measure the contributions of different countries to development on a consistent basis. Such a measurement would have value in a number of dimensions. First, it would enable countries to understand what they are contributing to development in absolute terms. Second, it enables comparison and learning between different approaches. Third, it can give a measure of the potential for improvement. Fourth, it can enable scrutiny of these areas to drive improvement, and finally, it can provide the underlying measures to enable contributions to be assessed against responsibilities. This chapter aims to take stock of our ability to measure contributions to effective development cooperation across different countries.

What do we mean by “development cooperation”? The principle I use here is that “development cooperation” refers to a country’s policies and how these affect the current and future welfare and growth of other countries’ people and economies.¹ So, even countries that do not have an explicit policy towards other countries are included because their policies—for example, on climate, migration, and trade—have a bearing on people elsewhere, regardless of their intent. In this respect, all countries are global citizens and have a development impact that can be measured. This is consistent with the universality (i.e. applying to all) of the Sustainable Development Goals (SDGs) and the 2030 Agenda for Sustainable Development, which sets the framework on the goals, targets, and measures applying to, and agreed on, by all countries.

In order for development cooperation measurement to be meaningful, it needs to be undertaken with reference to the “size” of the country being measured. Taking greenhouse gas emissions as an example, it is clear that—all else being equal—a more populous country will emit a greater absolute volume, so measuring that country’s effort will mean calculating a per head figure. Similar arguments can be made for “aid”, which is measured relative to the size of the economy. Using measures relative to country size can enable comparisons between countries at different income and population levels. Of course, this means that absolute contributions of countries are ignored. So, although the United States is the largest absolute aid donor, its government gives relatively little compared to the size of its economy. The absolute contribution (see below) is what matters for global impact, but when it comes to the measurement and comparison of efforts, this must be scaled by an appropriate measure of the relevant size of the country.

Differences in citizens’ footprints in individual countries notwithstanding, the largest (often the most populous) economies are likely to have the largest absolute spillovers on other countries—for that reason, their development cooperation should be of greatest interest to us. If the level of a country’s income means it should take a greater responsibility for its impacts, then the top right of Fig. 11.1 illustrates those countries of greatest interest.

The chart demonstrates that the G20—formed broadly on the basis of the size of members’ economies—is still essentially the largest economies,² and that the vast majority of countries with higher incomes are predominantly members of the OECD. Together, the G20 and OECD countries produce around 90 per cent of global GDP, and likely a similarly substantial portion of transboundary spillovers.

Note that we do not consider the actions of private actors in this analysis. This is not that the private sector is unimportant, but rather that country governments set the framework for the private sector and—in assessing development cooperation—we are concerned with what governments do, or do not do. However, the basic principle we discuss should also apply to companies—they should understand and measure where their activities in the market lead to spillover impacts on others outside the market. This would include energy companies quantifying their impacts on the climate, and tobacco companies quantifying their impacts on health. All companies should understand these impacts, as they are responsible for them. A promising initiative in this regard is the Global Reporting Initiative (2018), which helps to develop common standards for sustainability reporting. In my view, every company creating material spillovers should quantify these impacts in a move to “quantified corporate social responsibility”.

But why do we need to measure development cooperation when we have the targets and indicators underpinning the SDGs? The 2030 Agenda and the SDGs are the globally agreed framework around development, but they are not focussed on what one country does for another, and they offer only a partial picture of development cooperation. The 17 SDGs are supported by targets and quantitative indicators. However, the clear majority of SDG indicators relate to national performance rather than transboundary or spillover effects (see Table 11.1). Of the 244 indicators supporting the SDGs, fewer than 30 per cent measure a “transboundary” effect. Even SDG 17 (Strengthen the means of implementation and revitalise the global partnership for sustainable development) has over a third of its 25 indicators focussed on national performance. The OECD has undertaken a similar exercise, focussing on the 169 targets in the SDGs. It finds that 169 (57 per cent) can be described as having transboundary elements (Organisation for Economic Co-operation and Development [OECD] 2019). These indicators and targets therefore provide only a partial picture of development cooperation.

So what can we measure on development cooperation now, and what more do we need? The remainder of this chapter looks at the scope of what we should measure, considers the state of the measures in each area, and concludes with where future collaboration is needed.

Here, I consider development cooperation in three different, perhaps overlapping, areas,³ which I think is helpful conceptually. First, development finance—including aid and concessional finance—can be provided by a government to another country to support its development. Second, policies with bilateral international impacts—in particular those related to the exchange of goods, capital, people, and ideas (such as trade and migration policy)—have impacts both domestically and in other countries, often with mutual benefits. Third, countries’ actions contribute to, or detract from, global public goods (GPGs) (or regional ones)—in particular, on climate, but also in areas such as security. My colleague at CGD Charles Kenny has generated an illustrative (rather than exhaustive) list of 72 GPGs and examined their spatial and temporal distribution (Kenny et al. 2018) (Table 11.2).

The distinction between these three groups relates to the focus of the impact. The first two groups both relate to policies that bilaterally affect other countries. The first group—development finance—is broken out separately, as often its intention is to (positively) impact another country. In addition, development finance is often seen as a critical limiting factor in development and receives significant attention in development and the literature. The second group relates to policies that have a direct effect on a partner country and may or may not have a positive effect or mutually beneficial effect. The third group has impacts—intended or otherwise—that are international but affect several countries simultaneously, or have the characteristics of public goods or bads (i.e. countries cannot be excluded from their impacts and cannot diminish them). Table 11.3 elaborates on these descriptions with the domestic and international impacts.

We now consider each of these areas, in turn, and consider the most important areas to measure.

In this section, we look at (i) measures of quantity of concessional aid/finance, (ii) what we know about effective development finance and, (iii) what measures we have available on quality.

In considering what countries can do in terms of development cooperation, the contribution of development finance and aid is only one part of the overall picture. “Beyond aid” there are a number of other policies that have a major bearing on development. This is often referred to as the “policy coherence for development” agenda, which emphasises the role of different policy areas in affecting development. Policies and their coherence for development are being recognised in achieving the SDGs (Knoll 2014). The role of policy beyond aid in development has long been recognised. Since 2003, the Commitment to Development Index, developed by David Roodman (2005) and Nancy Birdsall (and now directed by me with colleagues of the Center for Global Development), has identified policy areas that affect development. It now tracks countries’ efforts in several policy areas, currently aid, environment, investment, migration, security, technology, and trade. Below, I expand on how policies matter in the flow of goods, people, finance, and ideas.

To illustrate the importance of policies beyond aid, consider China, which reduced the proportion of its population living on less than $1.90 per day from 66 to 0.7 per cent in only 25 years to 2015 (World Bank, n.d.-a), although it had benefitted from very little aid.¹⁰ Crucial elements of this transformation included domestic market reform and access to global markets, including membership in the World Trade Organization (Subramanian and Wei 2003).

Trade policy, then, is a crucial determinant in development. The substantial liberalisation of trade policy and the reduction in agricultural subsidies over the past four decades are likely to have played a key role in providing an international trade environment conducive to development. Of course, the recent trade disputes between the United States and China, along with the United Kingdom stepping back from its deep trade relationship with the European Union, illustrate that this environment is under threat, or even in reverse. As emerging economies grow, their markets and trade policies will be increasingly important to the development of others.

Migration is an area that has traditionally received less attention as a policy for development, often due to concerns with “brain drain”. My colleague Michael Clemens has refuted the idea that migration harms developing countries via a “brain drain” (Clemens 2009). He shows that 94 per cent of the countries that grew to middle-income status or higher between 1960 and 2013 also saw an increase in the proportion of their population emigrating (Clemens 2017, p. 5). Often the impacts on migrants themselves are forgotten in discussions of development and migration. Work by Clemens and Lant Pritchett shows that simply by moving from a poor country to a rich one, people can increase their incomes by 700–1000 per cent (Clemens 2011). This effect is a multiple of effective aid programmes (Pritchett 2016). The importance of migration is also illustrated through the sums of remittance flows to low- and middle-income countries: These amounted to $429 billion in 2016 (World Bank Group and Knomad 2018, p. 4)—almost four times the amount of ODA, and much more stable than private finance flows.

On ideas, countries’ policies centred on spreading ideas and knowledge also matter. One element of this effort if focussed on research and development, which we consider under GPGs. Other areas that matter are policies around knowledge transfer, for example on intellectual property rights (Park and Lippoldt 2014).

On finance, we have considered in the section above the quantity and quality of government-led development finance. But policy also governs several other important areas—in particular, policy on international tax plays a role in a developing country’s ability to raise tax revenue (domestic resource mobilisation). Countries also seek to influence or support private financial flows, for example with their investment policy and agreements. Work by the Research Centre for Policy Coherence and Development (CIECODE) (see Robinson et al. 2018, pp. 20–21) provides a framework for considering an equilibrium between ensuring that countries retain their right to regulate as they pursue public policy interests (including sustainable development objectives) while contributing to a favourable investment climate and protecting foreign investors from unjustified discrimination measures by the host state.

In summary, any assessment of countries’ development contributions should take these policy areas into account.

GPGs can be described simply as “goods, whose benefits or costs are of nearly universal reach or potentially affecting anyone anywhere” (Kaul 2013). In economics, examples of pure public goods are those where someone cannot be prevented from consuming a good (“non-excludable”) and where one person’s consumption does not detract from another’s (“non-rival”). Street lights have public good characteristics, and clean air is close to being a pure public good. GPGs are those public goods that are “global” or significantly transboundary. Examples include climate stability, security, and freedom from disease—we all benefit from these, but one country’s activity affects their overall supply.

A country’s contribution to GPGs is distinct from the policies in the prior section because one country’s activities affect all (or many) countries simultaneously, whereas in the prior section we were concerned with bilateral policy impacts. GPGs therefore require global—rather than bilateral—collaboration, and incentives are more diffuse.

In terms of countries’ contributions to GPGs, these can be positive or negative (and some activities—intentional or otherwise—create global public “bads” such as pollution). We can think of GPGs as assets that provide returns for a wide range of countries and people, but that countries can unilaterally add to, or deplete. The most obvious example is that of the climate. The Earth’s atmosphere is an asset that benefits the entire planet (to different degrees) by protecting us from the sun’s rays. Countries damage that asset when activity leads to greenhouse gas emissions, or they can contribute to that asset with forests, which absorb these gases and mitigate its depletion.

A key issue in accepting and allocating responsibility for maintaining GPGs is whether to consider the asset (i.e. the stock), or the current contributions to it (the flow). In this analysis, as we are concerned with current efforts, we focus on the latter, but it is clear that historic contributions are relevant in many cases, especially in assigning responsibility for enhancement or damage. Similarly, a country may contribute to a GPG by providing it directly, or by preserving it (which might include reducing or eliminating damaging activity).

Which GPGs should we be most concerned with? There is relatively little research assessing the value and importance of GPGs. Kenny et al. (2018) has produced a list and includes the categories of health (including vaccination); environment; economy; security; knowledge and technology; migration; and norms. I have attached a full list in Annex B. It is clear that countries’ actions in many of these areas have the potential to affect millions of lives overseas.

An important research question is the relative importance—and valuation—of these GPGs. There appears to be little work attempting to compare, say, the value of undertaking vaccination with climate mitigation. There have been calls for multilateral and bilateral development agencies to fund more projects that address these and other GPGs. As this would involve some trade-off of GPGs with other objectives—including poverty reduction—it is important for policy-makers to understand their relative value and effectiveness.

More effective development cooperation has the potential to lift large numbers of people out of extreme poverty. A common measurement between countries and institutions is extremely valuable in highlighting differences in approach, and it can proceed with or without targets, but it does rely on common standards and data. We can and should measure what we can now; if these comparisons are partial, this will signal clearly to policy-makers where better data is needed.

In this chapter, I have set out three main types of policy that matter for development, summarised in Table 11.4.

On development finance—including aid—there is a need for basic information on the quantity and terms of concessional finance. The Total Official Support for Sustainable Development measure is a helpful broader measure in development, but there remains a core question about how much concessional help is being funded by taxpayers, and it is important for recipients to understand whether any finance they receive is concessional or just effectively private lending through public bodies. New common standards and measures, which are not restricted to, or only led by, the OECD countries, are needed for all countries.

The political commitment to aid effectiveness and its measurement, at least in traditional OECD-DAC donors, has waned. The GPEDC, whose membership is broader, appears to lack buy-in from some key countries, although its data gathering is an important source of comparability in aid effectiveness, and it should be empowered and resourced to develop and extend its surveys. Countries will need to come together to agree how they will address effectiveness in the new environment where, for example, most of the extreme poor are potentially in fragile states. Without this focus on effectiveness, there is a risk that substantial sums of global aid are being used ineffectively, prolonging progress on the SDGs and keeping millions unnecessarily in poverty.

More generally, we need to be clear that concessional finance is not the only—or even perhaps the most important—policy supporting development. Policies around the flow of goods, people, and ideas are as important as those around capital. Similarly, GPGs receive too little attention, and little research exists that values these goods and considers the trade-offs between them and other areas. Common measures appear to be present in many of these issues.

The Center for Global Development will be continuing work in these areas, and it hopes to produce new measures of countries’ commitments to development, which apply to a broader group of major economies, and to evolve its measures of development finance. We hope these will support governments and leaders to reinvigorate their commitments to effectiveness, and help build a new consensus on effective aid and development that will accelerate the achievement of the global goals.