Policy, Program and Project Evaluation

Table of Contents

1. Frontmatter

2. Chapter 1. Introduction

   Anwar Shah

   Abstract

   This chapter provides an overview of various chapters in this book. It notes that in recent decades, evaluation as a discipline has progressed from providing answers to questions like economic viability, cost-effectiveness, and efficiency to questions that are critical for effective planning, financing, design, implementation, citizens’ monitoring, and ultimate success of a program. Now evaluation can provide valuable assistance in defining a problem, identifying program targets, designing of interventions, identifying winners and losers and organizational strengths and weaknesses, assessing the quality of interventions and performance of program delivery and its impact, suggesting modifications and alterations, and ultimately guiding a program or project to its successful end. Evaluation is useful for program or project appraisals but also serves an important tool in operational management and earning public confidence and trust.

3. Chapter 2. A Primer on Public Sector Evaluations

   Saubhik Deb, Anwar Shah

   Abstract

   This chapter provides a brief survey of the program evaluation methods, their objectives, strengths, and weaknesses. The chapter first presents methods like cost-benefit analysis, cost-effectiveness analysis, the social marginal cost of funds analysis, and data envelopment analysis that are appropriate for guiding efficient allocation and utilization of resources. This is followed by a discussion on multiple-criteria evaluation (MCE), which is much more holistic. Apart from efficiency, it addresses issues like the relevance of a program, effectiveness of a program in achieving its objectives, and sustainability of the program benefits. Newer MCE approaches such as the Iron Triangle and the Results-Oriented Management Evaluation are briefly sketched. Finally, theory-based evaluation is outlined, where the focus is not just on whether a program succeeds or fails but also on how and why a program succeeds or fails.

4. Chapter 3. Economic Evaluation of Projects

   Robin Boadway

   Abstract

   This chapter summarizes the economic principles that would help evaluate individual investment projects, general expenditure programs, government policies, and the regulation of economic activities. The objective of the project and program evaluation is to measure the net benefit of a project or program to those individuals in the society who are affected by such public initiatives. The chapter begins with theoretical perspectives on cost-benefit analysis and then proceeds to discuss the difficulties of implementing such an analysis and provides guidance on practical ways of dealing with those problems. The chapter recognizes project evaluation as “an art, though one with scientific underpinnings.” The chapter concludes with a plea for incorporating new insights into project evaluation from recent advances in economic theory and development economics, such as imperfect information, implications of unfettered markets, and the underground economy.

5. Chapter 4. The Marginal Cost of Public Funds: Concept, Measurement, and Applications

   Bev Dahlby

   Abstract

   This chapter elaborates upon the marginal cost of public funds (MCF)—a measure of the cost imposed on the private sector in raising an additional dollar of tax revenue. In this chapter, we provide a basic introduction to the MCF concept, with numerical examples using a simple demand and supply model. We then describe three approaches to measuring the MCF with examples from previous studies. Finally, we review two examples of the use of the MCF in public policy analysis. The first is an assessment of the gains from a tax reform that shifts some of the tax burden from the corporate to the personal income tax base in Alberta. The second example illustrates how the MCF can be used to derive optimal matching rates for intergovernmental infrastructure grants in a federation.

6. Chapter 5. Theory-Based Evaluations: Guidelines for Case Studies in Program and Project Evaluation

   Ewa Tomaszewska

   Abstract

   This chapter deals with theory-based evaluation. It traces the factors that contributed to a specific outcome for the project. This is a challenging task for project evaluation as it requires establishing a causal chain based upon theoretical considerations and analyzing data on various links in this causal chain. This chapter provides a guidebook for those interested in conducting case studies in project evaluation determining the impact of various projects in combating corruption. The author provides a conceptual framework and data requirements for such evaluation case studies for privatization programs, judicial and legal reforms, civil service reforms, trade liberalization, tax administration reforms, and direct anticorruption activities (anticorruption agencies, the office of the ombudsman, transparency rules, and decentralization).

7. Chapter 6. A Framework for Evaluating Anti-Corruption Policies and Programs

   Jeff Huther, Anwar Shah

   Abstract

   This chapter provides a framework for the evaluation of anti-corruption programs. The impact of corruption on public service delivery performance and poverty alleviation is widely recognized. A wide consensus has also recently emerged that corruption is a symptom of failed governance and hence curtailing corruption requires addressing the causes of bad governance. Nevertheless, the menu of potential actions to curtail corruption is very large, so a framework is needed that guides ordering potential actions. Prioritization of various actions depends on both the conceptual and empirical views of what works and what does not work in the context of a specific situation. Such a framework is also needed for evaluating country anti-corruption programs and policies. The chapter concludes that path dependency is critical in determining the relative efficacy of various anti-corruption programs.

8. Chapter 7. Evaluating External Analytical Advice on Budgetary Institutions and Allocations

   Jeff Huther, Anwar Shah

   Abstract

   This chapter presents a framework for evaluating the quality and timeliness of Public Expenditure Review (PER) as well as conducting a review of its impact. Public Expenditure Review (PER) is a widely used tool by the development assistance community to develop advice on budgetary institutions and allocations. This tool has also been used by both industrial and developing countries as an aid to public sector reforms. The chapter highlights specific elements to consider and to rate in such an analysis and how to develop final cost-efficiency and net benefit ratings.

9. Chapter 8. Inter-Sectoral Allocation Choices

   Stuart Landon

   Abstract

   This chapter addresses the impact of the composition of expenditures and the design of programs on the effectiveness of government expenditures. No clear empirical relationship has been found between growth and public expenditures. This may be due to differences in the efficacy of expenditure programs across countries, differences that are not reflected in expenditure data. If spending has not been implemented efficiently, or allocated optimally across spending categories, it may be possible to increase growth and welfare by redesigning spending programs, or shifting spending from one sector to another. This chapter also identifies specific problems and constraints that are likely to hinder the success of spending programs, so have implications for the choice and design of spending priorities. Although there are no universal rules, a review of the empirical evidence suggests that high priority should be assigned to develop an effective legal system to protect and enforce property rights, effective financial regulations, and maintenance of existing infrastructure. Spending priority should also be given to improvements in the quality and quantity of primary education and basic health care, water and sanitation, and well-targeted consumption subsidy programs for the very poor.

10. Chapter 9. Evaluation of Decentralization Programs

    Melville McMillan

    Abstract

    This chapter outlines an approach to evaluating decentralization initiatives. The basic methodology for evaluating individual decentralization projects is outlined with an elaboration of the components of decentralization and steps to be taken in the evaluation. The method for extending the evaluation across many projects follows. The problem of selecting or sampling the projects to be evaluated is discussed. Special treatment is given to projects on community-driven development. The chapter also presents thoughts on bringing the various analyses of a rather diverse set of projects together and putting the results into perspective.

11. Chapter 10. Evaluating the Conditionality of External Development Assistance Programs

    Anwar Shah

    Abstract

    This chapter evaluates the conditionality of development assistance in terms of its intended and unintended consequences. The chapter provides conceptual perspectives from the game theory, the public choice, the fiscal federalism, the political economy, the new institutional economics, and the new public management literature on the design of external assistance and its potential impacts. It shows how neglect of these conceptual considerations in the past resulted in the lack of effectiveness of aid conditionality and the waste of such assistance. It highlights the emerging consensus by the development assistance community on both the instruments of development finance and the associated conditions. It also briefly notes the progress, or lack thereof, for practice to conform to the emerging consensus. It cites examples where an inappropriate design of conditionality led to adverse consequences for project and program outcomes. Finally, the chapter provides lessons on major issues in the conditionality of development assistance.

12. Backmatter