Theory-based policy evaluation of 20 energy efficiency instruments

Realizing a 20% energy efficiency improvement in Europe by 2020 requires the introduction of good new energy efficiency policies as well as strengthening and enforcing the existing policies. This raises the question: what characterizes good and effective energy efficiency policies and their implementation? Systematic ex post evaluation of energy efficiency policies can reveal factors determining not only what works and what does not but also explain why. Ex post evaluation of 20 energy efficiency policy instruments applied across different sectors and countries in Europe among others showed that ex post evaluation does not yet have a high priority among policy makers: Often, quantitative targets and clear timeframes are lacking, and monitoring information is not collected on a regular basis. Our analysis, however, did reveal some general factors in the process of design and implementation of policy instruments that appear as important including (1) existence of clear goals and a mandate for the implementing organization, (2) the ability to balance and combine flexibility and continuity, (3) the involvement of stakeholders, and (4) the ability to adapt to and integrate adjacent policies or develop consistent policy packages. The analysis was performed using a uniform methodology called “theory-based policy evaluation”. The general principle behind this approach is that a likely theory is drawn up on the program’s various steps of logic of intervention to achieve its targeted impact in terms of energy efficiency improvement. The approach has several benefits over other ex post evaluation methods because (1) the whole policy implementation process is evaluated and the focus is not just on the final impacts, (2) through the development of indicators for each step in the implementation process, the “successes and failures” can be determined to the greatest extent possible, and (3) by applying this approach, we not only learn whether policies are successful or not but also why they succeeded or failed and how they can be improved.