Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
MTZ-Fachkongress

Antriebe und Energiesysteme von morgen


Austausch von Automobil- und Energiewirtschaft

Am 14./15. Mai geht es in Chemnitz um die Mobilitätswende durch Elektro- und Wasserstofffahrzeuge.
Erweiterte Suche
Anmelden
nach oben
Energy Efficiency
Erschienen in: Energy Efficiency 4/2014

01.08.2014 | Original Article

Experimentation and the evaluation of energy efficiency programs

verfasst von: Edward Vine, Michael Sullivan, Loren Lutzenhiser, Carl Blumstein, Bill Miller

Erschienen in: Energy Efficiency | Ausgabe 4/2014

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The use of experiments— particularly randomized controlled trials (RCTs) where subjects are randomly assigned to treatment and control conditions—has rarely been applied to the process of improving the design of energy efficiency programs and, more fundamentally, to determining the net savings from energy efficiency programs. This paper discusses the use of experimentation in the energy efficiency program field with the hope of explaining how these experiments can be used, and identifying the barriers to their use will cause more experimentation to occur. First, a brief overview of experimental methods is presented. This discussion describes the advantages and disadvantages of conducting experiments in the context of the development and evaluation of energy efficiency programs. It then discusses barriers to the use of experimental methods (including cost and equity issues) and suggests some ways of overcoming these barriers. Finally, recommendations are made for implementing key social experiments, discussing the types of energy efficiency programs and issues that can make use of experimentation and variables that one might use for selecting treatments.
Vorheriger Artikel Using dynamic model for determination of heat losses in cold weather from a typical house in Tabriz constructed by lightweight concrete blocks
Nächster Artikel Retrofitting social housing: reflections by tenants on adopting and living with retrofit technology

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren
Fußnoten
1
Taken from “The Ballad of East and West,” a poem by Rudyard Kipling and first published in 1889.
 
2
This paper does not address the use of “observational studies” where investigators observe subjects and measure variables of interest without assigning treatments to the subjects. The treatment that each subject receives is determined beyond the control of the investigator. For the evaluation of energy efficiency programs, the program itself is the treatment, so observational studies would be of limited use for evaluators.
 
3
In this paper, we do not review the experiments conducted in these other fields, but we encourage the reader to read them for seeing what lessons were learned in these experiments.
 
4
These benefits not only include the direct positive impacts resulting from a program (e.g., energy savings, improved environmental comfort, and increased number of retailers offering energy efficiency products) but also those impacts that lead to improved program design and implementation, including the offering of new energy efficiency technologies that were not originally provided in the program.
 
5
Energy efficiency may not be sufficient to justify this, but the challenges raised by introducing new technologies, many associated with the “smart grid,” provide other situations well served with good experimental design.
 
6
Universities have established federally mandated human subject principles and procedures that assure informed consent and good risk/benefit ratios for academic research. These could be a stumbling block to the design of experiments if universities were involved.
 
7
There are no hard and fast rules for defining a small sample size. It is more critical to compare the groups on important variables before interpreting or modeling the results of an experiment. (e.g., check to make sure that the treatment and control groups were not different from the onset of the experiment). Moreover, it is also highly desirable to employ balanced experimental designs (i.e., treatment and control groups of equal size) as it simplifies the estimation of impacts and guards against errors in estimating standard errors.
 
8
We design samples such that we can say we are 95 % confident (for example) that the sampling error on the dependent variable is no more than ±2 % (for example) and that 80 % of the time, we will be able to find a difference as small as 1 %. If the acceptable sampling error is very small, as it is when we are trying to find a very small effect like 1–2 %, then the sample size must be relatively large. For example, in testing for a 1 % change in energy consumption resulting from a behavioral intervention, the required sample size will range between 10,000 and 15,000—if monthly electricity or gas consumption is under study. Likewise, if the error we are willing to tolerate is in the range of ±10 %, only a few hundred observations will guard against the selection of treatment and control groups that are not statistically identical.
 
9
It is worth stressing that the RED design measures the impact of encouragement to take the treatment. Insofar as the encouragement causes an increase in the uptake of the treatment, it provides a legitimate measure of the impact of the treatment. However, if the encouragement also causes some other action that affects the outcome variable of interest, the effect of that action will be measured as a treatment effect. This can and has occurred in some well-known experiments involving RED designs, and it is a reason for caution in substituting a RED design for an RCT unless it is necessary to do so.
 
Literatur
Allcott, H. (2011). Social norms and energy conservation. Journal of Public Economics, 95(9–10), 1082–1095.CrossRef
Ayres, I., Raseman, S., & Shih, A. (2009). Evidence from two large field experiments that peer comparison feedback can reduce residential energy usage. NBER Working Paper No. 15386, Washington, DC: National Bureau of Economic Research.
Blumstein, C. (2010). Program evaluation and incentives for administrators of energy-efficiency programs: can evaluation solve the principal/agent problem? Energy Policy, 38, 6232–6239.CrossRef
Campbell, D. (1969). Reforms as experiments. American Psychologist, 24, 409–429.CrossRef
Campbell, D. (1988). The experimenting society. In D. Campbell & S. Overman (Eds.), Methodology and epistemology for social science: selected papers (pp. 290–314). Chicago, IL: University of Chicago Press.
Cappers, P. (2011). Personal communication with Peter Cappers, Lawrence Berkeley National Laboratory, February 25.
Coalition for Evidence-Based Policy (2009). Program Evaluation: A Variety of Rigorous Methods Can Help Identify Effective Interventions, Washington, DC: US General Accounting Office.
Cook, T., & Campbell, D. (1979). Quasi-experimentation: design and analysis issues for field settings. Chicago: Rand McNally.
Cook, T., & Shadish, W. (1994). Social experiments: some developments over the past fifteen years. Annual Review of Psychology, 45, 545–580.CrossRef
Cotterill, S., John, P., Liu, H., & Nomura, H. (2009). Mobilizing citizen effort to enhance environmental outcomes: a randomized controlled trial of a door-to-door recycling campaign. Journal of Environmental Management, 91(2), 403–410.CrossRef
Duflo, E., Glennerster, R., & Kremer, M. (2007). Using randomization in development economics research: a toolkit. London, UK: Centre for Economic Policy Research.
Ehrhardt-Martinez, K., & Laitner, J. (2009). Pursuing energy-efficient behavior in a regulatory environment: motivating policymakers, program administrators, and program implementers. Berkeley: California Institute for Energy and Environment.
Electric Power Research Institute [EPRI]. (2010). Guidelines for designing effective energy information feedback pilots: research protocols. Palo Alto: Electric Power Research Institute.
Faruqui, A., & Sergici, S. (2009). Household response to dynamic pricing of electricity: a survey of the experimental evidence. Washington, DC: The Brattle Group, Edison Electric Institute and the Electric Power Research Institute.
Faruqui, A., & Wood, L. (2008). Quantifying the benefits of dynamic pricing in the mass market. Washington: Edison Electric Institute.
Faruqui, A., Sergici, S., & Sharif, A. (2009). The impact of informational feedback on energy consumption: a survey of the experimental evidence. Washington: The Brattle Group.
Fischer, C. (2008). Feedback on household electricity consumption: a tool for saving energy? Energy Efficiency, 1, 79–104.CrossRef
Greenberg, D., & Shroder, M. (2004). The digest of social experiments (3rd ed.). Washington: Urban Institute Press.
Haynes, L., Service, O., Goldacre, B., & Torggerson, D. (2012). Test, learn, adapt: developing public policy with randomised controlled trials. London: Cabinet Office Behavioural Insights Team.
Hines, P. (2011). Personal communication with Paul Hines, University of Vermont, May 1.
Joskow, P., & Schmalensee, R. (1986). Incentive regulation for electric utilities. Yale Journal on Regulation, 4, 1–49.
Laitner, S., Ehrhardt-Martinez, K., & Knight, C. (2009). The climate imperative and innovative behavior: encouraging greater advances in the production of energy-efficient technologies and services. Berkeley: California Institute for Energy and Environment.
Lutzenhiser, L. (2009). Behavioral assumptions underlying California residential sector energy efficiency programs. Berkeley: California Institute for Energy and Environment.
Megdal, L., & Bender, S. (2006). “Evaluating media campaign effectiveness: others do it why don’t we?” Proceedings of the 2006 summer study on energy efficiency in buildings. Washington: American Council for an Energy-Efficient Economy.
Miguel, E., & Kremer, M. (2004). Worms: identifying impacts on education and health on treatment externalities. Kremer Econometrica, 72(1), 159–217.
Shadish, Cook W, T. & Campbell, D. (2002). Experimental and quasi-experimental design for generalized causal inference, Houghton Mifflin.
Stern, P. (2008). Environmentally significant behavior in the home. In A. Lewis (Ed.), The Cambridge handbook of psychology and economic behavior (pp. 363–382). Cambridge: Cambridge University Press.CrossRef
Sullivan, M. (2009). Using experiments to foster innovation and improve the effectiveness of energy efficiency programs. Berkeley: California Institute for Energy and Environment.
Summit Blue Consulting (2009). Impact evaluation of OPOWER SMUD pilot study. Boulder, CO: Summit Blue Consulting [now called Navigant Consulting].
Tiedemann, K. (2011). Behavioral change strategies that work: a review and analysis of field experiments targeting residential energy use behavior. In K. Erhardt-Martinez & S. Laitner (Eds.), People-centered initiatives for increasing energy savings, E-book. Washington: American Council for an Energy-Efficient Economy. Chapter 21.
Todd, A., Stuart, E., Schiller, S., & Goldman, C. (2012). Evaluation, measurement and verification (EM&V) of residential behavior-based energy efficiency programs: issues and recommendations, DOE/EE-0734, State and Local Energy Efficiency Action Network. Washington: US Department of Energy.
Torgerson, D., & Torgerson, C. (2008). Designing randomised trials in health, education and the social sciences. Basingstoke: Palgrave Macmillan.CrossRef
Vine, E. (2008). Strategies and policies for improving energy efficiency programs: closing the loop between evaluation and implementation. Energy Policy, 36, 3872–3881.CrossRef
Vine, E., Kushler, M., & York, D. (2007). Energy myth ten—energy efficiency measures are unreliable, unpredictable, and unenforceable. In B. K. Sovacol & M. A. Brown (Eds.), Energy and American society—thirteen myths. New York: Springer.
Vine, E., Hall, N., Keating, K., Kushler, M., & Prahl, R. (2012). Emerging issues in the evaluation of energy-efficiency programs: the US experience. Energy Efficiency, 5, 5–17.CrossRef
Vine, E., Hall, N., Keating, K., Kushler, M., & Prahl, R. (2013). Emerging evaluation issues: persistence, behavior, rebound and policy. Energy Efficiency, 6, 329–339.CrossRef
Metadaten
Titel
Experimentation and the evaluation of energy efficiency programs
verfasst von
Edward Vine
Michael Sullivan
Loren Lutzenhiser
Carl Blumstein
Bill Miller
Publikationsdatum
01.08.2014
Verlag
Springer Netherlands
Erschienen in
Energy Efficiency / Ausgabe 4/2014
Print ISSN: 1570-646X
Elektronische ISSN: 1570-6478
DOI
https://doi.org/10.1007/s12053-013-9244-4

Weitere Artikel der Ausgabe 4/2014

Energy Efficiency 4/2014 Zur Ausgabe

Original Article

Performance of ice slurry as cooling fluid for an indoor ice rink

Original Article

Retrofitting social housing: reflections by tenants on adopting and living with retrofit technology

Original Article

Looking for free riding: energy efficiency incentives and Italian homeowners

Original Article

End-user interfaces for energy-efficient semantically enabled smart homes

Original Article

Disaggregation of the electric loads of small customers through the application of the Hilbert transform

Original Article

Potential of new lighting technologies in reducing household lighting energy use and CO2 emissions in Finland