Social norms and energy conservation

Abstract

This paper evaluates a series of programs run by a company called OPOWER to send Home Energy Report letters to residential utility customers comparing their electricity use to that of their neighbors. Using data from randomized natural field experiments at 600,000 treatment and control households across the United States, I estimate that the average program reduces energy consumption by 2.0%. The program provides additional evidence that non-price interventions can substantially and cost effectively change consumer behavior: the effect is equivalent to that of a short-run electricity price increase of 11 to 20%, and the cost effectiveness compares favorably to that of traditional energy conservation programs. Perhaps because the treatment included descriptive social norms, effects are heterogeneous: households in the highest decile of pre-treatment consumption decrease usage by 6.3%, while consumption by the lowest decile decreases by only 0.3%. A regression discontinuity design shows that different categories of “injunctive norms” played an insignificant role in encouraging relatively low users not to increase usage.

Introduction

Climate change has emerged as one of the most important economic policy issues of the early 21st century, and many view energy efficiency as an appealing approach to reducing greenhouse gas emissions. Traditionally, economists and policymakers have focused on relative prices as the primary force driving energy demand. As a result, carbon cap-and-trade programs are the centerpiece of proposed climate change policies, and subsidies for energy efficient durable goods draw the vast majority of public energy efficiency funding in the U.S. (Gillingham et al., 2006).

There are three problems with price-based approaches to energy conservation. First, it has not been politically feasible to implement Pigouvian carbon taxes or a carbon emissions trading program in the U.S., suggesting that average wholesale energy prices are below social cost. Second, measuring the effects of an energy efficiency subsidy on energy use requires knowledge of the elasticities of demand for energy efficient durable goods and for energy conditional on capital stock. Lacking context-specific values of these parameters, subsidy-based programs are typically evaluated using a controversial approach called “deemed savings”; randomized controlled impact evaluations are exceedingly rare. A third problem is that while subsidies are in theory innocuous because they are transfers, they are in practice a large drain on increasingly-limited public funds.

Spurred by these problems, interest has dramatically increased in non-price energy conservation programs that are informed by insights from behavioral science and evaluated via randomized trials. Non-price interventions are typically inexpensive relative to subsidies, and as demonstrated by Bertrand et al. (2010) in the context of consumer finance, carefully-crafted psychological cues can have effects on demand that are comparable to large changes in relative prices. A critical challenge, however, is to craft interventions that are powerful and cost-effective when implemented at large scale.

This paper examines one of the most notable non-price energy conservation programs, which is run by a company called OPOWER. OPOWER mails Home Energy Report letters (HERs) that compare a household's energy use to that of similar neighbors and provide energy conservation tips. The neighbor comparisons were directly influenced by academic work showing that providing social norm information induces people to conserve energy (Schultz et al., 2007, Nolan et al., 2008). More broadly, the program was motivated by similar evidence on the power of social norms in a variety of domains, including voting (Gerber and Rogers, 2009), retirement savings (Beshears et al., 2009), and charitable giving (Frey and Meier, 2004). As of the end of 2010, OPOWER had contracts to run programs at 47 utilities in 21 states, including six of the largest ten utilities in the U.S.

The first parts of this paper are an impact evaluation of all of the OPOWER programs begun before the end of 2009. With nearly 600,000 households in treatment and control groups, this is one of the largest randomized field experiments in history. I show that the point estimates of the Average Treatment Effects (ATEs) of OPOWER's first 17 experiments range from 1.4 to 3.3%, with an unweighted mean of 2%.¹ While there is often concern over the durability of treatment effects in similar non-price interventions (Ferraro and Price, 2010), the Home Energy Reports appear to have constant or increasing effects as they are repeatedly delivered over the first two years of treatment.

These effect sizes have several different economic interpretations. First, different energy conservation programs are typically compared on a basis of program implementation cost per kilowatt-hour of electricity saved. OPOWER's initial set of programs have cost effectiveness ranging from 1.3 to 5.4 cents per kilowatt-hour, with an unweighted mean of 3.3. These results compare favorably to estimates for traditional energy efficiency programs, and because they are estimated using randomized trials, they are much more certain. The welfare effects, however, are ambiguous: the costs that households incur to reduce energy use are unobserved, as is the change in welfare from learning that one compares favorably or poorly to neighbors.

A second way of interpreting effect sizes is to calculate the energy price changes that would induce the same changes in demand. Calibrating with estimated price elasticities, I show that the effects of sending Home Energy Reports are equivalent to a 11 to 20% short-run price increase or a 5% long run price increase. Taken as a whole, these effects are remarkable: simply sending letters can significantly and cost-effectively affect energy use behaviors.

The remainder of the paper builds on theoretical predictions of heterogeneous treatment effects. In particular, many models predict that the “descriptive norm” element of the Home Energy Report treatment, in which a household's energy use is compared to that of its neighbors, would cause households that previously used more than the norm to decrease usage, but would cause households that used less than the norm to use more. Social psychologists call these unintended consequences “boomerang effects” (Clee and Wicklund, 1980), and they are certainly undesirable if the objective is to induce energy conservation. Combining data across all of OPOWER's experiments, I show that Conditional Average Treatment Effects are larger than 6% in the highest decile of pre-treatment usage and close to zero in the lowest decile, but even these households that compare most favorably to their neighbors do not increase energy use. In this sense, the OPOWER intervention does not cause a “descriptive norm boomerang effect.”

The Schultz et al. (2007) experiment that motivated OPOWER's work had found a boomerang effect for relatively low users. To combat this, they employed what social psychologists call “injunctive norms,” which convey that energy conservation is pro-social (Cialdini et al., 1990). Specifically, they added a treatment condition that included hand-drawn “smiley faces” on the descriptive norm feedback reports given to these relatively low users. Although the group of low users that received this injunctive norm did not use statistically significantly less energy than the group that that did not, this group's increase in energy use was also not statistically distinguishable from zero. Based on this result, it was believed that injunctive norms could eliminate the boomerang effect.

OPOWER's Home Energy Reports therefore include injunctive norms, which are defined based on sharp cutoffs. Households are labeled as “Great” if they use less than the 20th percentile of their neighbor comparison group, “Below Average” if they use more than the mean, and “Good” if they are in between. The “Great” group receives two “smiley face” emoticons, the “Good” group receives one, and the “Below Average” group initially received “frownie faces” until customer complaints ended this practice. The treatment effects are substantially different across the three groups, although this could be caused either by the categorizations or by other factors correlated with baseline energy use that could affect how households respond to the treatment: for example, high users may have lower-cost opportunities to conserve. Notice, however, that households that had used just more energy than the 20th percentile of their comparison group are in the limit identical to households that use just less energy, but the former are labeled “Good,” while the latter are labeled “Great.” Similarly, households using just more than the mean of their comparison group were labeled “Below Average,” while households using just less were labeled “Good.”

I use a regression discontinuity (RD) design to test for whether these normative categorizations cause differential effects on energy use. I show that while the treatment effects differ substantially for households in the three different categories, the causal effects of the categorizations themselves are “tightly estimated zeros.” Being labeled “Good” instead of “Great” has a differential treatment effect of less than 0.20 percentage points, or about one-tenth of the ATE. Being labeled “Below Average” instead of “Good” has a differential ATE of less than 0.16 percentage points. Therefore, the fact that we do not observe a descriptive norm boomerang effect is likely due not to the different categorizations. Instead, the potential effect is likely mitigated by the energy conservation tips or other aspects of the injuntive norms that affect all categories equally.

The paper proceeds by first giving more detail on the treatment and potential pathways of effects. The rest of Section 2 then provides background and descriptive statistics on OPOWER's experiments. Section 3 details the average treatment effects, from the econometric strategy to the parameter estimates and resulting cost effectiveness. In the spirit of Lalonde (1986), this section also documents the poor performance of non-experimental estimators. Section 4 discusses heterogeneous treatment effects and the RD design. Section 5 concludes.