Correction appended.
For energy efficiency, it’s time to put up or shut up.
With energy efficiency playing a big role in U.S. EPA’s Clean Power Plan for curbing the power sector’s greenhouse gas emissions and the Department of Energy continuing its rollout of power-sipping standards for appliances, hard questions are being asked about whether taxpayers are getting a good bang for their buck.
It’s economists versus advocates.
"In a way, we are speaking different languages," explained Arik Levinson, a professor at Georgetown University and a former senior economist for President Obama’s White House Council of Economic Advisers.
"I define savings from energy efficiency as the difference between consumption with energy-efficient appliances and an otherwise identical alternative world without energy efficiency," he explained.
On the other side of the ring, Levinson said, are engineers: "Lawrence Livermore Lab wants to just run two air conditioners side by side — one energy efficient and one not." Even with very sophisticated house mockups and models, he said, "that lab is not the place in the world where it is going to be used."
Levinson’s views and research are unpopular with efficiency advocates. They say he and other economists miss the mark with error-riddled studies and ignorant comparisons that just might cause policymakers to shy away from low-cost solutions for meeting power demands and cutting emissions.
"Good research is immensely valuable, but poor or mischaracterized research can be far more harmful in the short term, given the media’s penchant for counterintuitive stories," said Steven Nadel, executive director of the American Council for an Energy-Efficient Economy (ACEEE). "The result is to hinder the ability to make rational policy decisions."
Merrian Borgeson of the Natural Resources Defense Council added in a recent blog post, "Energy efficiency programs are continually tested and measured by state regulators — and the benefits generally far exceed the costs of the programs, or they wouldn’t go forward."
The two sides clashed this summer over a contentious study that found Michigan households that had made efficiency improvements through the federal weatherization program in 2009 realized less than half of the projected energy savings and saved only half of the $5,000 they spent on upgrades.
On the plus side, improvements did cut energy use up to 20 percent, says the report by E2e, a group of economists, engineers and behavioral scientists from the University of California, Berkeley; the University of Chicago; and the Massachusetts Institute of Technology’s Center for Energy and Environmental Policy Research.
"If taken out of context and generalized way beyond any justification, this new E2e working paper could be misused to attack critically important energy efficiency policies and programs," Martin Kushler, an ACEEE senior fellow, wrote in a critique of the report.
Kushler sees an alternative motive for economists to push for a "market-based approach" — namely a carbon tax — as the best solution for slashing heat-trapping greenhouse gas emissions. This is misleading, he said, because it is not an either-or scenario.
A carbon tax "should be seen as a complement to, not a replacement for, traditional energy efficiency programs," Kushler wrote.
Real world versus projections
Started 2½ years ago, E2e researchers "seek to understand the difference between what is technically possible and what is practically achievable for energy efficiency in a wide variety of settings," using "cutting-edge analytics and big data."
But new technology hasn’t made explaining the analyses easier.
Although there are many efficiency studies, the E2e white paper — by Meredith Fowlie of UC Berkeley, Michael Greenstone of the University of Chicago and Catherine Wolfram of UC Berkeley’s Haas School of Business — drew attention because the authors used "randomized controlled trials" to assess the program.
They randomly assigned about 30,000 people eligible for weatherization to either receive efficiency upgrades or to join a control group to compare the results — a technique often used in clinical trials for medicine and social sciences. They also eliminated another factor that can skew results: customer self-selection — people who already are efficiency go-getters getting the upgrades.
In addition to finding that the Michigan households saved just $2,400 after installing $5,000 worth of upgrades, the researchers said "broader social benefits" of the program, namely cutting carbon dioxide emissions, also were not cost-effective.
The study found CO2 reductions through slashed energy use of the weatherization upgrades cost $329 per ton, compared with the federal government’s social cost of carbon, $38 per ton.
"The urgency of the climate challenge means that it is critical to identify cost-effective strategies that will deliver real greenhouse gas emissions reductions," the E2e authors wrote in a blog responding to critics of their study.
"Energy efficiency is a crucial component of most climate change mitigation plans, underscoring the importance of developing a body of credible evidence on the real-world versus projected-returns on energy efficiency investments in the residential sector and beyond."
Efficiency advocates panned E2e for drawing broad conclusions about the effectiveness of energy efficiency programs from just one study.
"No one would suggest [weatherization programs] should be considered as entirely, or even primarily, a mechanism to fight climate change," Kushler said.
They also said the study compared the wrong savings estimates with the end results; failed to use "incremental" costs of appliances, which account for the fact that everyone usually pays some base price for heating and cooling; and misrepresented the goal of the weatherization program. The efficiency program, they argued, tries to improve the safety and quality of life for the residents, not necessarily just saving money and energy.
ACEEE’s Nadel also said the study’s approach cannot be generalized to other households and programs that have different geography and characteristics — highlighting one of the biggest issues in analyzing and implementing efficiency programs, comparing apples with apples.
The E2e authors counterpunched.
"Given similarities between the setting we evaluate and other efficiency applications, these findings likely generalize to a broader set of residential efficiency investments," they wrote, although they did caution about over-generalization, as well. "There is logic behind this implication, while also acknowledging the need for further experiments on the returns to energy efficiency investments in other contexts."
Similar differences on how to analyze efficiency’s impact erupted earlier this year after Georgetown’s Levinson published a white paper that found California’s famed building codes yielded homes that consume no less electricity than older buildings.
Specifically, he found new homes do not use 80 percent less energy — the number used by the California Energy Commission to justify the codes, he said.
Levinson’s study gained wider attention when the podcast "Freakonomics" did a show on his paper, offering him a platform to criticize the Obama administration’s focus on energy efficiency as a de facto environmental policy based on estimates of how efficiency cuts energy use and emissions rather than in-the-field experience.
Advocates from ACEEE and NRDC blasted Levinson’s premise that California regulators had set up the 80 percent cut in energy use as a target and took shots at other results from his paper. For instance, they said, excluding data associated with natural gas, whose heating and cooling elements are a significant component of energy efficiency’s success, makes his conclusions about California’s program meaningless.
In an interview, Levinson said he is working on a second draft of the paper that would include data from natural gas. And he defended his work, saying it begs the crucial question about the importance of analyzing energy efficiency programs in the real world — especially when figuring out the baseline or "business as usual" predictions that form the foundation of assessing the effectiveness of efficiency measures.
"Maybe people would have purchased fancy air conditioners anyway without building codes" or appliance standards, Levinson said.
"That is probably a stretch, but it is not like they would have done nothing. There is something between what people were doing in the 1970s and what people are required to do based on codes now, which they would have adopted anyway."
Levinson is quick to clarify he did not look at any other benefits besides energy reduction, such as a more comfortable environment or money savings, and he thinks energy efficiency is helpful for many reasons, but its limitations need to be better understood.
"Suppose we … do impose a carbon tax. Then I think energy efficiency would be one of the market responses to that," he said. "Efficiency regulations we pass now will have been sort of ‘no regrets’ — we won’t regret having invested in those."
He added: "I think that is the best case to be made for energy efficiency. But if energy efficiency is our only policy, then I think we are in trouble."
‘Multiplying effects’
The clash is further fueled by the fact that there are also many and varied approaches to measuring the success of energy efficiency programs across states, municipalities and utilities.
That’s a problem.
Says DOE: "The inconsistent results have limited the acceptance of reported energy savings."
So DOE, state regulators, utilities and other experts created the Uniform Standards Program. That program’s measurement and verification protocols are seen as significant to the success of the Clean Power Plan and securing efficiency’s role in the U.S. energy system.
The first set of voluntary protocols were issued in 2013 for commercial lighting and residential furnaces, and the group has continued to add to them through this January, when guidelines were offered for data centers and behavior-based programs.
"The goal is to strengthen the credibility of energy efficiency programs by improving the consistency and transparency of how energy savings are determined," DOE said.
Still, energy efficiency in many ways resists measurements in hard numbers.
"We are getting more sophisticated. I don’t know if we are getting better" at measuring efficiency’s impact, said Keith Dennis of the National Rural Electric Cooperative Association. "We are probably getting at the more complicated stuff, but we are also requiring the more complicated stuff more often."
He added: "It tends to be thought of as a one-size-fits-all, but it doesn’t necessarily shake out that way."
For example, reducing energy use means one thing when the primary power source is coal, he said, but it’s another when power is drawn from a hydroelectric dam or wind.
"All of these things need to be considered: Where are the opportunities?" he said. "Where do we want to spend the money? And is it worth the investment?
"It doesn’t take anything away from the fact that energy efficiency is a great thing. It is a very complicated issue."
Many cooperatives invest in efficiency improvements on the grid side of the system — such as improving loss of electricity during delivery — but it is not measured, and it would be very tough to measure, said Tracy Warren, a senior communications officer at NRECA.
Holmes Hummel, founder of Clean Energy Works, which advises utilities that invest in energy efficiency upgrades using a system called Pay As You Save, said the elimination of wasted energy also means more than just the dollars and emissions saved.
"Every time we save energy, we present an opportunity to use it for a better purpose," she said. "A child’s meal, elder’s health care or a new piece of technology. It means we are more competitive, multiplying the benefits of energy efficiency."
Correction: Holmes Hummel is the founder of Clean Energy Works, which advises utilities that invest in energy efficiency upgrades using a system called Pay As You Save. An earlier version of this story misidentified Hummel.
