In 2007, a team of federal energy researchers faced this question: Could their work be done on the cheap?
Their agency, the Energy Information Administration, was facing budget pressure. For 25 years, it had collected comprehensive data on commercial buildings, which use a fifth of the country's energy: bowling alleys, fast-food restaurants, hospitals and more.
Historically, EIA had used an army of surveyors to walk America's streets and update its building rosters. That gave researchers back at the office confidence that the data were accurate.
But putting all those boots on the ground was expensive. So for the 2007 survey, to save money, the researchers decided to take a risk.
They hired a well-established contractor at the University of Chicago to use an experimental method that needed fewer boots on the ground. EIA expected to save some 30 percent of the usual survey cost, a few million dollars.
The plan failed. Two weeks ago, EIA announced it won't release results from its 2007 survey because the contractor's data were shoddy.
"We've learned that we cannot cut corners on this survey," said Tom Leckey, director of the Office of Consumption and Efficiency Statistics at EIA.
Speaking of the experimental method, he said "it promised lower costs; it delivered on the lower costs, but it did not deliver on the validity of the sample."
Turning off the data headlights
Energy statistics programs are on the menu in the federal budget debate, leaving some energy experts fearful that the country will blind itself to energy trends in order to save relatively small sums.
In the recent budget compromise, Congress cut EIA's funding by 14 percent. EIA has chosen to suspend its work on the 2011 commercial building survey, known as CBECS.
That, plus the botched research of 2007, means the country's last snapshot of its commercial buildings comes from 2003.
Cliff Majersik, executive director of the Institute for Market Transformation, called it "penny-wise and pound-foolish."
"CBECS costs $4 million a year, and we can't find $4 million a year to have good data by which to measure ourselves and measure our progress?" he said. Energy efficiency advocates like Majersik argue that an entire community depends on these data to make decisions about buildings.
The Energy Star program at U.S. EPA, for example, lets building owners rate themselves against the average building nationwide. The comparison is based on data from CBECS.
Buildings that want a "green" label -- such as certification from the U.S. Green Building Council's Leadership in Energy and Environmental Design (LEED) program or from Green Globes -- use Energy Star as a baseline.
Some cities, such as New York and Washington, D.C., require buildings to release their Energy Star scores, so these policies also rely on CBECS, according to the National Institute of Building Sciences.
"We can work on 2003 data, and we are," said Al Skodowski, director of sustainability for Transwestern, a real-estate company. "But if we're going to continue to drive efficiency, we need to compare against the most current data set that's available."
If EIA gets funding for the 2011 CBECS, it will go back to the old way of collecting data, Leckey said. The agency has found a new contractor and was four months in before Congress shut off the tap.
Dropping updates to a computer model
Other cuts at EIA will affect NEMS, the software program used by the government, academics and researchers to study energy supply and demand, as well as how energy policies affect the U.S. economy. EIA said it will suspend updates to the program.
The government has used NEMS to model at least three climate bills, including H.R. 2454, the legislation offered by Reps. Henry Waxman (D-Calif.) and Ed Markey (D-Mass.) that passed the House in 2009, for their effects on jobs and energy costs (ClimateWire, Dec. 16, 2009).
NEMS has also provided ammunition for policy advocates of all stripes. David Kreutzer, a research fellow in energy economics and climate change at the Heritage Foundation, said he's using the NEMS model to study how EPA's greenhouse gas regulations will affect the economy.
Kreutzer said although NEMS is useful, "when you have to cut budgets significantly, you're going to lose some things that are valuable."
He doubted the private sector would notice. "We're not walking blind; this is not the only source of energy information, and it's not clear that it's the primary one upon which markets depend," he said.
For example, he said, a skilled businessman could look at weather reports or orders for drilling equipment in oil-rich states to predict where energy will be used or produced.
Others, such as Michael Greenstone, a professor of economics at the Massachusetts Institute of Technology, argue that NEMS is vital to cut through the food fight of reports and spreadsheets that flood Washington.
"It's not in the public's interest to have models only provided by the private sector," said Greenstone, who was chief economist for the White House Council of Economic Advisers from 2009 to 2010. "The important thing to make good public policy is to have models that reflect the entire country's needs, that are not being paid for by an industry group or an environmental group."
Other EIA cuts may cause fewer ripples in the energy world. EIA will no longer publish its annual greenhouse gas inventory, which has slight differences from the one EPA submits to the United Nations each year.
Perry Lindstrom, who has worked on EIA's inventories, said EPA bases its report on EIA's other energy data, not EIA's greenhouse gas inventory. He said EIA will continue to send those data to EPA and that it won't be affected by the budget cuts.