Governors' Biofuels Coalition

The Energy Department’s projections of everything from future oil prices to coal use became much more inaccurate in the past decade and created wide uncertainty for businesses, according to new research from Carnegie Mellon University.

The study, published yesterday in Nature Energy, examined 17 metrics in the U.S. Energy Information Administration’s Annual Energy Outlook going back to 1949. The projections are considered critical for companies and utilities weighing long-term investments on things such as the number of power plants to build and which types of fuel sources to use.

In the last decade, there was a jump in EIA’s most extreme errors, where projections were off by more than nearly all others in history. This extreme unpredictability was greater from 2005 to 2014 than in the previous two decades. At the same time, volatility — or extreme annual fluctuations in EIA data — was greater over the same period than in the previous three decades.

“The key takeaway is that any projections or forecasts of the long-term development of the U.S. energy system should be used with extreme caution,” said Evan Sherwin, an analyst in CMU’s Department of Engineering and Public Policy and lead author of the study.

He recommended that EIA broaden its modeling to produce more estimates.

“To their great credit, EIA currently produces a number of such side cases, but many of them simply tweak one or two parameters,” he said.

The team assessed “extreme” year-to-year volatility by documenting the largest-ever upward and downward percent changes for each of the 17 quantities. With unpredictability, extremes were defined as errors greater than 95 percent of others historically.

The EIA indicators studied by the university included prices, consumption and production of coal, natural gas and oil; electricity prices; electricity sales; and energy consumption in residential, transportation and commercial sectors. EIA currently makes projections several decades out.

The period from 2005 to 2014 saw nine extreme changes in volatility over decades, or more than a quarter of the 34 total. Those included recent, unexpected percent changes in oil production, total energy consumption and electricity sales. Only the 1950s and ’60s, when the United States was on a growth spurt after World War II, witnessed similar annual shifts, according to the study.

Similarly, the largest-ever overestimates by EIA on 10 factors, including fossil fuel production and consumption, electricity sales and total energy use, all occurred in the last decade. There also was a jump in errors on nearly every metric in comparison to what occurred in the 1990s.

The paper doesn’t offer a full explanation for all the wild zigzags and faulty prognoses but points to the unanticipated 2007 to 2009 financial crisis and the hydraulic fracturing boom, which caused prices across the energy sector to drop and created competition for coal more than expected.

“The widespread adoption of horizontal drilling with hydraulic fracturing in shale formations, particularly after 2007, is unquestionably a major factor in the 17 percent increase in oil production in 2014 and the 54 percent decline in natural gas prices in 2009,” the paper says.

That doesn’t explain all of the inaccurate numbers, though, as extreme errors started to increase before 2007. Sherwin said rising oil demand because of China and peaking U.S. vehicle travel could have contributed to the uptick.

What is certain is that faulty projections can be costly. The paper outlines how errors from 2005 to 2014 could have made the difference between a profit and a loss for a liquefied natural gas export terminal.

In an accompanying article in Nature Energy, University of California, Irvine, earth system science professor Steven Davis said volatility and uncertainty can also have a negative impact on the environment.

“To the extent that market uncertainty prolongs the operation of less-efficient and more polluting old plants, it also prolongs such plants’ disproportionate impact on global climate and human health,” Davis said.

Sherwin cautioned that the study doesn’t automatically mean that future EIA projections will be as inaccurate. “If we have learned anything from this analysis, it is that such long-term predictions should be taken with much more than just a grain of salt,” he said.

EIA did not respond to request for comment.

The agency has come under fire in the past from critics claiming it lowballed estimates of renewable power (Greenwire, April 5).

Previous | Next