WASHINGTON, DC – Oil prices have plummeted by about 65% from their peak in June 2014 (see chart below), and there is now intense debate about why. One thing we know for sure is that the oil market has undergone structural changes, thus making this latest episode different from previous dramatic price fluctuations.

The collapse in prices has been driven in part by supply-side factors. These include the United States’ rapid increase in shale-energy production in recent years, and the US government’s decision to end a 40-year crude-oil export ban. Moreover, oil output from war-torn countries such as Libya and Iraq has exceeded expectations, and Iran has returned to world oil markets following its nuclear agreement with the world’s major powers. And Saudi Arabia, the largest member of the Organization of the Petroleum Exporting Countries (OPEC), has increased production to defend its market share.

With this glut in oil, many commentators are now asking if OPEC still matters. High demand for oil since 2000 gave OPEC, and Saudi Arabia in particular, significant influence over prices, but it also spurred investments in higher-cost production methods in other locales, such as oil sands mining in Canada and ultra-deepwater oil extraction in Brazil.

Because of the delay between investment and production for conventional oil production, these projects in non-OPEC countries peaked around the same time the oil market began to slow down, and when expectations about future demand for oil started to falter.

This dynamic prompted OPEC to change its response to price fluctuations. In the past, OPEC, and Saudi Arabia in particular, would stabilize the oil market by cutting production when prices fell too low and increasing output when prices rose too high, relative to OPEC’s price target. This time around, however, at a November 2014 OPEC meeting, Saudi Arabia blocked a motion by other members to reduce production in response to falling prices.

The Saudis have instead boosted output, resulting in immense pressure on higher-cost non-OPEC producers.

Saudi Arabia seems to be taking a lesson from a 1986 price-fluctuation event, when massive, unprecedented production cuts in response to increased production by non-OPEC countries failed to stabilize oil prices.

Another factor keeping prices down is that non-OPEC producers have significantly reduced their costs. But this is likely a one-time event. In theory, as the chart below shows, the cost of producing oil is usually assumed to be constant and determined by immutable factors such as the type of oil and the geographical conditions where it is extracted.

In practice, however, the cost structure depends on a host of other factors, including technological improvements, human expertise, and so forth. So, in the case of shale-oil production, the industry appears to have enjoyed significant improvements to operational efficiency through old-fashioned trial and error, or "learning by doing.” Companies have quickly mastered the best methods as shale-oil production has matured in its investment cycle. But the cost of producing shale will now rise again, because such significant efficiency gains are not sustainable and the cost of capital is high.

That said, one defining feature of the "new oil market” brought about by the advent of shale oil is shorter, more limited oil-price cycles. Indeed, shale-oil production requires a lower level of sunk costs than conventional oil, and the lag between first investment and production is much shorter.

It is tempting to look at long-term oil futures – which increased gradually between 2000 and 2014, before falling abruptly after the November 2014 OPEC meeting – to predict where prices will go from here. However, futures have several limitations.

For starters, futures didn’t help predict the current market breakdown, most likely because long-term futures markets are, by default, slow to adjust to new information.

Moreover, futures contracts give only limited medium-term guidance because they either don’t extend out far enough, or the markets are not deep enough.

As in other commodity markets, oil futures are subject to an imbalance between long- and short-term positions.

For example, there is a higher demand among oil producers for short-term hedging than there is among manufacturers for long-term positions. Producers are typically willing to accept relatively lower prices to hedge their risk because they can’t pass cost increases on to their customers. Manufacturers, on the other hand, have more flexibility in this regard, because, even for energy-intensive manufacturing, oil is still only one small part of a company’s larger cost structure.

Some of the oil-price factors listed above are temporary, but others are structural and permanent. The debate about the causes and effects of the price slump since 2014 will continue, which both reflects and underscores a fundamental point: the conventional wisdom about the global oil market no longer applies.

Rabah Arezki is Chief of the Commodities Unit in the IMF Research Department.Copyright: Project Syndicate.