Applying physics to gdp forecasting

Models developed for gross domestic product (GDP) growth forecasting tend to be extremely complex, relying on a large number of variables and parameters. Such complexity is not always to the benefit of the accuracy of the forecast. Economic complexity constitutes a framework that builds on methods developed for the study of complex systems to construct approaches that are less demanding than standard macroeconomic ones in terms of data requirements, but whose accuracy remains to be systematically benchmarked. Here we develop a forecasting scheme that is shown to outperform the accuracy of the five-year forecast issued by the International Monetary Fund (IMF) by more than 25% on the available data. The model is based on effectively representing economic growth as a two-dimensional dynamical system, defined by GDP per capita and ‘fitness’, a variable computed using only publicly available product-level export data. We show that forecasting errors produced by the method are generally predictable and are also uncorrelated to IMF errors, suggesting that our method is extracting information that is complementary to standard approaches. We believe that our findings are of a very general nature and we plan to extend our validations on larger datasets in future works.

That is from A. Tacchella, D. Mazzilli, and L Pietronero in Nature.  Here is a Chris Lee story about the piece.  Via John Chamberlin.

Comments

I'll reserve my judgment until we can see how well it can forecast actual future outcomes; I'm not terribly impressed with how parsimoniously one can fit previous data if it has no predictive utility.

I didn't read the (gated) paper, but it's standard practice when building prediction models to use part of the data to build the model, and part of the data to test the predictive power. When they say they outperform IMF accuracy by 25% they presumably mean "when we test our model on data the model hasn't 'seen' before." Even if the data being predicted aren't really from the future, the data are new from the perspective of the prediction model, which is basically the same thing. Thus, the model does have predictive utility. Where your intuition is correct, is that the data generating process against which the model was built may not be constant for all time, so the predictive power of the model may still deteriorate over time.

it's all claims & fairy tales until they post their forecasts for 2019-2024.

I cross read the paper in 15minutes. Their model ain't that original, under the hood. Basically if you export raw stuff, their gdp forecast for you is very uncertain, basically the classical gdp forecast. If you export more refined stuff (stuff that only rich country export) then their gdp forecast for you is the classical gdp forecast shrunk to the mean of other countries that export refined stuff. Color me skeptic. Whoever reviewed this stuff either had an ax to grind or was not very interested in looking under the veneer.

"it's all claims & fairy tales until they post their forecasts for 2019-2024."

What statistical innovations have caused a sample size of five to produce reliable results?

I believe that kav is doubtful if the model will hold up for 5 years, let alone enough to prove high reliability. He's not saying that 5 is enough.

Boonton: if really the gap with the IMF model is as large as they claim, 5 years is a good start. Significance depends *also* on effect size. How many pills do you need to take to distinguish an unmarked box of rat poison from one of aspirin?

You are correct. They often use 80% of the data to predict the remaining 20% for in sample validation. The proof of the pudding is obviously to predict out of sample data.

Their claim is that the predictor is at least as good as others and provides information that the others do not provide. If that is the case, then wouldn't a better model combine the two approaches into one model? If the new model offers significant insights the improvements in predictive power should outweigh loss of parsimony.

From page 1 of the article:

"Most of the modelling efforts are in the direction of oversimplified representations that aim only at understanding the potential effects of a single variable, or of a limited set of them, in a controlled setting. Although these models may help grasp many isolated implications, they are rarely used directly to predict precise dynamics of complex systems such as, in the case of economics, countries’ growth or crisis. On the other side, the approaches explicitly developed for forecasting often depart from rigorous theoretical models, and are typically grounded on econometric or statistical techniques.From a physicist’s perspective, such a situation can be mapped to the more general problem of forecasting the evolution of a dynamical system of which we do not know the actual laws of motion."

Figure 3 from the article: China and Brazil have more or less the same GDP PPP per capita, but not the same "fitness" variable https://media.springernature.com/m685/springer-static/image/art%3A10.1038%2Fs41567-018-0204-y/MediaObjects/41567_2018_204_Fig3_HTML.jpg

My observation is that in any given state or condition of the economy, some variables are more predictive of future events than others. I might point out that, historically, a high level of inequality is predictive of a financial crisis, but does that suggest that forecasts of GDP growth at a time of a high level of inequality should rely primarily on that variable? Here's another variable that's especially significant for the U.S. economy: housing. In June, housing sales dropped. If July sales also dropped, I might be tempted to predict rough waters ahead for the economy, and that prediction may well be accurate. I believe it was Yogi Berra who said that predictions are hard, especially predictions about the future. And that's a good thing. Because if predictions were easy, they would be self-fulfilling, wouldn't they?

Here is a new research paper that confirms my beliefs about the importance of housing in our economy (growth, contraction, and, most importantly, slow growth following contraction): https://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.32.3.31 I credit Tim Taylor for the link: https://conversableeconomist.blogspot.com/2018/08/summer-2018-journal-of-economic.html

This is old news. Read "Housing IS the Business Cycle" by Ed Leamer. He makes a compelling case that production of durable goods (including esp. houses) explains business cycles. The idea is that long lived assets tie up resources in the short run and, because they are durable, any overhangs in supply cause unemployment of these resources long into the future. That is, if you build a bridge today you won't need resources to build that bridge for decades. The initial building draws resources in and then cuts them off upon completion. If you have a building boom of bridges (or houses) it displaces demand for a long time.

Workers and owners of capital could conceivably work save during the boom and smooth consumption over the bust, but they don't. Rational expectations takes a serious hit.

I'm not an economist but isn't there a notion of forecasts being better when one averages the results of multiple simple models rather than trying to tweak one complex model?

Not really, since "averaging many simple models" can itself be considered "one complex model".

Unfortunately this is correct in semantics, but incorrect mathematically.

Yes, averaging many models can be thought of as a single complex model in terms of aggregation of parameters.

In practice however, the effect of averaging simple (independent, > 50% accuracy) models mathematically different from fitting one complex model.

The simple models are fitted independently, unlikely the complex model where all the parameters are fitted at the same time.

Yes, they are called ensemble models.

The reason they work is due to bias-variance tradeoff (you can wiki it), or read this:
http://scott.fortmann-roe.com/docs/BiasVariance.html

Complex models (or more accurately, models with a high number of parameters) tend to overfit and generalize poorly. Averaging a bunch of simple models (that are independent, each with at least > 50% accuracy) tends to produce less-wrong forecasts over time (lower variance) compared to a single complex model.

This is alluded to in Makridakis et al's article in MIT Sloan.
https://sloanreview.mit.edu/article/why-forecasts-fail-what-to-do-instead/

"This case is exemplified by China in 1995. China at that time had a low GDP but high economic fitness. As predicted by the model, China experienced 20 years of steep economic growth, with its GDP increasing remarkably. In a standard economic analysis, this seems extraordinary. But, the researchers argue that this is actually expected behavior: much like a stretched spring being released to jump back to its position."

Well ok, but how did China's "spring" become stretched in the first place? Can you take the model back to 1985 and have it accurately predict 1995? Granted, the horribleness of Communism is an explanation, but how does the model encapsulate the change in that explanation with any predictive capability?

The article is in Nature Physics, which is a fully distinct journal from Nature. (It is not, for instance, the physics section of Nature.)

How much does this differ from an ARMAX model? Sounds similar.

I'm sorry, how is this at all an improvement upon the work of people like Cesar Hidalgo... Isn't this old hat? At least they got their single citation!

https://atlas.media.mit.edu/en/resources/about/

As someone involved in both econophysics and complexity economics, I guess I should comment, now that such are again allowed here.

So this is an offshoot of the long-running Hausmann project on defining complexity by how complicated in terms of the numbers of products an economy exports is. In standard complexity theory this is mere "complicatedness" rather than "complexity," which involves more than just a lot of sectors in the economy. However, that point does not mean that some measure of "fitness" based on "competitiveness" may not be useful in forecasting future GDP growth. That is an empirical question, and I have not dug into this paper enough to pass judgment on that claim.

It's not really a simplified model if "fitness" is some complex aggregate which is itself multi-dimensional; that simply hides a multivariate model within a single summary variable.

I fear models such as this will motivate some form of globalist neo-mercantilism; a continuous ratchet of abandoning sovereignty and market barriers in order to capture a greater share of the zero sum of export global market share, as the story of growth is simplified to a notion that "export competitiveness" is the magical "golden goose".

The economy is not just a complex system, it's a complex ADAPTIVE system. CAS's are notoriously hard to predict, for good reason. And the traditional method of building a model with half the data, then validating it with the other sequestered half does not work here because the economy you built the model for is not the economy of the present or the future. So at best, one can say of the model, "This is a model of how the economy responded to inputs during the years for which we had data." Whether that model is completely valid, partially valid, or completely invalid when modeling the future economy is an unknown, because the future is unknown.

Also, the economy as a CAS shares the features of other complex systems that make them impossible to predict- nonlinear responses to input, extreme sensitivity to initial conditions, etc. A good thought experiment is to ask yourself - if we could have a 'do-over' and start with the exact same economy we had in 1980, and supply the same inputs, would we get the same result? If the answer is no, then of what use is the model of how the economy did behave if it was only one of many possible ways it could have?

A couple of examples:

An anthill is a complex adaptive system. Let's say you start an anthill equidistant from two major food sources. Can you predict the future shape and direction of the colony? Two ants are foraging for food. One heads for one food source, the other heads for the other one. But one ant has a random leaf blown in front of it and has to detour, so the other ant gets to his food source first, and starts a pheromone trail, which attracts other ants. A feedback loop starts, and the entire colony moves in the direction of that food source. A very tiny change in initial conditions causes a massive change in the life cycle of the colony.

Let's say you develop a model for the economy from 1900 to 1939. The model seems to accurately predict the sequestered data. Is it therefore a good guide to what's going to happen to the economy in 1940?

Obviously, that model blows apart in a few months when a world war starts. But less obviously, what if Orville Wright had a random encounter with an individual who delayed him for a second, and therefore when he stepped into the road he was run over by a horse. Therefore, the Wrights do not invent the airplane first. Maybe Louis Bleriot wins instead. This causes a major swelling of pride in French aeronautics, incentivizing more people in France to study the field. This causes France to become the undisputed leader in aviation, which changes everything - markets, patterns of immigration, engineering start-ups, wars, etc. That one-second conversation of Orville Wright's just blew apart your finely constructed model.

If this was a rare occurrence, perhaps the model still has value. But if the future of the system is dominated by 'unknown unknowns', a model based on the knowns or even known unknowns is going to be of very limited value.

And the future is a giant sea of unknown unknowns.

It is very interesting to read the answers :)
Honestly, I hardly imagine a formula that takes into account ALL factors, but not 1, not 2, and not even 10. Moreover, even here it is necessary to take into account the human factor, although most people do not even understand why. What is now called GDP forecasting - I think this is a very simplified formula, designed for ideal conditions, which certainly can not be, always something can go wrong. And considering also foreign policy factors ....
This is a task that is much, MUCH harder than described in this post.

Comments for this post are closed