As far as I know, there has never been a rigorous
ex post evaluation of CGE [computable general equilibrium] models in practice, one that compares
predicted to actual outcomes.
Here is the broader argument, hat tip to Mark Thoma (there is more at the link), and comments are open so can you prove him wrong?
1. The Australian Treasury was given three months to use CGE models to generate estimates of the economic impact of the government’s proposed cap and trade carbon sytem. They have asked for another three months, it seems because the scenarios being modelled are so far outside previous experience that their models are blowing up. If the models have not had ex post evaluation, and as well are operating outside the data set, what are the results worth?
2. And does the same apply to climate models?
This may sound flip, but I thought that only people who were estimating CGE models actually believed in them. The reason why no one evaluates their predictions is that the models make horrible predictions.
Since demand curves are incalculable, how could a general equilibrium be?
An evaluation of the performance of applied general equilibrium models of the impact of NAFTA
Timothy J. Kehoe http://minneapolisfed.org/research/sr/sr320.pdf
Abstract
This paper evaluates the performances of three of the most prominent multisectoral static applied general equilibrium models used to predict the impact of the North American Free Trade Agreement. These models drastically underestimated the impact of NAFTA on North American trade. Furthermore, the models failed to capture much of the relative impacts on different sectors. Ex-post performance evaluations of applied GE models are essential if policymakers are to have confidence in the results produced by these models. Such valuations also help make applied GE analysis a scientific discipline in which there are well-defined puzzles with clear successes and failures for competing theories. Analyzing sectoral trade data indicates the need for a new theoretical mechanism that generates large increases in trade in product categories with little or no previous trade. To capture changes in macroeconomic aggregates, the models need to be able to capture changes in productivity.
John asked:
The difference between climate models and CGE models is in the nature of uncertainty. In CGE models the largest form of uncertainty is in functional relationships between variables (see page 15 of the Kehoe paper mentioned earlier). Climate models follow the laws of thermodynamics, unless you are a flat earther of some kind there’s no reason to doubt the functional relationships. The major uncertainty in climate models is in the input parameters, especially aerosol effects on albedo and rainfall, which is almost the opposite problem from economics where inputs are known very precisely, with the exception of black market activities, but relationships between terms are not.
UCSD hosts a group that maintains an ongoing evaluation of climate models versus predictions.
Comments on this entry are closed.