Category: Uncategorized

I have had fringe contact with more epidemiology than usual as of late, for obvious reasons, and I do understand this is only one corner of the discipline.  I don’t mean this as a complaint dump, because most of economics suffers from similar problems, but here are a few limitations I see in the mainline epidemiological models put before us:

1. They do not sufficiently grasp that long-run elasticities of adjustment are more powerful than short-run elasticites.  In the short run you socially distance, but in the long run you learn which methods of social distance protect you the most.  Or you move from doing “half home delivery of food” to “full home delivery of food” once you get that extra credit card or learn the best sites.  In this regard the epidemiological models end up being too pessimistic, and it seems that “the natural disaster economist complaints about the epidemiologists” (yes there is such a thing) are largely correct on this count.  On this question economic models really do better, though not the models of everybody.

2. They do not sufficiently incorporate public choice considerations.  An epidemic path, for instance, may be politically infeasible, which leads to adjustments along the way, and very often those adjustments are stupid policy moves from impatient politicians.  This is not built into the models I am seeing, nor are such factors built into most economic macro models, even though there is a large independent branch of public choice research.  It is hard to integrate.  Still, it means that epidemiological models will be too optimistic, rather than too pessimistic as in #1.  Epidemiologists might protest that it is not the purpose of their science or models to incorporate politics, but these factors are relevant for prediction, and if you try to wash your hands of them (no pun intended) you will be wrong a lot.

3. The Lucas critique, namely that agents within a model, knowing the model, will change how the model itself operates.  Epidemiologists seem super-aware of this, much more than Keynesian macroeconomists are these days, though it seems to be more of a “I told you that you should listen to us” embodiment than trying to find an actual closed-loop solution for the model as a whole.  That is really hard, either in macroeconomics or epidemiology.  Still, on the predictive front without a good instantiation of the Lucas critique again a lot will go askew, as indeed it does in economics.

The epidemiological models also do not seem to incorporate Sam Peltzman-like risk offset effects.  If you tell everyone to wear a mask, great!  But people will feel safer as a result, and end up going out more.  Some of the initial safety gains are given back through the subsequent behavioral adjustment.  Epidemiologists might claim these factors already are incorporated in the variables they are measuring, but they are not constant across all possible methods of safety improvement.  Ideally you may wish to make people safer in a not entirely transparent manner, so that they do not respond with greater recklessness.  I have not yet seen a Straussian dimension in the models, though you might argue many epidemiologists are “naive Straussian” in their public rhetoric, saying what is good for us rather than telling the whole truth.  The Straussian economists are slightly subtler.

4. Selection bias from the failures coming first.  The early models were calibrated from Wuhan data, because what else could they do?  Then came northern Italy, which was also a mess.  It is the messes which are visible first, at least on average.  So some of the models may have been too pessimistic at first.  These days we have Germany, Australia, and a bunch of southern states that haven’t quite “blown up” as quickly as they should have.  If the early models had access to all of that data, presumably they would be more predictive of the entire situation today.  But it is no accident that the failures will be more visible early on.

And note that right now some of the very worst countries (Mexico, Brazil, possibly India?) are not far enough along on the data side to yield useful inputs into the models.  So currently those models might be picking up too many semi-positive data points and not enough from the “train wrecks,” and thus they are too optimistic.

On this list, I think my #1 comes closest to being an actual criticism, the other points are more like observations about doing science in a messy, imperfect world.  In any case, when epidemiological models are brandished, keep these limitations in mind.  But the more important point may be for when critics of epidemiological models raise the limitations of those models.  Very often the cited criticisms are chosen selectively, to support some particular agenda, when in fact the biases in the epidemiological models could run in either an optimistic or pessimistic direction.

Which is how it should be.

Now, to close, I have a few rude questions that nobody else seems willing to ask, and I genuinely do not know the answers to these:

a. As a class of scientists, how much are epidemiologists paid?  Is good or bad news better for their salaries?

b. How smart are they?  What are their average GRE scores?

c. Are they hired into thick, liquid academic and institutional markets?  And how meritocratic are those markets?

d. What is their overall track record on predictions, whether before or during this crisis?

e. On average, what is the political orientation of epidemiologists?  And compared to other academics?  Which social welfare function do they use when they make non-trivial recommendations?

f. We know, from economics, that if you are a French economist, being a Frenchman predicts your political views better than does being an economist (there is an old MR post on this somewhere).  Is there a comparable phenomenon in epidemiology?

g. How well do they understand how to model uncertainty of forecasts, relative to say what a top econometrician would know?

h. Are there “zombie epidemiologists” in the manner that Paul Krugman charges there are “zombie economists”?  If so, what do you have to do to earn that designation?  And are the zombies sometimes right, or right on some issues?  How meta-rational are those who allege zombie-ism?

i. How many of them have studied Philip Tetlock’s work on forecasting?

Just to be clear, as MR readers will know, I have not been criticizing the mainstream epidemiological recommendations of lockdowns.  But still those seem to be questions worth asking.