A widely followed model for projecting Covid-19 deaths in the U.S. is producing results that have been bouncing up and down like an unpredictable fever, and now epidemiologists are criticizing it as flawed and misleading for both the public and policy makers. In particular, they warn against relying on it as the basis for government decision-making, including on “re-opening America.”

“It’s not a model that most of us in the infectious disease epidemiology field think is well suited” to projecting Covid-19 deaths, epidemiologist Marc Lipsitch of the Harvard T.H. Chan School of Public Health told reporters this week, referring to projections by the Institute for Health Metrics and Evaluation at the University of Washington.

Others experts, including some colleagues of the model-makers, are even harsher. “That the IHME model keeps changing is evidence of its lack of reliability as a predictive tool,” said epidemiologist Ruth Etzioni of the Fred Hutchinson Cancer Center, home to several of the researchers who created the model, and who has served on a search committee for IHME. “That it is being used for policy decisions and its results interpreted wrongly is a travesty unfolding before our eyes.”

…The chief reason the IHME projections worry some experts, Etzioni said, is that “the fact that they overshot” — initially projecting up to 240,000 U.S. deaths, compared with fewer than 70,000 now — “will be used to suggest that the government response prevented an even greater catastrophe, when in fact the predictions were shaky in the first place.”

Here is the full story, from StatNews, by Sharon Begley with assistance from Helen Branswell, two very good and knowledgeable sources.  Via Matt Yglesias.

To be clear, I am (and always have been) fully aware that there are more nuanced epidemiological models “sitting on on the shelf,” just as is true for macroeconomics and many other areas.  But I ask you, where are the numerous cases of leading epidemiologists screaming bloody murder to the press, or on their blogs, or in any other manner, that the most commonly used model for this all-important policy analysis is deeply wrong and in some regards close to a fraud?  Yes I know you can point to a few tweets from the more serious people, but where has the profession as a whole been?  Who organized the protest letter and petition to The Wall Street Journal?

And to be clear, I have heard this model cited and discussed in many (off the record) policy discussions, this is not just something you can pin on the Trump administration narrowly construed (though they are at fault as well).


They were busy attending a conference on modeling climate change.

Good thing THOSE models don't "keep changing" or we might have "evidence of their lack of reliability as a predictive tool".

Might even call them a fraud.

But as Tyler would say, completely bogus climate models are not just something you can pin on the Trump administration (though they are at fault too).

Climate has been around far longer than COVID. Our data goes back hundreds of thousands of years with ice cores.

COVID has been around a couple months.

In 1896 Svante Arrhenius calculated the effect of a doubling atmospheric carbon dioxide to be an increase in surface temperatures of 5–6 degrees Celsius. Models have undergone continuous improvement since.

COVID models are a few months old.

So I ask you, is extrapolation from COVID to climate really the smart way to go?

Many "experts" reject the models and pronouncements of many other "experts". This is normal and not indictive of actual failure of a particular model or belief about this virus. Time will tell. One thing for sure; the attempt at reducing the spread through a shutdown/quarantine worked AND in working made all the models wrong. So do not forget this uncomfortable fact one someone demands we throw out all previous prognostications and adopt their new and better guesses. Once the shutdown/quarantine is ended there will be an uptick in cases and deaths. And when that happens the new models will be wrong.

But models included estimates based on social distancing measures, so that's no excuse.

That is a mistaken assumption. No one anticipated such a widespread and effective shutdown. This was unprecedented and could not have been factored into the model.

Saul - have you looked at the IHME output? It actually shows its assumptions on a few metrics that are various flavors of "shutdown" (non-essential services closed, mass gathering restrictions, schools closed, etc.) - https://covid19.healthdata.org/united-states-of-america/arkansas

If it's not correct in how it incorporates these assumptions, then it is indeed a bad model.

> is extrapolation from COVID to climate really the smart way to go?

As models get more complicated, the biases of the creator creep in. If model builders were bias-free, then half the models would overestimate, and half would underestimate over decade-long periods. But we have all 102 models overestimating warming (compared to satellite measurements) by a lot.


Virus models are the same. You tweak the knobs until it matches history (backcasting) and then you hope those knows correctly forecast. But there are an infinite number of solutions, and so one settings of knobs can match history perfectly and dramatically overstate the future, and another setting can match history perfectly and dramatically understate the future.

In the link I sent, look how good all the models did at catching major blips in history (1993, 1985). So you can be sure the models were "tuned" to respond to those historic events. But they all utterly failed to catch blips in the future. Meaning their tuning wasn't really tuning. It was just curve fitting.

I could do the exact same on stock market predictions (and I have--it was a very good learning experience). You build models and trading strategies and then set those strategies loose on 30 years of historic data and tweak the strategies to work really well so that over any 4 week period you are outperfoming the market by ~2X. And then unleash those exact strategies on live market data...and watch yourself get creamed in the market. Not quite the same as climate, because one you are attempting to model a physical system and the other you are, well, attempting to model a physical system. Remember, when you don't understand a system, the actors (human or otherwise) often appear as random operators. They aren't random, but they appear that way.

Well if I were to generalize, what if I said "people keep dying of COVID, and darned if the planet doesn't keep warming, and yet some people shout that we can't trust the models!"

It seems quite the "who are you going to believe, me or your lying eyes" moment.

In other words, let's remember that the models are secondary to the observed effect.

Yes, but as we learn today from the Stanford study, just 0.05% of those under 60 that catch this will die from this. That says get back to work and get back to school unless you have vulnerable people at home.

I agree that if this study holds up it will change policy discussion/options.

It won't be a model at that point, it will be an observed result.

BTW though, let's run those numbers assuming 100% reach. In 2016 the over-sixty population was 69 million (pdf warning). Sticking with 2016 for convenience, the total population was 323 million.

(323 - 69) * 0.05 = 12M?

Doh! Factor of 100. 120,000

Isn't this the formula you want? 323m - 69m = 254m under 60. They have about a 50% chance of catching the disease. If they catch it the have a .05% chance of dying. (.0005)
254,000,000 * .50 = 127,000,000 under 60 would catch it. 127,000,000 * .0005 = 63,560 of those would die. Then you would have to all the people over 60 who would die.

That's fine too. And yes I should have noted that was just the under 60 number.

You are both assuming that eventually 100% of the population gets it. If that were the case, then we'd not see burnout that we're seeing. It would keep ripping through the populations, especially places like Sweden. But it's very likely that a very high % of the population cannot catch this for whatever reason. That was true of the plague too. That was true of H1N1. Old people didn't catch H1N1, because it was believed that the Hong Kong flu from many decades prior had effectively immunized them.

Dr. Roy Spenser? Really? Next, let's let him model creationism for us.

The data speaks for itself. You have the models (which is easy to verify), and you have the measurements (which is easy to verify).

You might not like when they are shown side by side. But you must admit, those that made the climate models--99% overestimated by a lot.

And you must admit, if the models were bias free and of high confidence, then half would have overshot, and half would have undershot.

So, what precisely do you disagree with in the graph?

I think that Phinton writes "numerator" but means "denominator".

But I agree with the basic point. And it's been a hypothesis for some time: that this virus is more than "nothing", but that it's truly a problem because it combines flu-like mortality with much greater than flu-like rate of spread.

If correct, this calls for a significant relaxation of "shutdown", especially in areas where spread isn't likely to approach NYC or Lombardy levels (people don't travel by subway, social norms aren't kissing on the cheeks, etc.).

Perhaps no large gatherings, and companies (and workers) may decide that those who can easily work from home do so. Obviously focus on keeping this virus out of nursing homes and hospitals. But don't shut down society. People can go on with life, while wearing masks and not shaking hands.

INTERESTING: Antibody research indicates coronavirus may be far more widespread than known: Of 3,300 people in California county up to 4% found to have been infected. “The first large-scale community test of 3,300 people in Santa Clara County found that 2.5 to 4.2% of those tested were positive for antibodies — a number suggesting a far higher past infection rate than the official count. Based on the initial data, researchers estimate that the range of people who may have had the virus to be between 48,000 and 81,000 in the county of 2 million — as opposed to the approximately 1,000 in the county’s official tally at the time the samples were taken.”

This would be great news. It would mean that the infection fatality rate is 48 to 81 times smaller than the number you get by dividing deaths by positive tests.

Does simply having antibodies indicate you were infected in the past or simply exposed to the virus? Are they always the same thing?

I think infected, but we shall see. This should be very important news.

I'm asking because I'm not sure anyone has ever studied what it takes to be infected? Is it literally a single virus that can do the job or must you take in some critical number of viruses?

Let's say it takes a billion viruses to be infected. You may defeat the infection and never show symptoms so you don't even know it. Antibody test says positive.

Let's say less than a billion you won't get infected but while antibodies might be triggered, not in enough numbers to actually defeat the virus should you get a dose of a billion.

I'm also wondering if you get a mega-dose of virus....say you're working without masks in a hospital filled with it so day in day out you take in trillions of viruses might that overwhelm whatever antibodies you have.

I'm wondering if this is a spectrum of 'immunity' here.

Here’s an abstract to the study:


The MAJOR issue with this study is this: there are large economic and social incentives to being diagnosed as having the antibodies. People who suspect they have had the virus may be much more likely to respond to a facebook ad offering free antibody testing.
This can potentially bias the estimated share of infected upwards by a large amount.
The authors actually acknowledge this bias in passing in the paper.

From the abstract -“ Participants were recruited using Facebook ads targeting a representative sample of the county by demographic and geographic characteristics. We report the prevalence of antibodies to SARS-CoV-2 in a sample of 3,330 people, adjusting for zip code, sex, and race/ethnicity. We also adjust for test performance characteristics using 3 different estimates: (i) the test manufacturer's data, (ii) a sample of 37 positive and 30 negative controls tested at Stanford, and (iii) a combination of both.”

Where is the bias? Did they offer them big cash payments?

No cash payments, even better: knowing that you have antibodies is potentially very valuable: you can safely go back outside, work, etc.. or at the very least reduce precautions and fear.

People who suspected they had the virus (because of mild symptoms) for example might therefore be much more likely to participate.

The authors acknowledge this themselves in the text: "bias favoring those with prior COVID-like illnesses seeking antibody confirmation are
also possible. The overall effect of such biases is hard to ascertain."

Nice sleight of hand. They considered that the sample could be biased due to individuals having access to a car and Facebook, being healthy enough to get the test center and wanting to know if they ever were infected and you want to invalidate the study. Ha.

Not sure I understand your comment.

I don't want to invalidate their study, just point out to a potentially important bias (which the authors themselves recognize).

Offering free tests on facebook will get you a far from random sample.

Given the trillion $ implications of the findings, it is beyond me why we don't have the money and resources to do a truly random sample (I don't know , worse case govt comes in and "forces" 3k randomly selected people to give their saliva. If you can force 300 million to stay home, you can surely do that).

At the very least make it voluntary but don't promise test results.

I’m not a medical professional, but I’ve seen the term “viral load” and my understanding of it’s meaning is that the more you get, the more cells are infected on day one, the harder your immune systems has to work to catch up before major damage is done. So it would make sense to me that on the continuum of doses, you would go from no infection, to no symptoms, etc.

"I'm asking because I'm not sure anyone has ever studied what it takes to be infected?"

Wondered about this myself. In theory you can be infected by 1 virus, turns out.


(however is not corona but still)

69 death in Santa Clara County.

What is the false positive rate? Or, how many people would you expect to test positive if no one was affected? Many of these tests have high error rates

I spend so much time in SJC airport one can only hope I've picked it up along the way. I'd pay $500 for an anti-body test at this point to see if I'm "invincible" or not.

I hope you stay healthy - but 0 I guarantee you that you are not invincible.
I just saved you 500 bucks.

Maybe the model was not "wildly wrong", maybe the model had built in to it some of the best guesses about the future R naught and the future lethality, not of the exact population of coronaviri hitching rides in our respiratory systems on any given day the model was made, but about the potential future populations hitching future rides.

The virus kingdom is not really a kingdom, the way the animal kingdom or the plant kingdom is a kingdom, but it is populated by creatures who are, on current assessments, way smarter than the best projected AIs for 2025 or 2030, when it comes to defeating their adversaries.
I.e., us.
Think about it.

Like I said, you are not invincible to these creatures who, like I said, have managed to hitch rides in respiratory systems and elsewhere where they were usually not wanted for a long long time ----not for centuries (and think about it, current medical science with respect to illnesses (not midwifery, not nutrition, not chiropractic or osteopathy, but with respect to illnesses) really only passed the first derivative of being less harmful than helpful about 100 years ago --- but for aeons and aeons.

There's a flip side of that story. Sure "between 48,000 and 81,000" is large compared to "approximately 1,000 in the county’s official tally," but "between 48,000 and 81,000" is very small compared to a "county of 2 million."

If you let it rip, you easily get 500,000 cases, and 10x impact on hospitals.

Or to put it another way, you get scenarios we are already familiar with: Italy, NYC.

No need to jump to “let it rip.”

It’s extremely good news that the IFR is continually being adjusted downward as more data comes in.

What is the leadership from the White House this morning?

Is it for continued lockdown? Is it for an extensive system of test and trace?

Or is it for some weird demand for "liberty" to ignore such concerns?


Not sure how that’s a response to the latest downward revisions of the IFR, but....I applaud your attempt at derailment I guess.

Lower IFR is good news for everyone.

Democracy. It's not just a right, it's a responsibility.

Yes that's a completely logical, totally not a non-sequitur comment in a discussion of revisions to the IFR.

You nailed it

I think you are playing a tin ear as to how IFR does relate to national policy.

There’s a posted plan to reopen in phases. If you want to discuss it then address it directly.

No need to derail

That's a different kind of tin ear.

After all, it wasn't me who ignored the plan and shouted "LIBERATE MICHIGAN!" from the bully pulpit.

So you have nothing to say about recent downward revisions of IFR nor national policy.

You're here to complain about Twitter.

Should have just said that in the beginning on a top level thread

You have one weird kink. Now. I notice that Tyler wrote this up above:

And to be clear, I have heard this model cited and discussed in many (off the record) policy discussions, this is not just something you can pin on the Trump administration narrowly construed (though they are at fault as well).

Tyler gets the connection between modeling generally, IFR, and policy.

So I ask you, why play the tin ear, and pretend that this is some kind of "derailment?"

It is exactly Tyler's topic.

But you have nothing to say about modeling, IFR, nor national policy. Scroll back up and read your comments. So far your comments are links to tweets, links to articles about tweets, and a nonsensical virtue signaling comment about democracy.

There's a national policy that's been posted. You claim you want to discuss national policy; revealed preference says the opposite

I can see what happened. I tied this study to "let it rip" policy. You said we didn't have to. I pointed out what was in play.


Other people get it, but you want to fly cover for policy disruption.

There is no serious "let it rip" policy, because private decisions of individuals and companies wouldn't have it happen even if governments agreed to it (and governments wouldn't).

"Anonymous" is tilting at windmills.

He said that it's great that it is being downwardly revised, and he said the National Policy is lacking from the executive branch in terms of clear leadership. It's not at all difficult to discern that from his comments and I'm not sure why you are playing confused yet exude such utter genius otherwise.

If low IFRs and low hospitalization rates are true, they mean that "crush the curve" and "flatten the curve" actually separate.

If you're trying to just "flatten the curve" (same number of deaths and hospitalizations, longer time), then at low IFRs and hospitalization rates, it's entirely practical to just up capacity and get it done much quicker.
This is the "mitigation", not stressing healthcare argument, coming back to haunt us.

Some (of more decent memory than others) may remember how this was dismissed at the beginning of all this as just not worth the effort compared to "suppression" and mitigation and suppression became conflated, given the models of the time high IFRs and low healthcare capacity to supposedly deal with even 15% of the population infected. (This is the Toma Pueyo article!).

However, if those IFRs and hospitalization rates are really wrong, and if you have no "exit strategy" and you're really just doing "mitigation", and avoiding healthcare system stress, you may have to rethink what the least expensive, least debt incurring way to do that is. "Flattening the curve a bit", but not "crushing the curve" may come back.

I wonder what the false-positive rate for that test is. Hope it's not 4%

This is an important point. It is quite possible the false positive rates are dwarfing the actual infection rate.



This is huge, because for the first time we get a glimpse of the numerator, and we can finally know that the deaths from this infection are 0.5%. But that is for all ages.

In Seattle, just 8% of deaths are people aged 0 to 60. This means for working age populations, the death rate is 0.05%.

It is nothing.

This Stanford study says to everyone: Get back to work and/or school unless you have older people (>70) at home.

Let's imagine for a second those numbers are right: 0.05% is still 1 in 2000. Let's say that we had this death rate in, say, attending a game at the Dallas Cowboy's AT&T stadium. That's only 40 people that would die from attending. It's nothing!

Do you think they'd be able to fill the stadium with that death rate? It's not so low.

> Do you think they'd be able to fill the stadium with that death rate? It's not so low.

When an airplane crashes and you are on board, you have a 50% chance of dying. Your response: "Oh yeah! Can you imagine if 50% of people died from going to a movie!?"

You are comparing a weekly event to a once in a decade or once in a hundred year event.

H1N1 had a comparable death rate (0.05%) AND it disproportionately impacted young people. How many words do you spend worried about H1N1 in the various forums a decade ago? Zero? that's what I thought.

In terms of death rates, we're now in very familiar territory given the Stanford study.

You didn't answer his question... do you think they would fill the stadium?

I'll ask you one further, would you go if they sold tickets?

Would I take that bet routinely? No. Would I take it versus losing my job once in a lifetime? In a heartbeat.

People routinely participate in behaviors that are far more risky all the time. Yes, I think people would take those odds readily if it meant they could go back to the way things were (go to job everyday, get a steady check, drink with friends on weekends, go out to eat, go to a movie...)

Would you make that once-in-a-lifetime bet to maintain a quality of life you loved? Of course you would. You already made that bet if you kept working during H1N1.

Yes, easily, and people will pay crazy prices for beer and pretzels too. You have an a widely exaggerated view of people's risk aversion. People pay 6-digit sums to climb Mt. Everest.

Furthermore, try augmenting your argument to be the same setup you suggested, but also add "if you don't go, you have a 20% chance of losing your job".

500 per million is a little high for a death rate. I would be more comfortable if it was closer to 100 per million ( which btw I think might be closer to the true number than the 500).
As a reference in the US , annual automobile accident deaths are 124 per million.

Maybe we shouldn't be so inured about those automobile deaths either.

I think you could cut it in California if you just gave the Highway Patrol (a variety of) unmarked cars. They have been making their light bars a bit stealthier, which has to help.

Bob - bad argument, to the point where I wonder if you are overly negative on purpose.

(1) There's a large jump from "do what most of the U.S. is doing now" to "80,000 people at a football game". We could loosen restrictions *considerably* and still not have the latter.

(2) Your math assumes that everyone in the stadium contracts this viral disease, which isn't what would happen.

And: where do all those N95 masks reside? (I will be pleased to trust a 3M model, manufactured in the good ol' US of A, whenever I should see one.)

I believe they are currently growing in the British Columbian forests. We should probably move it to Texas.

"...the Harmac mill is the world’s only producer of the particular grade of paper pulp used in the manufacture of surgical masks and gowns..."


Surgical masks are not N95 masks

You are correct, but that doesn't mean N95 mask production does not have serious bottlenecks that can only be opened with time. Money is not the object.

Close to a fraud? Someone is really starting to get worried about something.

Luckily for us, Trump was not using the IMHE results when making his decisions regarding a 180° turn in his administration's approach to a pandemic, but instead the Imperial College team’s estimate of likely coronavirus deaths. As was noted by Tyler Cowen March 18, 2020 at 1:35 am. And there is no way that anyone not afflicted with TDS could even suggest pinning anything on Trump by using the same information that also informed UK policy makers changing course too.

Probably too early in the spring to pick cherries, but maybe there are still some nice flowers to see.

This must be a prior post. Obviously passive aggressive, but obscure enough to be nearly indecipherable.

“will be used to suggest that the government response prevented an even greater catastrophe..."

Maybe Trump will take credit but model or no model the same people were already going to either praise him or disparage him anyways.

I believe the original overshot model will be used to suggest that the government purposefully exaggerated the threat in order to accomplish what they wanted.

I believe the current model intentionally undershoots to politically damage Trump by November.

70,000 reported dead in the whole country? Hell, New York alone can report that many "And what are you gonna do about it? Nothin. That's what."

Dr. Osterholm discussed the models in his most recent or next to last podcast. He's generally skeptical of all models, but if you assumed the spread of an influenza virus reaching 50% of the population you should ask, as a decision maker, what would be the mortality rate for that spread, as to whether you should act and how fast and how much. (At least that's what I recall from the podcast, so there could be some correction here, but I don't want to go back and transcribe and re listen.

Or, you could look at the country[s that started off with the herd mortality, sorry, herd immunity response, and whether they stuck to it as deaths mounted and hospitals were overrun.

How's Boris Johnson doing by the way?

Anyone planning a trip to Italy?

It's just getting started there, so how's Brazil doing?

You can listen to the Osterholm podcasts and medical podcasts here: https://www.cidrap.umn.edu/covid-19/podcasts-webinars

Only a theoretical economist would say,
As their house was burning down,
That the house did not burn or collapse
As the way
The Model would predict.

Whereas others would say,
Get me a firehose,
Fck the model you idiot.
My house is burning!

Don't get me wrong, though, I think models are important, just as are fire detectors.

+1. Economists have been amazingly bad here. They model they seem to operate by is "see how bad the car crash is then decide whether to put your seat-belt on".

When did Tyler or any group of economists argue for “crashing the car”, as it were? They’ve been defending the policies put in place but asking for accurate models.

Turing Test: Failed

Hey look if you're heading towards a brick wall at 50 mph, it's very hard to model what that will do to your car. You need to consider the brick, the motor, how strong the wall is, the angle your car hits it, the dynamics of the impact on your frame.

I'm sure crash engineers have models like that. But I'm also sure the models give all types of diverse answers depending on how you tweak just one or two of those many elements. Such models are useful in their own context.

That context, though, has nothing to do with the policies for what to do when driving towards a brick wall at 50 mph. At that point the policy is slam on the breaks, turn away from the wall, do whatever the F you can to avoid that crash or lessen the impact as you can.

Afterwards spend all the time and money you want refining the models and adding more and more variables to them. That will have no impact on what to do next time it happens.

What about if you don't know whether the wall is made of brick or plywood, but you know that slamming the breaks means 1 in 5 people nationwide lose their jobs. Also, most of those have no savings to speak of at the tail end of a great economic boom - https://www.cnbc.com/2019/01/23/most-americans-dont-have-the-savings-to-cover-a-1000-emergency.html

With the kind of confidence you have, you must either go through the trolley problem and its variants in 30 seconds flat, or be unable to make a decision at all?

So to put this in context of running into a brick wall suppose you're running late for work, the wall is in your way and you really need the job. Should you try to run thru the brick wall because maybe it's just a movie prop made out of Styrofoam? Or maybe I need to get a job that will cut me some slack and won't fire me for being five minutes late so that I'm running my car thru brick walls.

What you are saying is we should not shutdown the economy because our economy cannot handle a shutdown. Is there a law of biology that says viruses will never be so bad as to ask too much from our fragile economy? I don't see why this is an argument against shutdown unless we just have to throw up our hands and say we are unable to deal with any possible pandemics.

Five minutes is one thing, five months would be something else.

And, no, there is no such law. There can be such viruses. I claim that this is not one of them. We can deal with it without a shutdown, or could have, because now the shutdown and the fallout from it are fait accompli.

Unless I misunderstand you, your argument is "There can be truly terrible pandemics, which must be eradicated at ANY (emphasis!) cost to the economy. COVID-19 is one such. Therefore...." I agree with the first premise of the syllogism, and not with the second one.

@Bill, haiku: a study once found due to false positives, smoke detectors are useless. In Greece they don't build with wood, so aside from smokers in bed, detectors are useless. But I agree with your implicit point, that R0 changes with behavior, and if say the USA did nothing with C-19, and say 60% of the population was infected, C-19 would have burned out by now but we'd have had about 5 m more deaths now. I think most Americans could not stomach that. Third Worlders would be different IMO.

5 million is a crazy number, Ray.

BTW, does anyone here truly believe that Ray actually is who/what he claims to be?

Alex is doing better, though he has learned his lesson about treating coronavirus as just like the flu, and continues to recuperate, without taking on any of his government duties yet (he remains out of the news and not publicly appearing, at least). His pregnant wife, also infected, is doing well too.

'On April 10, his father, Stanley Johnson, said the Prime Minister "almost took one for the team" and that he will need a period of rest as he begins to recover from Covid-19.

He downplayed suggestions the Prime Minister will quickly return to work at Number 10, saying he "has to take time".

A spokesperson for the PM also said: “The Prime Minister has been able to do short walks, between periods of rest, as part of the care he is receiving to aid his recovery.”

Now I know this is just mortality fear. Tyler would profit from interrogating his emotional responses as fiercely as he does intellectual ones.

Every model should IMHO be compared with the "Caveman Model":
1. Plot Active cases on linear (not log-log) scale until you can identify the initial point P of inflection. (Might be hard as the curve may ascend diagonally for awhile; use the midpoint of the diagonal segment up to where it starts to round over. The world active-case curve athttps://www.worldometers.info/coronavirus/ seems to show this right below the 1M mark about April 3.)
2. Take the piece from the origin O (say, 50 cases) to P and make three copies. Rotate one 180 degrees and flip the other two over.
3. Glue two copies to make the top of a bell-like-but-flatter curve. The end of the third copy is the point of inflection P' opposite P, but we will be concerned with a point R somewhere midway between P' and the peak Q.
4. Place the fourth copy to complete the symmetry, but then tug its end S rightward until the diagonal line from S to R has the same slope as the third piece does at R. This process uniquely determines R.
5. Add up the total cases---this needs separate knowledge of case outcomes and when, which are not part of the active-case curve itself.
The point is, the world, many countries, and several US states can already make their "Caveman Model" projections.

Forgot to include: one needs to straighten out reporting at irregular times, such as France yesterday and China's adjustment which gave their peak an extra hump when graphed on the report-by date rather than assigned to the actual times.

Good stuff, I'm afraid it's lost in the noise. I also notice numerous double peaks at the John Hopkins site for every country, possibly reporting errors by date rather than a 'second wave' of new cases.

Bonus trivia: did the Greeks finally do something right? Their graph of new cases is very good, tending towards zero. Strict lockdown here, not even church on Holy Week allowed (actually the priest and the chanter are allowed to perform the service, alone).

I think this is something I, and some actual epidemiologists, have answered here in the past.

We don't trust models. In fact, to go further, we might not trust people who trust models!

This is still a "fog of war" situation, and probably will be until we are past the time for tactical decisions to be made. And so those decisions must be made based on rude measures of what seems to be working, and what seems to be not.

Basically, we need to all channel early Taleb on this.

channeling Taleb, who, bless his heart, is a courageous thinker, but a thinker who almost never changes his mind, is the last thing you want to do.

I don't follow Taleb seriously enough these days to have an opinion. But obviously I was wary enough to specify "early Taleb." I found his first book humorous and not too polemic.

He remains brilliant, but he is very very reluctant to change his mind.

He bragged (and with good reason) about his "precautionary principle" with respect to the millions or so of dead we are likely to see from this latest version of a coronavirus, but he has been awful quiet about the possibly even worse downstream disasters that little men in positions of power are going to make out of this official pandemic, with tens of millions out of work and hundreds of millions cut off from the economic supply stream which Taleb never mentioned in all those self-congratulatory tweets, if I am not mistaken.

I like the guy but there is no such thing as a precautionary principle, there is only risk assessment.

and if you are reading this, Taleb, if fat Tony was giving good advice in 1985, with all the respect people gave to Fat Tony's of his day, the insurance companies will all tell you --- ALL THE FAT TONYs of the 80s are very sick or dead, here and now in 2020.

so don't pull your fat Tony trump card on anyone who was, as I was, prosperous and full of common sense in 1985.

not that I am not overweight myself, but .....

the stanford antibody results that tyler linked to earlier are a game changer and this whole panic is going down in flames if they are not debunked soon.

Methinks you underestimate how wedded people are to their panic if you think scientific evidence is likely to persuade them calm tf down.

Dream on.


Why should we suddenly start trusting numbers out of China?

Think of it if an antibody study suggested that 10% to 30% of the population of Wuhan had in fact contracted the disease?

That would be a problem for many parts of the CCP narrative. It would say that testing detected a very low percentage of cases. It would call into question official numbers on fatalities. It could indicate that the various policy decisions in fact didn't do much to contain the spread, at least in Wuhan.

In other words, CCP censorship doesn't prove that these results are wrong. It does say, however, that certain results simply wouldn't and couldn't be reported.

its an good&nteresting study
how exactly did the stanford antibody study change the game?


Wait a second.

No one wants panic.

Everyone wants results. Just because there is early incompetency does not mean that others do not pick up the ball later.

No one wants anyone to die.

The Stanford study is not a game-changer. It is very clever and was done very expeditiously, and should be MUCH COPIED, but it is just Santa Clara County, a place that is not very infected. Also not very dense. Also pretty rich and can Work From Home. Also clamped down really early (Facebook by Mar 5, other big tech really soon). But then remember the study of 220 or so ladies who delivered in 2 NYC hospitals from Mar 22 to April 4. All got tested so that hospital knew what to do. 20% were infected (including 5% guess for false-negatives) and only 2% had symptoms. So some places, dense, high users of public transportation, are much more infected than the leafy, airy, car-using suburbs south of San Francisco.

> The Stanford study is not a game-changer.

Yes, it is. So, you'd expect the general population to have even more infected (as a %) sans symptoms? Which makes the numerator even bigger still?

The Stanford study is important because: It's large (3300), it's saturated with people that frequent Asia (tech workers) (ensuring that the the mix will be meaningful--if you did 3300 in Laramie, WY then you'd not see enough sick to draw a conclusion).

The Stanford study tells us that 3.3% of the population was sick but showed no symptoms. Santa Clara is 2M people--we might expect 66K to have it. But only 69 have died. That is 0.1% of the 66K.

And pull out the old people (those over 60), and you could even see that drop to 0.01%.

If NYC has an even higher % that is asymptomatic, then even better still. It means this is even far less lethal that what Santa Clara is seeing.

I think that Phinton writes "numerator" but means "denominator".

But I agree with the basic point. And it's been a hypothesis for some time: that this virus is more than "nothing", but that it's truly a problem because it combines flu-like mortality with much greater than flu-like rate of spread.

If correct, this calls for a significant relaxation of "shutdown", especially in areas where spread isn't likely to approach NYC or Lombardy levels (people don't travel by subway, social norms aren't kissing on the cheeks, etc.).

Perhaps no large gatherings, and companies (and workers) may decide that those who can easily work from home do so. Obviously focus on keeping this virus out of nursing homes and hospitals. But don't shut down society. People can go on with life, while wearing masks and not shaking hands.

Is it just me, or do people sometimes say "panic" because they want to diffuse "concern?"

If so, it is not a very nice tactic.

Anyone who suggests that any model can be used for prediction is an idiot.

All you can do is guess worst case scenarios, then see what happens.

Instead of complaining about models, tell me your guesses, and why you think they are of interest.

Consider the discussion of models as a psychological strategy to keep the educated occupied, just as the edict against masks was to prevent the hoi polloi from hoarding them.

At this point I'd settle for a model that can go back and predict what already happened.

I agree. I still don't think there is enough information available.

But even then, it is simply a lesson in why to not use models to try to predict. When a decision is needed, there isn't adequate information.

"At this point I'd settle for a model that can go back and predict what already happened."

I'm sorry, sir, we'd love to hire you but the IPCC already has all of the modeling experts it needs, thank you very much!

We may have entered an age in which cynicism is more useful than before.

My model is so crude as to be embarrassing, except that it's held up better than the fancy versions. Here's what it's saying right now:

The worst may be over for NY and Louisiana, almost surely for Washington.

New Jersey, Connecticut, Michigan, Massachusetts, DC have high death counts and have yet to calm down. Hopefully turn in next couple days.

Rhode Island is right behind with deaths surging.

Illinois, Indiana, Pennsylvania, Maryland, Delaware have reasonably high death counts and have yet to calm down. Hopefully turn in next couple days.

For everybody else (including Georgia), it appears that they will get through the first wave as well or better than the average European country.

Utah must have 'state capacity' or something.

I suspect that the quality of whoever runs the public health department is important. There isn't much that can be done, but some things done at the right time in the right amount make a difference.

Like having a book somewhere with some basic decisions done beforehand, like who is in charge, delegation of responsibility, and some basic policy triggers, contact information for hospital administration and nursing home contacts, and infection protocol documentation.

There isn't time to make basic decisions on the fly. Much of this stuff needs to be done every year, obviously at a different scale.

And some clear thinking about the tradeoffs and effectiveness of various strategies.

A very very simple thing like not having staff go from nursing home to nursing home, which can be a deadly infection source. It was implemented here a few weeks ago, but Ontario just a few days ago.

If you have to decide "jogger by themselves on a beach" or "people sitting in vehicles listening on the radio" on the fly, you should be fired for incompetence. Not for the decision, but for not thinking these things through beforehand.

That is what State Capacity looks like.

A model is just a series of related guesses. To say that people can guess but not predict is a self-contradictory statement.

Epidemiologists - Fix this now. I mean you!

"Somebody, Somebody *has* to, you see.” Then she picked out two somebodies, Tyler and me.

I was curious so I did a SIRD model myself. I just copied a paper. It was a quick hack job. It took me 3 hours and in that time my model could reproduce the deaths in New York.
Underdetermined, it predicts nothing useful. It’s manipulating too many not well known parameters and getting the right curve shapes but getting very uncertain answers further out.
Yes my model had varying Ro, masks and all that but I still think it’s mostly useless.
These models need accurate constraining data to work. That’s been in short supply.

If you had a choice:

1. The perfect model for NYC that fit the deaths
2. An imperfect model which caused you to act earlier and stronger, so as to substantially reduce death

Which would you choose?

Exactly. like we said to ourselves if you had to choose between Soviet domination of Europe and an occasional false flag attack blamed on the communist- which would you choose?

I think that philosophy perfectly encapsulates climate science today. False flags with good intentions.

I could do with a little less incandescent rage at so called “deniers”, in fact to call them “deniers” (thanks Al Gore I believe) is tendentiousness and unfair.

Bill - you've made answer 2 a tautology by assuming that it will reduce death.

The intellectual dishonesty is breathtaking.

Which side of the error term do you want to be on. And, your criticism of my comment is that more people would be killed by being restrictive than by being open. How did you get there?

It can be breathtaking, literally, if you are wrong in one direction by being too loose, which is the point of the comment, even though you apparently do not understand what the word tautology means. Look it up.

Also, you can't claim someone is intellectually dishonest if you can't even explain what you mean clearly, and, if I interpret what you say correctly, you are criticizing me for not being tough enough.

Maybe that's a tautology (Hint: it isn't; it's just repetition)

Yep. The critics who are calling for models that better account for heterogeneity, super-spreaders, time-varying R0, etc. are right in theory, but where are they going to get the data to estimate the parameters of their fancy models, or even the not-so-fancy ones? Data with good accuracy, granularity, timeliness, and consistent measurement scales and definitions.

A couple of days ago New York re-categorized 3,700 deaths as due to the novel coronavirus because although the victims were untested their symptoms were consistent with coronavirus.

Not only do we not know what the infection rates are, we do not know how many deaths we're having.

That doesn't mean the data are useless, nor that models are useless, far from it. But the need for epistemic humility has never been higher (and seems to be in short supply (I would say especially in MR's comments but that goes without saying).

It would appear that New York has just been ignoring nursing homes until 5 minutes ago. No testing, no reporting.

Why is Matt Yglesias still participating in this conversation, after Vox's breathtaking feat of being absolutely wrong about everything?

Yesterday, he was like "Durr... what's up with Rhode Island?" and I'm thinking Hell, I'm still doing my actual day job and even I knew Rhode Island was a problem a week ago. Can Matty math at all?

There's a reason they call him Ydiot.

Vox is largely a bunch of people who don't know what the hell they're talking about offering "explanations". Other than Yglesias, they tend to be young. Not surprising that they are therefore far too full of themselves. Many of us outgrow that. Some don't - that includes people who stay at Vox past the age of 30.

It's much like what The Economist has sadly become, though that publication doesn't attach names to individual articles. Those who do outgrow bad errors can therefore move on in life without being tagged for specific egregiously bad analysis.

If we had taken more aggressive travel restrictions earlier on, we could have had near zero dead, just like Taiwan.

Where is that model?

I think Nancy Pelosi dragged it through Chinatown on Feb 24 and burned it for being racist.

It's amusing that Trump supporters tout the China travel ban as something important when it's clear at this point it accomplished absolutely nothing.

I think that's right. I mean you did repeat that it accomplished nothing enough times with your Vox peers . . . so it must be true by now.

See in the alternative universe where the rest of the world is doing massive shutdowns yet the tracking dashboards show so much red that monitor makers report pixel shortages yet the US remains open and almost entirely virus free, in that alternative universe it would be really easy to say "see Trump banned Chinese people from coming and look how great America is doing"....

In this universe though we see nothing of the sort to indicate that. You want to hang your hat on can anyone prove it 'did nothing'. Well I suppose if Trump randomly shot ten people named Stan that might have also 'done something' as well.

You win a war by actually having victories. If you want a gold star for 'doing something' that produces no measurable results go find a therapist or escort to help boost your delicate self-esteem.

Yeah, it did accomplish nothing. I predicted in my comments here at the time that Canada would do no worse than the US even though it had no travel ban, and that I would change my mind on the effectiveness of the travel ban if Canada did do significantly worse than the US. I was not only right but in fact Canada has actually done significantly better than the US. Surely, if Canada was having a worse outbreak than the US, people would point to our travel ban and say it worked.

The problem with people who are still defending Trump’s travel ban on China is that they are now essentially advancing a hypothesis that is not falsifiable. That’s a sign of dogma, not scientific thinking.

Canada with a population density 1/10,000th that of NYC wouldn't fair worse than the USA?

And you predicted that all by yourself, Nostradamus?

/golf clap

Actually, the parts of Canada where most of the population lives have densities higher than most US cities (NY excepted).

And BC, the province with the largest Chinese population, has the lowest incidence of covid-19. Anecdotally, Chinese people in Vancouver were already donning masks and making their family members go into isolation when returning from China in late January...

Fact is that most infection in Canada came from Quebecers returning from the US after spring break.


A problem for keyboard pilots. The average people per square mile statistics you get from Google need to be thought about. Canada has lots of empty space but that doesn't matter for an infection unless everyone in Canada spreads themselves out evenly.

I think it's clear at this point that travel restrictions were necessary but not sufficient.

And if they were treated as sufficient, that was a problem.

I think it's clear the travel restriction was really just Trump playing the only card he knows how to do, random travel and trade restrictions to grandstand. Models have shown that if you did a 100% ban on all incoming travel you might delay your outbreak by a few weeks, even a month. The targeted type ban might buy you an extra few days. This is all well and good if you are on the ground with a real action plan to snuff out the virus that's already brewing.....but then if you're doing that you probably don't even need the travel plan.

This amounts to peeing on the floor while there's a grease fire in the kitchen. Yea a puddle of pee will slow down a fire a little bit when it gets to the puddle, but hitting the fire with a fire extinguisher will solve the problem....and make the pee unnecessary. Going back to the living room to play Playstation because you 'did something' means you might as well have not even bothered.

Travel restrictions certainly were a go-to move, and so we might be a bit rueful, but that doesn't make them wrong.

They were popular in the pandemic of 1919 I believe.

I think the point of bringing up the travel ban is in direct response to TDS news pushing a 24 hr/day narrative that trump 'didn't act fast enough'. It doesn't matter if it was effective to debunk that narrative. If you want a smarter response from trump fans, you're going to have to come up with smarter smears against trump.

And btw, if a travel ban was hailed as racist, can you even imagine what Acosta and Maddow would say if Trump had ordered a national shutdown in February (when plenty of media outlets were still running stories saying covid was just a bad flu season)?

> If we had taken more aggressive travel restrictions earlier on, we could have had near zero dead, just like Taiwan.

No, you would have merely slowed the arrival.

Case 1: You let in 64 infected people over two weeks and then close the gates

Case 2: You let in 32 infected in one weeks and then close the gates:

If we're seeing 5 days doubling, then you bought yourself 5 days by shutting travel sooner (case 2).

Remember, Biden wouldn't have closed China travel by Mid March. He was at least 6 weeks slower than Trump. That is huge.

The travel ban did nothing. Thousands of cases were already in the US

If indeed this had worked its way through the west coast in the fall, then I'd agree with you. But if in fact the first case was in January in Snohomish, then what Trump did was correct and it did slow the spread.

If this has been ripping through the world since the fall, then we made a massive mistake closing everything, but that could only have been know after the fact.

And given the info at the time, Trump's move was far more correct than Biden, Pelosi, media, etc.

Trump did not do the ban in January, he did it Feb 2. He did not ban all travel, he did not ban US citizens returning from China, he did not ban travellers who came from China but were entering the US via some other nation. The virus was already in the US and almost certainly had infected at least several thousand people by that point.

If there was a choice between what we did and having a time machine that would have caused us not to do the ban but instead start preparing from Feb 2nd onwards, we would be much better off to activate that time machine.

But the fact remains NO DEMOCRAT WOULD HAVE BANNED TRAVEL, EVEN FROM EU. They all pissed all over the EU ban.

You are arguing that Trump didnt' ban travel soon enough. Perhaps. But he stopped it as soon as practically possible. Remember, China was encouraging massive parties during LNY. At that point, a sane person would think "OK, they got this covered I guess"

Maybe there were thousands of cases here already. That's not clear yet.

But still, trump took actions every dem condemned. They were foolishly encouraging people to party and ride the subway.

If you had to pick who was more correct: Trump and his actions, which may have been too little too late, or Cuomo and his actions ("sell our ventilators, ride the subway!")

Trump was more correct. Hands down. His instincts are far superior to any dem (in our out of office) you might present.

The first known case was an American returning from Wuhan in January. That person wouldn’t have been covered by the travel ban anyway as the travel ban doesn’t prevent Americans from returning home.

> as the travel ban doesn’t prevent Americans from returning home.

No, he's not an American citizen. He's a lawful permanent resident with a Chinese passport. Permanent residents continual to hold citizenship of another country. But yes, he was allowed to return under the rule. Can you be American without being a citizen? I'm not sure. If I knew of an American that had permanent residency in France (but kept their US passport) and they considered themselves French I'd probably think that odd.

By the time of the travel ban, Americans returning from Wuhan were quarantined on military bases. If that had happened earlier - which may have been the case if the CCP didn't lie about its evidence of human-to-human transmission - that would have happened to this person also.

True, but Taiwan didn’t simply close borders.

They screened everyone and then quarantined those traveling from hot spots.

Which the US did not do, and probably lacks the capability to do.

Yeah but what did we do with the extra 5 days we bought? Nothing. If cases started exploding 5 days earlier and we started social distancing and testing 5 days earlier too, on net it would have made no difference.

Taiwan banned travel from China on February 6, four days after we did: https://www.cdc.gov.tw/En/Bulletin/Detail/KMAEC24Yf_5cm94oNL4Jxg?typeid=158.

Also, the first two countries to ban travel from China were North Korea and Russia. Russia is now experiencing a pretty fast growing outbreak and “let’s copy North Korea” is rarely a good idea.

Simple reference point:
2017/18 flu killed 60K probably infected 60M.

What would happen if we did nothing?

Best fatality rate being quoted is 0.4% which would be 2.4M people.

At the moment we have over 30K dead in a month *despite* doing massive social distancing and shutdown (imperfect shutdown but still massive shutdown). In other words we are burning thru a 2017/18 flu season at a rate of once every two months right now.

Of course that's best best best case for doing nothing:

1. We've seen much higher fatality rates. 0.4% seems to be when hospitals are doing great. When they are using duct tape to get 3 people hooked to one ventilator and nurses are using coffee filters and rubber bands for masks the fatality rate seems much higher.

2. 60M got the flu in 2017/18 but much of the population either had some pre-existing resistance to the flu from past infections and vaccinations as well as those who got the flu shot. That doesn't exist for Covid so why wouldn't more than 60M get it under a 'do nothing' regime?

3. Modelling a do nothing regime assumes everyone goes about their business as usual as deaths rack up by the tens of thousands each day and hospitals collapse. In the real world panic would soon breakout and you would have a massive 'bottom up' shutdown so you can't really model a true 'do nothing' policy because you can't hide millions of dead bodies from the public.

That's bad math. 0.4% wouldn't be 2.4 million people in the U.S even if all ~330 million people here contracted it (and they wouldn't).

Haven't read any criticism about it yet, and don't know the GRE scores or political leanings of the modellers, but here goes anyways.

Researchers at the University of Texas at Austin released a new model that finds the number of covid-19 deaths in the country has not yet peaked and will likely not reach an apex until after May 1.

Using geolocation data from cellphones to determine the impact of social distancing within each state, the model predicts that only New York and Louisiana will be past their peak by Sunday with an 80 percent certainty. Other states (New Jersey, Michigan, Colorado, Connecticut, Florida, Nevada and Massachusetts, along with the District of Columbia) have an 80 percent chance of reaching their peak in April.

The new projection counters the often-cited University of Washington’s Institute for Health Metrics and Evaluation, which predicted that deaths already peaked on Monday. The model from the University of Texas findings are slightly more grim. It forecasts an 80 percent chance that deaths will peak by May 7.

A widely followed model for projecting Covid-19 deaths in the U.S. is producing results that have been bouncing up and down like an unpredictable fever,

An ICBM takes off. A network of satellites and radar start tracking it. It appears to be heading towards NYC, but there's like a 50 mile radius circle of where it could hit. Could land in Central Park or it could land miles offshore in the ocean. What type of warhead is it carrying? Maybe 0.5 megaton, maybe 5 megatons. How many might die? Millions? Hundreds of thousands? Less than a hundred thousand.

Well clearly our biggest problem here is our model is giving us a range of different predictions.

The sky is falling. We don't know if it's a meteor, a comet, a hailstorm or some rain.

But we better sacrifice 100 bulls and 5 virgins just in case.

The shutdown has already been justified IMO by the data present. If we manage to get by with 'only' 60K or less dead that would be great.

You need to see trouble coming and react to it. Any model that you insist tell you how bad trouble will be will always give you a huge range of results. If someone is about to shoot a gun at me the range of results could go anywhere from nothing to minor flesh wound to instant death.

The rate of increase of deaths per day has been declining on average about 1 percentage point for the past eight days. It probably won't continue on that path exactly but if so there would be 400 deaths on Earth Day the 23rd when the rate of increase reaches 1% with 41,000 deaths. There will likely be between 100 and 400 deaths for a while so the final number of deaths this round will be around 50,000+.

Not sure what this means. Currently the US has about 32K deaths. Italy has 22K. Italy is 1/5th the population of the US which implies 100K for the US is quite possible.

Why has the rate of increase of deaths per day been declining 1% for the past 8 days? Almost certainly because two weeks ago or so the virus was finding it harder to infect new people, hence move forward in time there are fewer new people the virus can kill now. That means just like you can make hot cinders re-ignite with a spark, you can start increasing the daily death growth if you start allowing more infections starting today.

That's a simple 'signal model' whose purpose is not to figure out exactly how many will die a month from now but to advise whether it's sensible or not sensible to let up. There are still ways for that simple model to be wrong. For example, if almost everyone has already had the virus and has immunity then it wouldn't matter. There's few new people to get infected therefore the virus can't kill many more. But if that is true we would see impacts in other areas, like the hospitals and morgues emptying out almost immediately as soon as the current group of patients resolve....like snow melting after a fast warmup.

I've been tracking Japan, Korea, Iran and large countries in Western Europe along with radical Sweden and could see how death increases in Italy and Spain, the leaders, where decreasing as well as the pattern emerging in the U.S. It probably didn't point to a 1 percent drop in death increases a day and more around 0.7%, but I just wanted a rough estimate. Knowing that 41,000 was likely low, I added 20,000 but think U.S. deaths for this period will be around 50,000.

"Italy is 1/5th the population of the US which implies 100K for the US is quite possible."

I forgot to respond that 100,000 American deaths isn't possible for this round. I can't see it going above 60,000 but possible maybe 70,000 is possible. One recent study put the maximum at 70,000.

My signal model wouldn't change things much if the 'right' figure is 60, 70 or 80. At the moment we are at 36K.

How do you determine what a 'round' is and when one ends and another begins? If tomorrow deaths start picking up are we still in 'this round' or the second?

100k U.S. deaths compares to just under 3 million deaths of all causes in a typical year. COVID-19 deaths are obviously personal tragedies for everyone affected, but the age and co-morbidities indicate that these deaths are largely (well over 50%) pulled forward for people who would have died within the next few years absent COVID-19.

This is not how you do things. For example, if homicides are tracking a few hundred down for the year and one day someone shows up and starts shooting at seniors getting off a bus to go to the casinos at Atlantic City, the responses are not:
"Well he only has a hundred bullets at most, no need for us to send anyone"
"Well he's just pulling forward some deaths from the future".

In cities like NY the daily death toll is several times any normal high. I guess death is going to drop to zero for the rest of the year once the virus passes.

Being on a ventilator for a week or two has unknown long term effects on health, but it is doubtful it will be positive. How much lifespan will be contracted in those who survived the virus? You avoid risks where the left tail is unknown and likely fat tailed.

"An ICBM ... appears to be heading towards NYC, but there's like a 50 mile radius circle of where it could hit."

Perfect analogy, but you should extend it further: as the ICBM progresses on its trajectory and we collect more and more info about it, we can narrow the range of possible impacts--that 50-mile CEP--to a much smaller and more localized area.

The criticisms of the IMHE model in the linked article are well founded, and have been discussed widely in epidemiological circles. The accusatory tone of this post, on the other hand, is unjustified, pointless, and very un-Tyler. One suspects that Tyrone is involved...

Ahhh Tyler's evil twin. I'm reminded of Garthe Knight, the evil counter part to Michael Knight in Knight Rider....and his semi-truck Goliath.

I don’t know that I agree with the rest of your commentary, but the Garth/Goliath episodes were awesome.

I have seen serious criticism from the get go. It does curve fitting, which is not a good approach for unknown situations where things are changing, like distancing.

I wonder if epidemiologists will face the animosity that economists faced for "failing to predict the financial crisis".

They will face animosity for terrible models and grossly overstating the risk. Even now people are focused solely on the virus, pointlessly debating CFR percentages while ignoring the economic tsunami that is approaching. The virus doesn't matter anymore. The economic dominos have started falling, slowly now but they will accelerate soon.

The Commonwealth of Massachusetts has been at war with IHME for weeks now, as the IHME model has made absolutely no sense for at least that long. They rely on models from their public health authorities and local epidemiologists. This is in the governor's daily briefings.

While the press took too long to pick up on this, it has been obvious for a _very_ long time that actual epidemiologists and those in the trenches were unhappy with the model.

The problem here is risk. The epidemiologists are taking the risk to human lives in their ideas of how one should proceed. The risk of human death is far less important to economists. Economist say death at what cost?

Everyone gravitated to IHME because it was beautiful, fast, and comprehensive. They didn't care about accuracy because, well, look how quickly I would tell you'd need 40K ventilators! And I could tell you exactly the day you'd need 40K ventilators. Turns out nobody needed 40K ventilators. Ever.

Models that are brain-dead simple are of little use other that helping someone understand the most basic levers at work, and models that are sophisticated are rife with biases (see climate models).

How did so many comments completely miss the point that Tyler was trying to make?

He's highlighting the lack of vigor in the field of epidemiology. This was epidemiologists' time to shine, to take the lead, and they didn't. There should have been a highly visible public debate between respected epidemiologists with different projections. The public would then assess the facts on either side and settle on some equilibrium determined by who has a stronger argument. The stock market, our political system, board of directors, etc...all have that kind of "push and pull" dynamic.

That lack of gusto is endemic in the whole scientific field more broadly. There is a misguided cultural expectation in the scientific community to put your head down and be "humble" in these situations, when in fact the nation is missing out on critical expert guidance. I think this neutered culture was born out of the rigid hierarchies in the scientific field. Smart scientists who otherwise should be given a platform live in fear of being publically denounced and flamed by more "respected" scientists. See the history of Alzheimer's research for Exhibit A. The solution is to add more sources of reward for intelligent scientists who maybe don't have tons of citations yet or are earlier in their career. Their career success shouldn't be determined by who is lording over them or how "reasonable" they seem, but rather by their merit and contributions to the broader community.

You know, Carl,

There is this epidemiological site highlighting the lack of vigor in the field of economics.

2020 2008, 2001, 1990,1972, etc. to 1929were the years for the economists time to shine, take the lead, and they didn't. There should have been a highly visible public debate between respected economists with different projections. The public would then assess the facts on either side and settle on some equilibrium determined by who has a stronger argument. The stock market, our political system, board of directors, etc...all have that kind of "push and pull" dynamic.

See the history of Alzheimer's research for Exhibit A.

That was gold Bill. Perhaps to your surprise, I am in complete agreement with your implied opinion.

I am glad we are of the same spirit.

Perhaps we can model that behavior for others.


We absolutely can and should model that behavior for others. Modern internet technology has enabled a new degree of adjudication of not only expert opinions, but each assumption underlying each expert opinion.

When making a projection, no expert can be deemed to be "correct." However, if we force the expert to divulge their underlying assumptions, we can find assumptions which are dubious. We then switch out the faulty assumptions with the assumptions of another expert which are deemed not to be dubious. By adjudicating the inputs to expert models, each of which can be debated by a pool of experts, you can create a hybrid model which is more likely to be correct and free of idiosyncratic errors, biases, white lies, Straussian nonsense, or hidden motivations.

And who is to decide what is dubious and what is not?

And are you seriously suggesting that scientists--who are, as I assume you are aware, human beings who put their pants on one leg at a time, just like the rest of us--are "...free of idiosyncratic errors, biases, white lies, Straussian nonsense, or hidden motivations..."? Let's try this on:

1. All humans have agendas and make mistakes.
2. All scientists are human.
3. Therefore, all scientists have agendas and make mistakes.

>"Who is to decide what is dubious...?"

The group of expert scientists can vote on each assumption. Wisdom of the crowds is usually used to adjudicate between model outputs, but here I propose using wisdom of the crowds to judge the salient inputs.

>"are you....seriously suggesting....scientists are free of errors, biases...."
Of course not, and that's the point, but I believe a large group of expert scientists would fare better on this front than a single person. I would compare the risk of a single scientist's errors and biases to idiosyncratic risk in the markets - you generally do not want it, you can mitigate it by diversifying your position, and the larger the pool of securities the less exposure you have to idiosyncratic risk - controlling for the % of your portfolio that each security takes. (not a 100% perfect analogy but you get my point)

Additionally, I said that the modified output from a group of scientists would be "more likely" to be correct and free....than that from an individual, so please don't get me wrong, the output will almost never be free of those things, but I believe it would most likely be some % better on that front. Lastly, there would have to be stringent procedures to avoid groupthink, which would actually lead to more bias. There are many studies that show how to do this, including forcing voters to cast their opinion before discussion or seeing other votes.

One of the problems, Carl, that I see is we should specify what is the purpose of the model first.

If you are an actuary, your interest may lie in one direction; if you are a policymaker, you may only be interest in direction and a range of magnitudes.

There should have been a highly visible public debate between respected epidemiologists with different projections. The public would then assess the facts on either side and settle on some equilibrium determined by who has a stronger argument. The stock market, our political system, board of directors, etc...all have that kind of "push and pull" dynamic.

Are you freaking kidding? Is this a joke? I'm sorry but there's no debate, what you're describing is shopping. "Ohhh I'll take the estimate that says 30K dead rather than 300K"

This is a bizarre take. There is no "correct" prediction. The future is not obligated to follow your pet model.

Sure, people might pick the model that follows their political bias. So what? When your preferred model is shown to be flawed, their political bias will be reinforced 10x anyway. Noble lies are just a great way to garner unneeded distrust.

Noble lie implies some 'good' model that is being hidden because a bad model will achieve some needed policy.

Thanks HealthActuary. I would add that in a theoretical debate between 30k and 300k, if the experts who estimate 30k have a stronger argument but a couple dubious assumptions, and the experts who estimate something closer to 300k have a weaker argument due to a higher proportion of dubious assumptions or flawed modeling, then we might settle at something closer to 70k. The assumptions would be adjudicated by a crowdsourced group of epidemiologists who are far greater in number than the epidimiologists who have the time to make models and projections. The result of all this is you would take the best assumptions and modeling methodology from each camp and hybridize it, resulting in a model which is more likely to be accurate.

“though they are at fault as well”
-this whole post is about how IHME isn’t necessarily a bad model, but somehow the trump administration is “at fault”? Tyler’s TDS is getting pathological.

"…The chief reason the IHME projections worry some experts, Etzioni said, is that “the fact that they overshot” — initially projecting up to 240,000 U.S. deaths, compared with fewer than 70,000 now — “will be used to suggest that the government response prevented an even greater catastrophe, when in fact the predictions were shaky in the first place.”

WILL SOMEONE PLEASE TELL ME if I should be mad a trump for underplaying the risks and murdering hundreds of thousands of Americans of for overplaying them and crashing the economy just to make it look like he handled things competently.

Seriously, I didn't vote for trump, don't plan to next time around, but if things calm down, then we get a second wave and it heats up again, I'm going to be too dizzy from all the spin to stand.

Sure someone has said this above..... this is all silly. I think the model (ok, I can't remember whether this one or another) predicted 60k deaths at the lower end. Seems to me like we are getting there.

Excuse me, but after Tyler's post, 149 comments, and following a lot of links, I still do not know what the IHME model is. Can someone point me to a page where this model is scientifically described? what are the parameters, the variables, the assumptions, the evolution equations?

That's a good point Joel. As far as I can tell, they have two scrutable publications regarding COVID-19 with an explanation of their methodology. They can both be found at the link below. However, the latest of the two was published March 26. Ideally they would have posted revisions for every change in their projections. Otherwise it is difficult to tell which changes are due to underlying changes in the model, or changes to the input factors which are derived from new observations.


Are you saying that you want them to be screaming or that you think this isn't that important since they're not screaming?

I complained recently in a response to one of your posts that this projection had no accountability, that without being able to see how its previous projections fit actual data, we were constantly given a moving projection of unknown credibility. (And, of course, each project assumes some mitigation level that some countries or states aren't using and others are using even more of.)

Australia is currently at 2.5 COVID-19 deaths per million. The US is at 112 deaths per million. Given roughly comparable standards of living an per capita GDP and the similar warning, people in the US should be dying at a similar rate to Australia.

The US is spending over $600 billion a year a year on it's military for the supposed purpose of protecting US lives, but couldn't even manage to stop a virus from taking over 34,000 lives instead of the one or two thousand that may have been lost with a more competent response. This is shameful.

It's summer in Oz. The attack vector was already in the USA by CES in January.

I don't know what you think was possible even now, in heinsight. You think with 6 deaths the country try was going to shut down the economy?

Politically that wasn't possible.

Due to the lack of physical separation between the Northern and Southern hemispheres, it's not possible for it to be spring in the US and summer in Australia.

If you really think warmer conditions in Australia are responsible for the 40 times difference in death rates, then look at the following deaths per million for:

New Zealand: 2
Japan: 2
China: 3
South Korea: 5

Bad luck plays a part, but not as large a part as 40 times Australia's death rate. That takes willful incompetence. The Australian response was woefully incompetent but -- and here's the difference -- it did not willfully persist in this incompetence once the evidence we were on the path to disaster accumulated.

Just wanna be sure I am tracking the core complaint here.

The profession that has missed predicting every major economic event in modern history, and still can't tell us why people buy lottery tickets, is upset that another forecasting profession was unable to predict what would happen with a literally brand new disease, with effectively zero data, with literally human lives at stake, while the national political leadership was pretending it didn't even exist, and then missed their estimate by less than one tenth of one percent of the US population.

…The chief reason the IHME projections worry some experts, Etzioni said, is that “the fact that they overshot” — initially projecting up to 240,000 U.S. deaths, compared with fewer than 70,000 now — “will be used to suggest that the government response prevented an even greater catastrophe, when in fact the predictions were shaky in the first place.”

I think this needs some clarity. At first glance it sounds like this is saying if we did not shutdown, we would have been facing 240,000 dead but now we are expecting only 70,000.

But that does not make any sense. Even with the lowest mortality rate a 'do nothing' policy would have produced hundreds of thousands to millions dead.

What this is saying I think is with all our shutdown policies, we were looking at 240,000 dead. Now with all our shutdown policies we are looking at 70,000. It's a statement about a model's projections that have no policy implications, you still shutdown.

This guy wants to shoot you, he'll kill you, take cover! Ohhh wait he only wants to aim for your legs, instead of killing you you'd only get a wound. Doesn't change the policy you still take cover. Is the fact that your 'model' for what will happen if the guy shoots you jumps all over something that is really important to nail down?

Whenever someone criticizes something, I try to take a careful look at their suggested alternative. Here's that stat news article's:
"Three weeks ago a SEIR model from researchers at the Massachusetts Institute of Technology projected that total U.S. cases will plateau later this week, reaching 600,000 and then adding ever-fewer cases each day. So far it’s pretty much on the money, with the U.S. case count at 650,000 on Thursday and new daily cases remaining mostly flat." (1)

That model, based on the data so far, wasn't much better. They projected a peak of 600,000 active cases, but we're at 612000 right now according to worldometers and continuing to climb. If we compare like with like, and look at the IHME paper from a week prior to that paper (giving a large advantage to the stat news article's preferred method in addition to their advantage of being allowed to cherry pick any projection they want), then the IHME's paper on projected cumulative deaths (https://www.medrxiv.org/content/10.1101/2020.03.27.20043752v1.full.pdf) put the total at 81,114 deaths. Three weeks later and worldometers data is still in the ballpark of their projections. True, the IHME way overestimated on hospitalizations, but they did okay on the peak date of hospitalizations. And their deaths projection appears to be roughly as accurate as MIT's active cases projection.

1 The stat news article is wrong about the time frame. The MIT paper was published based on data available up until April 1st, two weeks and two days before the stat news article was written; not three weeks before. The stat news article also mistakenly looked at the total cases rather than the active cases when making a comparison.

In short, the linked article is nonsense. Very interesting. Thanks.

The volatility problem with the models may not be the models themselves, but in the data they are being fed. We now that we know almost nothing about any of the relevant figures: how many have had the virus; how many have had it but displayed no symptoms or relatively mild ones; or even how many have died OF the virus, as opposed to its being in their bodies when they died.

My guess is, until we are on the back side of the growth curve--the logarithmic part of the "S-curve," past the inflection point--we won't know how to answer any of these questions. Nor is that idea particularly original with me.

Trees don't grow to the sky, a wise man once told me. You cannot naively extrapolate a trend until you know the shape of the trend line. Since such data as we have is noisy and possibly corrupt--to say nothing of a lack of a common definition of seemingly straightforward concepts such as "deaths from the virus"--we won't actually know anything until we are further along in the process.

I'm not sure why you think this proves your point? This model is an exercise in curve fitting, and it has been loudly criticized by Carl Bergstrom among others. Find a SEIR model built by a prominent group that says the same thing and then maybe you'd have a point.

As for why it's influential, people like to choose the models that confirm their biases. As a right leaning economist, I'd assume you're very familiar with this approach.

Comments for this post are closed