Here is a new paper by Seth Benzell, Avinash Collis, and Christos Nicolaides:
To prevent the spread of COVID-19, some types of stores and gathering places have been shut down while others remain open. The decision to shut down one type of location and leave another open constitutes a judgement about the relative danger and benefits of those locations. Using location data from a large sample of smartphones, nationally representative consumer preference surveys, and government statistics, we measure the relative transmission risk benefit and social cost of closing 30 different location categories in the US. Our categories include types of shops, schools, entertainments, and public spaces. We rank categories by those which should face stricter regulation via dominance across eight dimensions of risk and importance and through composite indexes. We find that from February to March, there were larger declines in visits to locations that our measures imply should be closed first. We hope this analysis will help policymakers decide how to reopen their economies.
Here is a summary picture:
MOMA take note!
Obviously such rankings are somewhat speculative, but sooner or later some kind of disaggregated road map like this is going to be necessary. And I would say sooner.
Here is Johan Giesecke, Swedish epidemiologist, interviewed by Freddie Sayers for a little over half an hour, one of the most interesting set pieces I have heard this year. “I’m going to tell you what I really think. I don’t usually do that.” He also gives his account of what Sweden did right and wrong, and he argues that more Swedes than Norwegians have died because a) Sweden has much larger nursing homes, and b) Swedish immigrants. How he puts matters is of great interest as well.
It is a good exercise to figure out exactly where and why his claims might be wrong. For a start, I don’t think his extreme claims about fatality and infection rates can be true, even if you agree with much of what he says.
And from the excellent Samir Varma here is a new Bloomberg article about the Swedes claiming success.
Here is part of the list of winners, there are more to follow soon, and I am happy to cite Mercatus Center, George Mason University as home to the project.
I would again like to thank everyone who helped to make this possible, most of all those who have offered very generous financial support.
To date Fast Grants has made 67 awards to support biomedical research. Fast Grants did not exist as recently as twelve days ago and it already has distributed more than $12 million.
From my email box, here are perspectives from people in the world of epidemiology, the first being from Jacob Oppenheim:
I’d note that epidemiology is the field that has most embraced novel and principles-driven approaches to causal inference (eg those of Judea Pearl etc). Pearl’s cluster is at UCLA; there’s one at Berkeley, and another at Harvard.
The one at Harvard simultaneously developed causal methodologies in the ’70s (eg around Rubin), then a parallel approach to Pearl in the ’80s (James Robins and others), leading to a large collection of important epi people at HSPH (Miguel Hernan, etc). Many of these methods are barely touched in economics, which is unfortunate given their power in causal inference in medicine, disease, and environmental health.
These methods and scientists are very influential not only in public health / traditional epi, but throughout the biopharma and machine learning worlds. Certainly, in my day job running data science + ml in biotech, many of us would consider well trained epidemiologists from these top schools among the best in the world for quantitative modeling, especially where causality is involved.
From Julien SL:
I’m not an epidemiologist per se, but I think my background gives me some inputs into that discussion. I have a master in Mechatronics/Robotics Engineering, a master in Management Science, and an MBA. However, in the last ten years, epidemiology (and epidemiology forecasting) has figured heavily in my work as a consultant for the pharma industry.
[some data on most of epidemiology not being about pandemic forecasting]…
The result of the neglect of pandemics epidemiology is that there is precious little expertise in pandemics forecasting and prevention. The FIR model (and it’s variants) that we see a lot these days is a good teaching aid. Still, it’s not practically useful: you can’t fit exponentials with unstable or noisy parameters and expect good predictions. The only way to use R0 is qualitatively. When I saw the first R0 and mortality estimates back in January, I thought “this is going to be bad,” then sold my liquid assets, bought gold, and naked puts on indices. I confess that I didn’t expect it to be quite as bad as what actually happened, or I would have bought more put options.
…here are a few tentative answers about your “rude questions:”
a. As a class of scientists, how much are epidemiologists paid? Is good or bad news better for their salaries?
Glassdoor data show that epidemiologists in the US are paid $63,911 on average. CDC and FDA both pay better ($98k and $120k), as well as pharma (Merck: $94k-$115k). As explained above, most are working on cancer, diabetes, etc. So I’m not sure what “bad news” would be for them.
b. How smart are they? What are their average GRE scores?
I’m not sure where you could get data to answer that question. I know that in pharma, many – maybe most – people who work on epidemiology forecasting don’t have an epidemiology degree. They can have any type of STEM degree, including engineering, economics, etc. So my base rate answer would be average of all STEM GRE scores. [TC: Here are U. Maryland stats for public health students.]
c. Are they hired into thick, liquid academic and institutional markets? And how meritocratic are those markets?
Compared to who? Epidemiology is a smaller community than economics, so you should find less liquidity. Pharma companies are heavily clustered into few geographies (New Jersey, Basel in Switzerland, Cambridge in the UK, etc.) so private-sector jobs aren’t an option for many epidemiologists.
d. What is their overall track record on predictions, whether before or during this crisis?
CDC has been running flu forecasting challenges every year for years. From what I’ve seen, the models perform reasonably well. It should be noted that those models would seem very familiar to an econometric forecaster: the same time series tools are used in both disciplines. [TC: to be clear, I meant prediction of new pandemics and how they unfold]
e. On average, what is the political orientation of epidemiologists? And compared to other academics? Which social welfare function do they use when they make non-trivial recommendations?
Hard to say. Academics lean left, but medical doctors and other healthcare professionals often lean right. There is a conservative bias to medicine, maybe due to the “primo, non nocere” imperative. We see that bias at play in the hydroxychloroquine debate. Most health authorities are reluctant to push – or even allow – a treatment option before they see overwhelming positive proof, even when the emergency should encourage faster decision making.
…g. How well do they understand how to model uncertainty of forecasts, relative to say what a top econometrician would know?
As I mentioned above, forecasting is far from the main focus of epidemiology. However, epidemiologists as a whole don’t seem to be bad statisticians. Judea Pearl has been saying for years that epidemiologists are ahead of econometricians, at least when it comes to applying his own Structural Causal Model framework… (Oldish) link: http://causality.cs.ucla.edu/blog/index.php/2014/10/27/are-economists-smarter-than-epidemiologists-comments-on-imbenss-recent-paper/
I’ve seen a similar pattern with the adoption of agent-based models (common in epidemiology, marginal in economics). Maybe epidemiologists are faster to take up new tools than economists (which maybe also give a hint about point e?)
h. Are there “zombie epidemiologists” in the manner that Paul Krugman charges there are “zombie economists”? If so, what do you have to do to earn that designation? And are the zombies sometimes right, or right on some issues? How meta-rational are those who allege zombie-ism?
I don’t think so. Epidemiology seems less political than economy. There are no equivalents to Smith, Karl Marx, Hayek, etc.
i. How many of them have studied Philip Tetlock’s work on forecasting?
Probably not many, given that their focus isn’t forecasting. Conversely, I don’t think that Tetlock has paid much attention to epidemiology. On the Good Judgement website, healthcare questions of any type are very rare.
And here is Ruben Conner:
Weighing in on your recent questions about epidemiologists. I did my undergraduate in Economics and then went on for my Masters in Public Health (both at University of Washington). I worked as an epidemiologist for Doctors Without Borders and now work as a consultant at the World Bank (a place mostly run by economists). I’ve had a chance to move between the worlds and I see a few key differences between economists and epidemiologists:
Trust in data: Like the previous poster said, epidemiologists recognize that “data is limited and often inaccurate.” This is really drilled into the epidemiologist training – initial data collection can have various problems and surveys are not always representative of the whole population. Epidemiologists worry about genuine errors in the underlying data. Economists seem to think more about model bias.
Focus on implementation: Epidemiologists expect to be part of the response and to deal with organizing data as it comes in. This isn’t a glamorous process. In addition, the government response can be well executed or poorly run and epidemiologists like to be involved in these details of planning. The knowledge here is practical and hands-on. (Epidemiologists probably could do with more training on organizational management, they’re not always great at this.)
Belief in models: Epidemiologists tend to be skeptical of fancy models. This could be because they have less advanced quantitative training. But it could also be because they don’t have total faith in the underlying data (as noted above) and therefore see fancy specifications as more likely to obscure the truth than reveal it. Economists often seem to want to fit the data to a particular theory – my impression is that they like thinking in the abstract and applying known theories to their observations.
As with most fields, I think both sides have something to learn from each other! There will be a need to work together as we weigh the economic impacts of suppression strategies. This is particularly crucial in low-income places like India, where the disease suppression strategies will be tremendously costly for people’s daily existence and ability to earn a living.
And here is from an email from epidemiologist Dylan Green:
So with that…on to the modelers! I’ll merely point out a few important details on modeling which I haven’t seen in response to you yet. First, the urgency with which policy makers are asking for information is tremendous. I’ve been asked to generate modeling results in a matter of weeks (in a disease which I/we know very little about) which I previously would have done over the course of several months, with structured input and validation from collaborators on a disease I have studied for a decade. This ultimately leads to simpler rather than more complicated efforts, as well as difficult decisions in assumptions and parameterization. We do not have the luxury of waiting for better information or improvements in design, even if it takes a matter of days.
Another complicated detail is the publicity of COVID-19 projections. In other arenas (HIV, TB, malaria) model results are generated all the time, from hundreds of research groups, and probably <1% of the population will ever see these figures. Modeling and governance of models of these diseases is advanced. There are well organized consortia who regularly meet to present and compare findings, critically appraise methods, elegantly present uncertainty, and have deep insights into policy implications. In HIV for example, models are routinely parameterized to predict policy impact, and are ex-post validated against empirical findings to determine the best performing models. None of this is currently in scope for COVID-19 (unfortunately), as policy makers often want a single number, not a range, and they want it immediately.
I hope for all of our sakes we will see the modeling coordination efforts in COVID-19 improve. And I ask my fellow epidemiologists to stay humble during this pandemic. For those with little specialty in communicable disease, it is okay to say “this isn’t my area of expertise and I don’t have the answers”. I think there has been too much hubris in the “I-told-ya-so” from people who “said this would happen”, or in knowing the obvious optimal policy. This disease continues to surprise us, and we are learning every day. We must be careful in how we communicate our certainty to policy makers and the public, lest we lose their trust when we are inevitably wrong. I suspect this is something that economists can likely teach us from experience.
One British epidemiologist wrote me and told me they are basically all socialists in the literal sense of the term. not just leaning to the left.
Another person in the area wrote me this:
Another issue that isn’t spoken about a lot is most Epidemiologists are funded by soft money. It makes them terrifyingly hard working but it also makes them worried about making enemies. Every critic now will be reviewed by someone in IHME at some point in an NIH study section, whereas IHME, funded by the Gates Foundation, has a lot of resilience. It makes for a very muted culture of criticism.Ironically, outsiders (like economist Noah Haber) trying to push up the methods are more likely to be attacked because they are not a part of the constant funding cycle.I wonder if economists have ever looked at the potential perverse incentives of being fully grant funded on academic criticism?
As a consequence of missing data on tests for infection and imperfect accuracy of tests, reported rates of population infection by the SARS CoV-2 virus are lower than actual rates of infection. Hence, reported rates of severe illness conditional on infection are higher than actual rates. Understanding the time path of the COVID-19 pandemic has been hampered by the absence of bounds on infection rates that are credible and informative. This paper explains the logical problem of bounding these rates and reports illustrative findings, using data from Illinois, New York, and Italy. We combine the data with assumptions on the infection rate in the untested population and on the accuracy of the tests that appear credible in the current context. We find that the infection rate might be substantially higher than reported. We also find that the infection fatality rate in Italy is substantially lower than reported.
Here is a very good tweet storm on their methods, excerpt: “What I love about this paper is its humility in the face of uncertainty.” And: “…rather than trying to get exact answers using strong assumptions about who opts-in for testing, the characteristics of the tests themselves, etc, they start with what we can credibly know about each to build bounds on each of these quantities of interest.”
I genuinely cannot give a coherent account of “what is going on” with Covid-19 data issues and prevalence. But at this point I think it is safe to say that the mainstream story we have been living with for some number of weeks now just isn’t holding up.
For the pointer I thank David Joslin.
Here’s the latest video from MRU where I cover some interesting papers on the effect of pollution on health, cognition and productivity. The video is pre-Covid but one could also note that pollution makes Covid more dangerous. For principles of economics classes the video is a good introduction to externalities and also to causal inference, most notably the difference in difference method.
Might I also remind any instructor that Modern Principles of Economics has more high-quality resources to teach online than any other textbook.
Hawaii Department of Health officials said today that the state’s tally of coronavirus cases has risen to 553, up 12 from Thursday.
Of all the confirmed cases in Hawaii since the start of the outbreak, 48 have required hospitalizations, with three new cases reported today, health officials said.
The state’s coronavirus death toll stands at nine, unchanged from Thursday. Six of the deaths were on Oahu, while three were in Maui.
The population of Hawaii is about 1.4 million. Three days ago, Hawaii was the lowest infection rate in the United States, but of course more and better data are needed. We’ll see, with the passage of time, if this remains a true heterogeneity. But do note this:
It is also noteworthy that Hawaii tests for coronavirus at a considerably higher rate than most states. According to data compiled by Vox, Hawaii continues to rank among the top 10 states for testing per capita, which suggests Hawaii’s infection rate may be more accurate than rates reported by some other states.
A widely followed model for projecting Covid-19 deaths in the U.S. is producing results that have been bouncing up and down like an unpredictable fever, and now epidemiologists are criticizing it as flawed and misleading for both the public and policy makers. In particular, they warn against relying on it as the basis for government decision-making, including on “re-opening America.”
“It’s not a model that most of us in the infectious disease epidemiology field think is well suited” to projecting Covid-19 deaths, epidemiologist Marc Lipsitch of the Harvard T.H. Chan School of Public Health told reporters this week, referring to projections by the Institute for Health Metrics and Evaluation at the University of Washington.
Others experts, including some colleagues of the model-makers, are even harsher. “That the IHME model keeps changing is evidence of its lack of reliability as a predictive tool,” said epidemiologist Ruth Etzioni of the Fred Hutchinson Cancer Center, home to several of the researchers who created the model, and who has served on a search committee for IHME. “That it is being used for policy decisions and its results interpreted wrongly is a travesty unfolding before our eyes.”
…The chief reason the IHME projections worry some experts, Etzioni said, is that “the fact that they overshot” — initially projecting up to 240,000 U.S. deaths, compared with fewer than 70,000 now — “will be used to suggest that the government response prevented an even greater catastrophe, when in fact the predictions were shaky in the first place.”
Here is the full story, from StatNews, by Sharon Begley with assistance from Helen Branswell, two very good and knowledgeable sources. Via Matt Yglesias.
To be clear, I am (and always have been) fully aware that there are more nuanced epidemiological models “sitting on on the shelf,” just as is true for macroeconomics and many other areas. But I ask you, where are the numerous cases of leading epidemiologists screaming bloody murder to the press, or on their blogs, or in any other manner, that the most commonly used model for this all-important policy analysis is deeply wrong and in some regards close to a fraud? Yes I know you can point to a few tweets from the more serious people, but where has the profession as a whole been? Who organized the protest letter and petition to The Wall Street Journal?
And to be clear, I have heard this model cited and discussed in many (off the record) policy discussions, this is not just something you can pin on the Trump administration narrowly construed (though they are at fault as well).
I will be doing a Conversation with him, mostly about his ideas on Covid-19 response and testing, though we will cover other topics as well. So what should I ask him?
How do you feel about that statement? I take this as one psychometric test.
If your reaction is: “My goodness, these are tragic times but it is splendid and noble how we all can come together and sacrifice for a common endeavor!”…well…
…you have failed my test and I will suspect a wee bit of mood affiliation. Most likely it is bad news if the relative safety (for some) of the current moment comes from social distancing. Because at some point social distancing must end, or at least be significantly curtailed, and then a higher danger level may well reemerge.
Possibly you have inside information that a cure will be ready next week, but somehow I doubt it. You are happy because you like something about the process.
Alternatively, if you hear “social distancing is working so well!” and immediately feel a deep sense of foreboding, and begin to calculate whether good short-term results are correlated with better or worse long-term results. And then you calculate how how long the distancing can last for, due to governmental budget constraints, and then try to figure out what kinds of progress we might make in the meantime while the distancing lasts, and then start worrying about how reliant on social distancing we are becoming…
…But then you undertake a second-order calculation about how the greater danger spurred by the forthcoming decline in social distancing also might spur innovation…
And then you think “would it not be better if the current progress came from a more sustainable source, what might that be, how about faster than expected herd immunity amongst a relatively small group of heterogeneous super-spreaders, now what is the chance of that?”…
…and finish your analysis confused…
Then you are my kind of weirdo.
We are living in a time of psychometric tests.
That is the topic of my latest Bloomberg column, here is one excerpt:
Now consider issues beyond specific user groups. The U.S. will almost certainly need to introduce a “track and trace” system, using information technology, preferably with privacy safeguards. One version of this idea uses geolocation methods, which tracks where people are in physical space and sends individuals a text message if they come into close contact with others diagnosed with Covid-19.
That technology requires participants to have a smartphone. The federal government probably will not mandate smartphone usage, which would both be politically unpopular and difficult to enforce. Nonetheless, businesses are likely to turn to such schemes to increase workplace safety. But again, exactly who already owns or afford a smartphone? Some of the jobs with the closest physical contact, such as service jobs, employ relatively low paid workers.
Companies may well decide to help workers buy smartphones, perhaps with government subsidies too. But that would then make having a smartphone a job requirement, including in the retail and public sectors.
This would create a new and in some ways more serious digital divide. Imagine you want to visit your local shopping mall. Its owners might require that you subscribe to one of the Covid-19 tracing apps. Or imagine not being able to get your license renewed without a smartphone certifying your health status.
All of a sudden the U.S. will have a new segregation — between those who have smartphones and those who don’t. If you’re on the wrong side of that divide, many places and services will be hard if not impossible to reach.
And to close:
It is plausible that the U.S. could end up with 10% or more of the population exiled from many key institutions of American life — simply because they lack the right kind of technology.
Don’t get me wrong; the digital divide deserves the additional attention soon to come its way. The trick will be ensuring that any proposed solutions don’t just trade one kind of divide for another.
I can’t even figure out how to work those parking spots that are “app only” for the parking meter. Pity me!
As you may recall, the goal of Fast Grants is to support biomedical research to fight back Covid-19, thus restoring prosperity and liberty.
Yesterday 40 awards were made, totaling about $7 million, and money is already going out the door with ongoing transfers today. Winners are from MIT, Harvard, Stanford, Rockefeller University, UCSF, UC Berkeley, Yale, Oxford, and other locales of note. The applications are of remarkably high quality.
Nearly 4000 applications have been turned down, and many others are being put in touch with other institutions for possible funding support, with that ancillary number set to top $5 million.
The project was announced April 8, 2020, only eight days ago. And Fast Grants was conceived of only about a week before that, and with zero dedicated funding at the time.
I wish to thank everyone who has worked so hard to make this a reality, including the very generous donors to the program, those at Stripe who contributed by writing new software, the quality-conscious and conscientious referees and academic panel members (about twenty of them), and my co-workers at Mercatus at George Mason University, which is home to Emergent Ventures.
I hope soon to give you an update on some of the supported projects.
Under Swiss law, every resident is required to purchase health insurance from one of several non-profit providers. Those on low incomes receive a subsidy for the cost of cover. As early as March 4, the federal health office announced that the cost of the test — CHF 180 ($189) — would be reimbursed for all policyholders.
The U.S. government will nearly double the amount it pays hospitals and medical centers to run Abbott Laboratories’ large-scale coronavirus tests, an incentive to get the facilities to hire more technicians and expand testing that has fallen significantly short of the machines’ potential.
Abbott’s m2000 machines, which can process up to 1 million tests per week, haven’t been fully used because not enough technicians have been hired to run them, according to a person familiar with the matter.
In other words, we have policymakers who do not know that supply curves slope upwards (who ever might have taught them that?).
The same person who sent me that Swiss link also sends along this advice, which I will not further indent:
“As you know, there are 3 main venues for diagnostic tests in the U.S., which are:
1. Centralized labs, dominated by Quest and LabCorp
2. Labs at hospitals and large clinics
3. Point-of-care tests
There is also the CDC, although my understanding is that its testing capacity is very limited. There may be reliability issues with POC tests, because apparently the most accurate test is derived from sticking a cotton swab far down in a patient’s nasal cavity. So I think this leaves centralized labs and hospital labs. Centralized labs perform lots of diagnostic tests in the U.S. and my understanding is this occurs because of their inherent lower costs structures compared to hospital labs. Hospital labs could conduct many diagnostic tests, but they choose not to because of their higher costs.
In this context, my assumption is that the relatively poor CMS reimbursement of COVID-19 tests of around $40 per test, means that only the centralized labs are able to test at volume and not lose money in the process. Even in the case of centralized labs, they may have issues, because I don’t think they are set up to test deadly infection diseases at volume. I’m guessing you read the NY Times article on New Jersey testing yesterday, and that made me aware that patients often sneeze when the cotton swab is inserted in their noses. Thus, it may be difficult to extract samples from suspected COVID-19 patients in a typical lab setting. This can be diligence easily by visiting a Quest or LabCorp facility. Thus, additional cost may be required to set up the infrastructure (e.g., testing tents in the parking lot?) to perform the sample extraction.
Thus, if I were testing czar, which I obviously am not, I would recommend the following steps to substantially ramp up U.S. testing:
1. Perform a rough and rapid diligence process lasting 2 or 3 days to validate the assumptions above and the approach described below, and specifically the $200 reimbursement number (see below). Importantly, estimate the amount of unused COVID-19 testing capacity that currently exists in U.S. hospitals, but is not being used because of a shortage of kits/reagents and because of low reimbursement. This number could be very low, very high or anywhere in between. I suspect it is high to very high, but I’m not sure.
2. Increase CMS reimbursement per COVID-19 tests from about $40 to about $200. Explain to whomever is necessary to convince (CMS?…Congress?…) why this dramatic increase is necessary, i.e., to offset higher costs for reagents, etc. and to fund necessary improvements in testing infrastructure, facilities and personnel. Explain that this increase is necessary so hospital labs to ramp up testing, and not lose money in the process. Explain how $200 is similar to what some other countries are paying (e.g., Switzerland at $189)
3. Make this higher reimbursement temporary, but through June 30, 2020. Hopefully testing expands by then, and whatever parties bring on additional testing by then have recouped their fixed costs.
4. If necessary, justify the math, i.e., $200 per test, multiplied by roughly 1 or 2 million tests per day (roughly the target) x 75 days equals $15 to $30 billion, which is probably a bargain in the circumstances.
5. Work with the centralized labs (e.g., Quest, LabCorp., etc.), hospitals and healthcare clinics and manufactures of testing equipment and reagents (e.g., ThermoFisher, Roche, Abbott, etc.) to hopefully accelerate the testing process.
6. Try to get other payors (e.g., HMOs, PPOs, etc.) to follow CMS lead on reimbursement. This should not be difficult as other payors often follow CMS lead.
Just my $0.02.”
TC again: Here is a Politico article on why testing growth has been slow.
Will the U.S. economy re-open prematurely?:
New NBER survey of U.S. small companies nber.org/papers/w26989 Here is the percent, by industry, saying their business will still exist if the crisis lasts 6 months: All retail (except grocers): 33% Hotels: 27% Personal services: 22% Restaurants and bars: 15%
That is from Derek Thompson. Or when will the non-payment of mortgages render the banking system insolvent and beyond saving by the Fed?
At some point, irreversible, non-linear economic damage sets in, and we won’t let that happen, no matter how many times someone tells you “there is no trade-off between money and lives.”
For some time now I have thought that America will reopen prematurely, with a very partial and indeed hypocritical reopening, but a reopening nonetheless. In May, in most states but at varying speeds, including across cities.
You can see from this Chicago poll of top economists that virtually all of them oppose an early reopening. I don’t disagree with their analysis, but they are too far removed from the actual debate.
America is a democracy, and the median voter will not die of coronavirus (this sentence is not repeated enough times in most analyses). And so we will reopen pretty soon, no matter what the full calculus of lives and longer-run gdp might suggest.
Lyman Stone favors ending the lockdown. It does not matter whether you agree with him or not. Matt Parlmer predicts revolution if we don’t reopen in time. I don’t agree with that assessment, but he is thinking along the right lines by not regarding the reopening date as entirely a choice variable.
The key is to come up with a better reopening rather than a worse reopening.
Any model of optimal policy should be “what should we do now, knowing the lockdown can’t last very long?” rather than “what is the optimal length of lockdown?”
And our best hope is that the risk of an early reopening spurs America to become more innovative more quickly with masks, testing, and other methods of reducing viral and economic risk.