Category: Medicine

It is better to do lots of tests, even if they are not entirely accurate

We find that the number of daily tests carried out is much more important than their sensitivity, for the success of a case-isolation based strategy.

Our results are based on a Susceptible-Exposed-Infectious-Recovered (SEIR) model, which is age-, testing-, quarantine- and hospitalisation-aware. This model has a number of parameters which we estimate from best-available UK data. We run the model with variations of these parameters – each of which represents a possible present state of circumstances in the UK – in order to test the robustness of our conclusion.

We implemented and investigated a number of potential exit strategies, focusing primarily on the effects of virus-testing based case isolation.

The implementation of our model is flexible and extensively commented, allowing us and others to investigate new policy ideas in a timely manner; we next aim to investigate the optimal use of the highly imperfect antibody tests that the United Kingdom already possesses in large numbers.

There is much more at the link, including the model, results, and source code.  That is from a team led by Gergo Bohner and also Gaurav Venkataraman, Gaurav being a previous Emergent Ventures winner.

The essential Ben Thompson on isolation and quarantine

…while I have written about Taiwan’s use of cellphone-enforced quarantines for recent travelers and close contacts of those infected, I should also note that every single positive infection — symptomatic or not — is isolated away from their home and family. That is also the case in South Korea, and while it was the case for Singaporean citizens, it was not the case for migrant workers, which is a major reason why the virus has exploded in recent weeks.

Here’s the thing, though: isolating people is hard. It would be very controversial. It would require overbearing police powers that people in the West are intrinsically allergic to. Politicians that instituted such a policy would be very unpopular. It is so much easier to let tech companies build a potential magic bullet, and then demand they let government use it; most people wouldn’t know or wouldn’t care, which appears to matter more than whether or not the approach would actually work (or, to put it another way, it appears that the French government sees privacy as a club with which to beat tech companies, not a non-negotiable principle their citizens demand).

So that is why I have changed my mind: Western governments are not willing to take actions that we know work because it would be unpopular and controversial (indeed, the fact that central quarantine is so clearly a violation of liberties is arguably a benefit, because there is no way people would tolerate it once the crisis is over). And, on the flipside, that makes digital surveillance too dangerous to build. Politicians would rather leverage tech companies to violate liberty on the sly, and tech companies, once they have the capability, are all too willing to offload the responsibility of using it wisely to whatever government entity is willing to give them cover. There just isn’t much evidence that either side is willing to make hard choices.

That is from Ben’s Stratechery email newsletter, gated but you can pay to get it.  There is currently the risk that “test and trace” becomes for the Left what “chloroquine” has been for Trump and parts of the political right — namely a way to make otherwise unpalatable plans sound as if they have hope for more than “develop herd immunity and bankrupt the economy in the process.”

To be clear, I fully favor “test and trace,” and I’ve worked hard to help fund some of it.  That said, I wonder if we will anytime soon reach the point where it is a game changer.  So when people argue we should not reopen the economy until “test and trace” is in place, I increasingly see that as a kind of emotive declaration that others do not care enough about human lives (possibly true!), rather than an actual piece of advice.

Covid-19 liability reform for the eventual reopening

That is the topic of my latest Bloomberg column, here is one excerpt:

If an infected but asymptomatic worker shows up at work and sickens coworkers, for example, should the employer be liable? The answer is far from obvious. Liability exists not to shift unmanageable risk, but rather to induce management to take possible and prudent measures of precaution.

Another problem with liability law in this context is that the potential damages are high relative to the capitalization of most businesses. Covid-19 cases often pop up in chains; there have been many cases from a single conference, or in a single church choir, or on a single cruise ship. If a business or school is host to such a chain, it could be wiped out financially by lawsuits. In these cases the liability penalties do not have their intended deterrent effect because the money to lose simply isn’t there…

Another problem with liability in this setting has to do with jury expertise. Are random members of the public really the best people to determine acceptable levels of Covid-19 risk and appropriate employer precautions? Juries are better suited for more conventional applications of liability law, such as when the handyman fixing your roof falls off your rickety ladder. Given the unprecedented nature of the current situation, many Covid-19 risk questions require experts.

Finally, there is the issue of testing. Businesses could be of immeasurable help by testing their employees for Covid-19, as additional testing can help limit the spread of the virus (if only by indicating which workers should stay home or get treatment). Yet the available tests are highly imperfect, especially with false negatives. If businesses are liable for incorrect test results, and their possible practical implications, then business will likely not perform any tests at all, to the detriment of virtually everybody.

I recommend modest liability for some sectors, and zero liability, bundled with a New Zealand-like accident compensation system, for other sectors.  And of course some very dangerous sectors should not be allowed to reopen at all, though I am more sympathetic to regional experimentation than are some people on Twitter.

Nursing home estimates of the day

The Bali heterogeneity?

From a Kevin Kelly email to me:

Another weird data point on the highly heterogeneous nature of this virus.

A friend of mine who lives in Bali says there have been 2 confirmed Covid-19 deaths on their island of 4.3 million residents. Yet according to him:

That makes it around 25.000 tourists from mainland China every week.

And until mid-January 2020, before the outbreak of the Corona pandemic, there were 5 direct flights from Wuhan per week.

During January 2020, 113,000 tourists from China visited Bali. During December 2019 when the Coronavirus was already spreading the number of arrivals from China was even higher because December is very busy in Bali.

So during the months of December 2019 and January 2020, approximately 220,000 tourists arrived from China alone.

Here are the official Covid-19 numbers as of 17th April 2020.

Confirmed cases: 113 | Recovered: 32 | Deaths: 2

The Crematorium in Bali’s capital city Denpasar does not see any increase in the number of cremations.

The hospitals do not have a flood of patients. There is hardly any talk on Social Media by people reporting about folks falling ill with Corona like symptoms.

The only thing I could find in Social Media groups is that business owners in Bali have reported an unusually high number of employees falling ill during November and December 2019.

Here is his Medium post
https://medium.com/@dominik.vanyi/bali-covid19-an-involuntary-experiment-in-epidemiology-immunology-a62485ce5924

Why the low status of opposition to child abuse?

Michael Kaan emails me:

Hi Tyler, I’m a healthcare professional in Canada and a long-time reader of your blog. For the past couple of years, observing the culture wars and various elections, I’ve noticed that child abuse is an extremely rare topic among the cultural left: the highly visible progressive segment that drives wokeness, is culturally powerful, etc. You know what their dominant concerns are. (On the right it’s basically non-existent.)

While there’s nothing obviously wrong with their attention to sexual and racial discrimination, the energy put into it is disproportionate to the massive social cost of child abuse. Rates vary around the world, but in general it looks like about 30% of all children globally suffer some sort of serious maltreatment each year, often many times a year, repeated over multiple years.
So one can easily estimate that billions of people have experienced this. In other words, more people have been abused as children than have experienced war, famine, or epidemics.
The impact and costs of this have been measured (low academic achievement, health problems, low earnings, drug and alcohol use, etc.), and child abuse is sometimes lethal. What puzzles me is why it has no legs politically. Among the woke crowd, if child abuse is mentioned it’s usually in terms of discrimination against girls or sexual minorities. But there are really no prominent voices actively campaigning to mitigate child abuse generally.
Why is this? Is it overly complex? Is the phenomenon too widely dispersed demographically, so that an evil agent group isn’t easily identified? Does its persistence foreground chronic failures of the welfare state (if that’s the case)? Is it boring?

For a start, I would note that virtually everyone is again child abuse, so opposing it doesn’t make anyone significant look worse.  But I am sure there is much more to it than that.

Spit Works

A new paper finds that COVID-19 can be detected in saliva more accurately than with nasal swab. As I mentioned earlier a saliva test will lessen the need for personnel with PPE to collect samples.

Rapid and accurate SARS-CoV-2 diagnostic testing is essential for controlling the ongoing COVID-19 pandemic. The current gold standard for COVID-19 diagnosis is real-time RT-PCR detection of SARS-CoV-2 from nasopharyngeal swabs. Low sensitivity, exposure risks to healthcare workers, and global shortages of swabs and personal protective equipment, however, necessitate the validation of new diagnostic approaches. Saliva is a promising candidate for SARS-CoV-2 diagnostics because (1) collection is minimally invasive and can reliably be self-administered and (2) saliva has exhibited comparable sensitivity to nasopharyngeal swabs in detection of other respiratory pathogens, including endemic human coronaviruses, in previous studies. To validate the use of saliva for SARS-CoV-2 detection, we tested nasopharyngeal and saliva samples from confirmed COVID-19 patients and self-collected samples from healthcare workers on COVID-19 wards. When we compared SARS-CoV-2 detection from patient-matched nasopharyngeal and saliva samples, we found that saliva yielded greater detection sensitivity and consistency throughout the course of infection. Furthermore, we report less variability in self-sample collection of saliva. Taken together, our findings demonstrate that saliva is a viable and more sensitive alternative to nasopharyngeal swabs and could enable at-home self-administered sample collection for accurate large-scale SARS-CoV-2 testing.

The FDA has also just approved an at-home test collected by nasal swab, a saliva test should not be far behind.

Hat tip: Cat in the Hat.

Trading off consumption vs. covid-19 deaths

Somehow I missed this April 6 paper by Hall, Jones, and Klenow:

This short note develops a framework for thinking about the following question: What is the maximum amount of consumption that a utilitarian welfare function would be willing to trade off to avoid the deaths associated with COVID-19? Our baseline answer is 26%, or around 1/4 of one year’s consumption.

So what does that imply for optimal policy?  Will we manage to lose both?  Via Ivan Werning.

Claims about networking and social distancing

Our models demonstrate that while social distancing measures clearly do flatten the curve, strategic reduction of contact can strongly increase their efficiency, introducing the possibility of allowing some social contact while keeping risks low. Limiting interaction to a few repeated contacts emerges as the most effective strategy. Maintaining similarity across contacts and the strengthening of communities via triadic strategies are also highly effective. This approach provides empirical evidence which adds nuanced policy advice for effective social distancing that can mitigate adverse consequences of social isolation.

That is from a new paper by Per Block, et.al.  I do not consider this a confirmed result, but it is consistent with how my intuitions have been developing, and the success in containing Covid-19 on various smallish islands.

My Conversation with Philip Tetlock

Here is the audio and transcript, here is part of the summary:

He joined Tyler to discuss whether the world as a whole is becoming harder to predict, whether Goldman Sachs traders can beat forecasters, what inferences we can draw from analyzing the speech of politicians, the importance of interdisciplinary teams, the qualities he looks for in leaders, the reasons he’s skeptical machine learning will outcompete his research team, the year he thinks the ascent of the West became inevitable, how research on counterfactuals can be applied to modern debates, why people with second cultures tend to make better forecasters, how to become more fox-like, and more.

Here is one excerpt:

COWEN: If you could take just a bit of time away from your research and play in your own tournaments, are you as good as your own best superforecasters?

TETLOCK: I don’t think so. I don’t think I have the patience or the temperament for doing it. I did give it a try in the second year of the first set of forecasting tournaments back in 2012, and I monitored the aggregates. We had an aggregation algorithm that was performing very well at the time, and it was outperforming 99.8 percent of the forecasters from whom the composite was derived.

If I simply had predicted what the composite said at each point in time in that tournament, I would have been a super superforecaster. I would have been better than 99.8 percent of the superforecasters. So, even though I knew that it was unlikely that I could outperform the composite, I did research some questions where I thought the composite was excessively aggressive, and I tried to second guess it.

The net result of my efforts — instead of finishing in the top 0.02 percent or whatever, I think I finished in the middle of the superforecaster pack. That doesn’t mean I’m a superforecaster. It just means that when I tried to make a forecast better than the composite, I degraded the accuracy significantly.

COWEN: But what do you think is the kind of patience you’re lacking? Because if I look at your career, you’ve been working on these databases on this topic for what? Over 30 years. That’s incredible patience, right? More patience than most of your superforecasters have shown. Is there some dis-aggregated notion of patience where they have it and you don’t?

TETLOCK: [laughs] Yeah, they have a skill set. In the most recent tournaments, we’ve been working on with them, this becomes even more evident — their willingness to delve into the details of really pretty obscure problems for very minimal compensation is quite extraordinary. They are intrinsically cognitively motivated in a way that is quite remarkable. How am I different from that?

I guess I have a little bit of attention deficit disorder, and my attention tends to roam. I’ve not just worked on forecasting tournaments. I’ve been fairly persistent in pursuing this topic since the mid 1980s. Even before Gorbachev became general party secretary, I was doing a little bit of this. But I’ve been doing a lot of other things as well on the side. My attention tends to roam. I’m interested in taboo tradeoffs. I’m interested in accountability. There’re various things I’ve studied that don’t quite fall in this rubric.

COWEN: Doesn’t that make you more of a though? You know something about many different areas. I could ask you about antebellum American discourse before the Civil War, and you would know who had the smart arguments and who didn’t. Right?

And another:

TETLOCK:

…I had a very interesting correspondence with in the 1980s about forecasting tournaments. We could talk a little about it later. The upshot of this is that young people who are upwardly mobile see forecasting tournaments as an opportunity to rise. Old people like me and aging baby-boomer types who occupy relatively high status inside organizations see forecasting tournaments as a way to lose.

If I’m a senior analyst inside an intelligence agency, and say I’m on the National Intelligence Council, and I’m an expert on China and the go-to guy for the president on China, and some upstart R&D operation called IARPA says, “Hey, we’re going to run these forecasting tournaments in which we assess how well the analytic community can put probabilities on what Xi Jinping is going to do next.”

And I’ll be on a level playing field, competing against 25-year-olds, and I’m a 65-year-old, how am I likely to react to this proposal, to this new method of doing business? It doesn’t take a lot of empathy or bureaucratic imagination to suppose I’m going to try to nix this thing.

COWEN: Which nation’s government in the world do you think listens to you the most? You may not know, right?

Definitely recommended.

The Subways Seeded the Massive Coronavirus Epidemic in New York City

New York City’s multitentacled subway system was a major disseminator – if not the principal transmission vehicle – of coronavirus infection during the initial takeoff of the massive epidemic that became evident throughout the city during March 2020. The near shutoff of subway ridership in Manhattan – down by over 90 percent at the end of March – correlates strongly with the substantial increase in the doubling time of new cases in this borough. Maps of subway station turnstile entries, superimposed upon zip code-level maps of reported coronavirus incidence, are strongly consistent with subway-facilitated disease propagation. Local train lines appear to have a higher propensity to transmit infection than express lines. Reciprocal seeding of infection appears to be the best explanation for the emergence of a single hotspot in Midtown West in Manhattan. Bus hubs may have served as secondary transmission routes out to the periphery of the city.

That is from a new NBER working paper by Jeffrey E. Harris.

The Japanese coronavirus story

You may recall that some time ago MR posted an anonymous account of how the coronavirus problem actually was much worse in Japan than was being admitted by the Japanese government and broader establishment.  It is now clear that this Cassandra was correct.

I can now reveal to you the full story of that posting behind the first link, including my role in it.  Here is the opening excerpt:

By March 22nd, I strongly suspected there was a widespread coronavirus epidemic in Japan. This was not widely believed at the time. I, working with others, conducted an independent research project. By March 25th we had sufficient certainty to act. We projected that the default course of the epidemic would lead to a public health crisis.

We attempted to disseminate the results to appropriate parties, out of a sense of civic duty. We initially did this privately attached to our identities and publicly but anonymously to maximize the likelihood of being effective and minimize risks to the response effort and to the team. We were successful in accelerating the work of others.

The situation is, as of this writing, still very serious. In retrospect, our pre-registered results were largely correct. I am coming forward with them because the methods we used, and the fact that they arrived at a result correct enough to act upon prior to formal confirmation, may accelerate future work and future responses here and elsewhere.

I am an American. I speak Japanese and live in Tokyo. I have spent my entire adult life in Japan. I have no medical nor epidemiology background. My professional background is as a software engineer and entrepreneur. I presently work in technology. This project was on my own initiative and in my personal capacity.

I am honored to have played a modest role in this story, though full credit goes elsewhere, do read the whole thing.  Hashing plays a key role in the longer narrative.

The Roadmap to Pandemic Resilience

Led by Danielle Allen and Glen Weyl, the Safra Center for Ethics at Harvard has put out a Roadmap to Pandemic Resilience (I am a co-author along with others). It’s the most detailed plan I have yet seen on how to ramp up testing and combine with contact tracing and supported isolation to beat the virus.

One of the most useful parts of the roadmap is that choke points have been identified and solutions proposed. Three testing choke points, for example, are that nasal swaps make people sneeze which means that health care workers collecting the sample need PPE. A saliva test, such as the one just approved, could solve this problem. In addition, as I argued earlier, we need to permit home test kits especially as self-swab from near nasal appears to be just as accurate as nasal swabs taken by a nurse. Second, once collected, the swab material is classified as a bio-hazard which requires serious transport and storage safety requirements. A inactivation buffer, however, could kill the virus without killing the RNA necessary for testing and thus reduce the need for bio-safety techniques in transportation which would make testing faster and cheaper. Finally, labs are working on reducing the reagents needed for the tests.

Understanding the choke points is a big step towards increasing the quantity of tests.

From my email

One thing that some people fail to realize is the following:  This disease will have about the same fraction of population infected plus recovered In the post-lockdown equilibrium regardless of the policy path that gets us there. However, that does not mean that the number of dead is the same for all policies, because the infection fatality rate is so heterogenous with this disease.

The (often unrecognized) elephant in the room is that one set of policies may sort the least vulnerable population to be infected first while another set of policies may sort the most vulnerable population to be infected last.  Protecting the most vulnerable effectively while infecting the least vulnerable quickly could theoretically save almost everyone for this particular disease.

Since the old and sick people often live in relative “lockdown” even at normal times, the general lockdown does the opposite of beneficial sorting by slowing down infections among the least vulnerable.  The general lockdown kills more people over the whole epidemic by tilting the sorting in an unfavorable direction.

The hospital crowding is in my opinion a relatively unimportant issue compared to this because there is no effective “silver bullet” therapy for the disease.

That one is anonymous!  And from another reader:

A lot people are citing a paper that looks at the impact of general lockdowns on ultimate deaths (over 24 month window) during the 1918 Spanish flue epidemic in the US. It’s important to understand that the 1918 epidemic and 2020 epidemic have a sorting effects that go in the opposite direction.

The 1918 disease was most dangerous to people with strong immune systems (young adults), and those people were also the ones that were most active in society and had most interpersonal contacts. Absent any general lockdown, those people were infected first and didn’t benefit from the long-run equilibrium of “herd immunity.” The general lockdowns during the 1918 disease epidemic reduced these vulnerable people’s infection probability relatively more than that of the less vulnerable people.  This improved sorting and thereby saved lives.

The 2020 disease works in the opposite way. It is the most dangerous to old, sick people with weakest immune systems.  Those people are relatively inactive at normal times and don’t have a large number of social contacts. The general lockdown increases those vulnerable people’s relative infection probability, because their routine doesn’t change much while less vulnerable people social distance. This adverse sorting due to general lockdowns causes more deaths, in theory at least.

In my opinion, the 1918 lockdown evidence should be interpreted as evidence of the importance of sorting, not evidence that general lockdowns are the right thing to do now.

What should we believe and not believe about R?

This is from my email, highly recommended, and I will not apply further indentation:

“Although there’s a lot of pre-peer-reviewed and strongly-incorrect work out there, I’ll single out Kevin Systrom’s rt.live as being deeply problematic. Estimating R from noisy real-world data when you don’t know the underlying model is fundamentally difficult, but a minimal baseline capability is to get sign(R-1) right (at least when |R-1| isn’t small), and rt.live is going to often be badly (and confidently) wrong about that because it fails to account for how the confirmed count data it’s based on is noisy enough to be mostly garbage. (Many serious modelers have given up on case counts and just model death counts.) For an obvious example, consider their graph for WA: it’s deeply implausible on its face that WA had R=.24 on 10 April and R=1.4 on 17 April. (In an epidemiological model with fixed waiting times, the implication would be that infectious people started interacting with non-infectious people five times as often over the course of a week with no policy changes.) Digging into the data and the math, you can see that a few days of falling case counts will make the system confident of a very low R, and a few days of rising counts will make it confident of a very high one, but we know from other sources that both can and do happen due to changes in test and test processing availability. (There are additional serious methodological problems with rt.live, but trying to nowcast R from observed case counts is already garbage-in so will be garbage-out.)

However, folks are (understandably, given the difficulty and the rush) missing a lot of harder stuff too. You linked a study and wrote “Good and extensive west coast Kaiser data set, and further evidence that R doesn’t fall nearly as much as you might wish for.” We read the study tonight, and the data set seems great and important, but we don’t buy the claims about R at all — we think there are major statistical issues. (I could go into it if you want, although it’s fairly subtle, and of course there’s some chance that *we’re* wrong…)

Ultimately, the models and statistics in the field aren’t designed to handle rapidly changing R, and everything is made much worse by the massive inconsistencies in the observed data. R itself is a surprisingly subtle concept (especially in changing systems): for instance, rt.live uses a simple relationship between R and the observed rate of growth, but their claimed relationship only holds for the simplest SIR model (not epidemiologically plausible at all for COVID-19), and it has as an input the median serial interval, which is also substantially uncertain for COVID-19 (they treat it as a known constant). These things make it easy to badly missestimate R. Usually these errors pull or push R away from 1 — rt.live would at least get sign(R – 1) right if their data weren’t garbage and they fixed other statistical problems — but of course getting sign(R – 1) right is a low bar, it’s just figuring out whether what you’re observing is growing or shrinking. Many folks would actually be better off not trying to forecast R and just looking carefully at whether they believe the thing they’re observing is growing or shrinking and how quickly.

All that said, the growing (not total, but mostly shared) consensus among both folks I’ve talked to inside Google and with academic epidemiologists who are thinking hard about this is:

  • Lockdowns, including Western-style lockdowns, very likely drive R substantially below 1 (say .7 or lower), even without perfect compliance. Best evidence is the daily death graphs from Italy, Spain, and probably France (their data’s a mess): those were some non-perfect lockdowns (compared to China), and you see a clear peak followed by a clear decline after basically one time constant (people who died at peak were getting infected right around the lockdown). If R was > 1 you’d see exponential growth up to herd immunity, if R was 0.9 you’d see a much bigger and later peak (there’s a lot of momentum in these systems). This is good news if true (and we think it’s probably true), since it means there’s at least some room to relax policy while keeping things under control. Another implication is the “first wave” is going to end over the next month-ish, as IHME and UTexas (my preferred public deaths forecaster; they don’t do R) predict.
  • Cases are of course massively undercounted, but the weight of evidence is that they’re *probably* not *so* massively undercounted that we’re anywhere near herd immunity (though this would of course be great news). Looking at Iceland, Diamond Princess, the other studies, the flaws in the Stanford study, we’re very likely still at < ~2-3% infected in the US. (25% in large parts of NYC wouldn’t be a shock though).

Anyways, I guess my single biggest point is that if you see a result that says something about R, there’s a very good chance it’s just mathematically broken or observationally broken and isn’t actually saying that thing at all.”

That is all from Rif A. Saurous, Research Director at Google, currently working on COVID-19 modeling.

Currently it seems to me that those are the smartest and best informed views “out there,” so at least for now they are my views too.