challenge trials

The public is fine with Human Challenge Trials

A vaccine for COVID-19 is urgently needed. Several vaccine trial designs may significantly accelerate vaccine testing and approval, but also increase risks to human subjects. Concerns about whether the public would see such designs as ethically acceptable represent an important roadblock to their implementation, and the World Health Organization has called for consulting the public regarding them. Here we present results from a pre-registered cross-national survey (n= 5; 920) of individuals in Australia, Canada, Hong Kong, New Zealand, South Africa, Singapore, the United Kingdom, and the United States. The survey asked respondents whether they would prefer scientists to conduct traditional trials or one of two accelerated designs: a challenge trial or a trial integrating a Phase II safety and immunogenicity trial into a larger Phase III efficacy trial. We find broad majorities prefer for scientists to conduct challenge trials (75%, 95% CI: 73-76%) and integrated trials (63%, 95% CI: 61-65%) over standard trials. Even as respondents acknowledged the risks, they perceived both accelerated trials as similarly ethical to standard trial designs, and large majorities characterized them as “probably” or “definitely ethical” (72%, 95% CI:70-73% for challenge trials; 77%, 95% CI 75-78% for integrated trials). This high support is consistent across every geography and demographic subgroup we examined, including people of diverging political orientations and vulnerable populations such as the elderly, essential workers, and racial and ethnic minorities. These findings bolster the case for these accelerated designs and can help assuage concerns that they would undermine public trust in vaccines.

Here is the paper by David Broockman, et.al.

Challenge Trials in Britain!

From the FT:

London is to host the world’s first Covid-19 human challenge trials — in which healthy volunteers are deliberately infected with coronavirus to assess the effectiveness of experimental vaccines. The UK government-funded studies are expected to begin in January at a secure quarantine facility in east London, according to several people involved in the project, which will be announced next week.

…The project’s academic leader is Imperial College London, and it will be run by hVivo, a spinout from Queen Mary University of London that was bought earlier this year by Open Orphan, a Dublin-based pharmaceutical research organisation.

…The petition organiser of 1Day Sooner in the UK is 18-year-old Alastair Fraser-Urquhart who is devoting his time to the campaign before going to University College London to study cancer biology next year.

All hail Alastair Fraser-Urquhart!

This part enraged me:

The NIH is also investigating the technical and ethical requirements for challenge trials. But Nadine Rouphael, a leading vaccine researcher at Emory University in Atlanta and one of several scientists who are keen to carry out challenge studies in the US, said: “There is no urgency at NIH. The UK is well ahead — and that’s great.”

No urgency!!! I raised challenge trials with the administration in April.

Addendum: Previous MR posts on challenge trials. And here is the UK petition and the Canadian petition from 1daysooner.

Oxford’s Jenner Institute to Prepare for Challenge Trials for COVID-19

I am one of the signatories to an open letter from 1DaySooner on challenge trials sent to Dr. Francis Collins at NIH. A major development announced with the letter is that 1Day Sooner and Oxford’s Jenner Institute are collaborating to prepare viral production for use in challenge trials.The Jenner Institute is the creator of the AstraZeneca produced vaccine, the vaccine farthest along in development.

A key goal of the letter is to encourage the NIH to start its own preparation for challenge trials:

The undersigned urge the U.S. government…its allies, international funders, and world bodies (e.g. the World Health Organization), to undertake immediate preparations for human challenge trials, including supporting safe and reliable production of the virus and any biocontainment facilities necessary to house participants.

Among the signatories are 15 Nobel prize winners including Oliver Hart and Al Roth, Molecular geneticist Mario Capecchi, professor of medicine William G. Kaelin and physician Barry Marshall (who knows a thing or two about volunteer trials.)

As I discussed earlier, since challenge trials restrict the volunteers to be young and healthy, you can’t apply their results directly to the sick and elderly (the external validity problem) but “challenge trials could help us whittle down [candidate vaccines]… to the best two to three, substantially speeding up the vaccine discovery process.” You could also use challenge trials to help figure out the right dosing which is unusually important in the current situation because if a vaccine can work with .5ml instead of 1ml that’s equivalent to doubling the available supply. The Director of the Jenner Institute, Adrian Hill, agrees writing:

We see considerable potential in the use of human challenge studies to accelerate COVID-19 vaccine development, down-select and help validate the best candidate vaccines, and optimise vaccination approaches.

You can read the whole letter here.

The NIH Should Run Human Challenge Trials for COVID

As I have been warning, social distancing measures are making it more difficult to test COVID vaccines even as the cost of COVID remains very high.

WashPost: The Oxford group earlier boasted that it had an 80 percent chance of developing an effective vaccine by September. Hill said the difficulty of testing the vaccine in Britain may mean there’s only a 50 percent chance of success within that time frame now.

The probability of an Oxford vaccine by September has fallen by 30 percentage points. Oxford isn’t the only vaccine and we may be able to find clinical trial candidates in Brazil and the United States where infections continue to occur. So let’s be generous and convert this into say a 10% increase in a one month’s delay of any vaccine. The world economy is losing $375 billion a month so this means we have lost an expected $37.5 billion. That number highlights why we should be willing to pay large sums to speed vaccines and it also indicates the immense value of human challenge trials.

More than 28,000 people have already volunteered to be part of a challenge trial and if we paid a few hundred volunteers a million dollars each it would be worthwhile (and would surely increase the number of volunteers).

The main impediment to human challenge trials appears to be skittish firms rather than bureaucratic governments which is why challenge trials should test multiple vaccines under the auspices of the NIH. The NIH umbrella can protect the firms and increase the efficiency of the trials.

Addendum: China is adopting a bold approach. We used to be bold. Apathy is killing us.

Vaccine Testing May Fail Without Human Challenge Trials

In Why Human Challenge Trials Will Be Necessary to Get a Coronavirus Vaccine I asked, “What if we develop a vaccine for COVID-19 but can’t find enough patients–healthy yet who might get sick–to run a randomized clinical trial?” Exactly that problem is now facing the Oxford vaccine in Britain.

An Oxford University vaccine trial has only a 50 per cent chance of success because coronavirus is fading so rapidly in Britain, a project co-leader has warned.

…Hill said that of 10,000 people recruited to test the vaccine in the coming weeks — some of whom will be given a placebo — he expected fewer than 50 people to catch the virus. If fewer than 20 test positive, then the results might be useless, he warned.

As I wrote, “A low infection rate is great, unless you want to properly test a vaccine.” Challenge trials have issues of external validity and they take time to setup properly but they produce results quickly and they can be especially useful in whittling down vaccine candidates to focus on the best candidates.

1DaySooner now has over 25 thousand volunteers from over 100 countries.

Human Challenge Trials

What if we develop a vaccine for COVID-19 but can’t find enough patients to run a randomized clinical trial? It sounds absurd, but this problem has happened in the past. Ebola was identified in 1976, and candidate vaccines were proven safe and effective in mice and primates in 2004 and 2005, respectively. But no human vaccine was produced [at that time] because it was extremely difficult, bordering on impossible, to trial an Ebola vaccine. The problem? Ebola is so deadly that people take precautionary measures long before a vaccine can be tested.

A few pieces have been written about human challenge trials, clinical trials in which healthy people are infected with a disease in order to see if a treatment or vaccine works, but most of them focus on the ethical issues. I don’t think there are serious ethical issues so writing at The National Interest I focus on why challenge trials are useful statistically and why they may even be necessary.

Even health care workers, however, have a low enough infection rate that you either need many months to determine if there is a significant effect, or you need large populations. In Italy, about 6,000 doctors were infected over two months, out of a population of about 241,000 Italian doctors. This is a monthly infection rate of 1.2 percent. If the vaccine is 50 percent effective, then to detect this within a month, you need a sample size of 7,776 people equally divided between a vaccinated group and a non-vaccinated group. You could run the test in a smaller sample of 1,322 but then the trial would take six months. A more effective vaccine would make detecting an effect easier, but flu vaccines work at 40 to 60 percent effectiveness, so an assumption of 50 percent is not unreasonable.

But will Italian doctors still be getting infected at a rate of 1.2 percent per month when a vaccine becomes available for trial in six months or a year? We hope not. The hope is that social distancing and the use of personal protective equipment will have greatly lowered the infection rate. A low infection rate is great, unless you want to properly test a vaccine.

…The virtue of a challenge trial is that the results would be available very quickly, within a few weeks, and using only a small population. If the vaccine is 50 percent effective, for example, then we would need around 100 volunteers or perhaps even fewer depending on how many people exposed to the virus in laboratory conditions contract the disease.

By advancing a vaccine by many months, a challenge trial could save many thousands of lives and spare the world the huge economic costs of the lockdowns and social distancing that we will be using to combat the virus.

Challenge trials, however, don’t solve all problems. In particular, to limit the risk we would want to restrict the patients in a challenge trial to be young and healthy. But that raises a problem of external validity. We also want the vaccine to be safe and effective in less healthy and elderly people which requires secondary challenge trials or field testing in that population. Nevertheless, as Athey, Kremer, Synder and myself argue in our NYTimes op-ed, the high risk of vaccine failure means that we would like 15-20 vaccine candidates and challenge trials could help us whittle this number down to the best two to three substantially speeding up the vaccine discovery process.

One more point is worth bearing in mind.

[A]n ordinary vaccine trial is not without risk—a vaccine could backfire and make the disease worse—so exposing fifty or so volunteers to the virus in a challenge trial must be balanced against exposing thousands to a potentially dangerous vaccine in an ordinary clinical trial.

Thus, the total risk may be lower with a combination of challenge trials and longer, larger field trials.

Challenge trials have a long history in medicine and their statistical advantages make them powerful and even necessary. As The Guardian notes:

Scientists, however, increasingly agree that such trials should be considered, and the WHO is the latest body to indicate conditional support for the idea.

“There’s this emerging consensus among everyone who has thought about this seriously,” said Prof Nir Eyal, the director of Rutgers University’s Center for Population-Level Bioethics in the US.

Human Challenge Trials for vaccines

From an anonymous reader:

As you are of course aware, testing on vaccines for Covid-19 are beginning to be undertaken. The scientific community has seemingly decided that Human Challenge Trials (HCT) where test subjects are directly exposed to the virus following vaccination are unethical, instead using the typical protocol of vaccine/placebo inoculation followed by months of observation in order to observe effectiveness. This seems to me a grave moral error based on the following argument.

1) There exists a large cohort of young, healthy, fully informed, willing participants who would undergo HCT.

2) Given the mortality profile of this disease, these participants would be undertaking an exceptionally small mortality risk (perhaps 5-10 per 100k, based on data from Spain/Italy/NYC, assuming zero vaccine effectiveness).

3) Society deems acceptable other activities with much higher fatality risk (at least 5-10x) in both professional (soldiers, logging workers) and recreational (motorcycling, mountaineering) capacities.

4) HCT would speed up the vaccine testing process by many months, saving tens of thousands of lives and avoiding enormous economic damage.

5) HCT actually poses significantly less risk to participants in terms of allergic reaction or ADE risk compared to a standard testing protocol since the number of participants could be much smaller and they would be medically observed.

I fail to find any ethical justification for the current stance of the medical community, from either a utilitarian or deontological perspective, and believe a highly consequential error is being made. This error may be based on false analogies to past unethical testing practices in history where participants were not informed or willing and danger was significant. The current case bears no ethical resemblance, in my judgement, to these past cases.

The simplest model of such errors is that many members of the biomedical establishment do not wish to have bad feelings about any “sins of commission” and to see their status lowered as a result of “dirty hands,” and the readily criticized logistics of Human Challenge Trials.  Since HCTs do not “feel right” to them, they self-deceive into associating that feeling with a concern for the greater public good.

You should not be surprised to see grave moral errors committed in a crisis, however.  Our “mainstream” protection against grave moral errors, in normal circumstances, simply is that usually we are not given the opportunity to commit them.

I do understand that a Human Challenge Trial does not necessarily suffice to show that a given vaccine is safe.  Nonetheless it should be in the “armor of our discourse,” so to speak, as a morally acceptable alternative.  So if you are a biomedical professional, or a public intellectual, I hope you will speak up.

Here is a Matt Yglesias piece on the urgency of developing a vaccine as quickly as possible.  Eric Weinstein notes that women risk their lives every time they proceed with having children.

Failing the Challenge

CNN says “In one word, this is why there likely won’t be a vaccine available before Election Day: biology.” Wrong. The one word is complacency. What CNN refers to as biology is the time it takes to run clinical trials.

Here’s how the trials work: You take 30,000 people, give half of them a vaccine and half of them a placebo, which is a shot of saline that does nothing. Then those 30,000 people go about their lives, and you wait to see how many in each group become infected and sick with Covid-19, the “endpoint” in medical parlance.

That waiting takes time, especially since the coronavirus vaccines currently being studied in the US are two-dose vaccines with each dose several weeks apart.

But it gets worse because trial volunteers are not random:

“Who’s in the trials – the kind of people who tend to stay at home or the kind of people who attended the Sturgis rally?” said John Moore, an immunologist at Weill Cornell Medicine, referring to a motorcycle rally in South Dakota that led to at least dozens of cases of Covid-19.

Historical precedent, as well as the demographics of the participants in the current coronavirus vaccine trials, suggest more the stay-at-home type.

That does not bode well for bringing the trials to a speedy conclusion.

Typically, those who volunteer for clinical trials tend to be “White, college-educated women,” said Frenck, who has been the principal investigator on dozens of vaccine clinical trials, and has served on the Data and Safety Monitoring Board for many others.

All three of those factors are potentially bad news for the coronavirus clinical trials, because data indicates White college-educated women are at lower risk for being exposed to the novel coronavirus.

None of this, however, is actually about biology. It’s about complacency. We could have run human challenge trials and paid for diverse volunteers but we decided that was too risky or too new or too radical or too something and so thousands of people die every week as we wait.

Addendum: Previous posts on challenge trials.

Peremptory Challenges

During the jury selection process, attorneys may request that a potential juror be stricken for cause, e.g. the juror is related to the defendant. Attorneys also have a limited number of peremptory challenges, typically between 3 and 20 depending on the state and the seriousness of the charges, which are essentially accepted without question. In Batson v. Kentucky the Supreme Court ruled that peremptory challenges may not be based solely on race but it’s widely acknowledged that Batson has no teeth because attorneys can easily come up with pretexts–which need not rise to the level of causes–to strike.

Next month the Supreme Court will revisit peremptory challenges and race. I don’t have strong opinions on the issue, although a small number of peremptory challenges seem fine to me, if only to keep the system moving and reduce the time and resources spent on jury selection. One reason I don’t have strong opinions is that I don’t think peremptory challenges are as biased as a NYTimes article seems to suggests.The NYTimes article, for example, never mentions that defendants also get peremptory challenges!  A second more subtle reason is that diversity of the jury pool constrains the jury even when there are no minorities on the jury. Here, from an earlier post, I comment on the findings of The Impact of Jury Race in Criminal Trials:

What the authors discover is that all white juries are 16% more likely to convict black defendants than white defendants but the presence of just a single black person in the jury pool equalizes conviction rates by race. The effect is large and remarkably it occurs even when the black person is not picked for the jury. The latter may not seem possible but the authors develop an elegant model of voir dire that shows how using up a veto on a black member of the pool shifts the characteristics of remaining pool members from which the lawyers must pick; that is, a diverse jury pool can make for a more “ideologically” balanced jury even when the jury is not racially balanced.

Thus, diversity of the jury pool may be as important as diversity of the jury–in a way that’s fortunate since it’s easier to make the jury pool diverse (as we have done with required randomization) than the jury. Instead of eliminating peremptory challenges, I’d raise their cost. For example, suppose that both sides get 3 “free” peremptory challenges but if they wanted one more they would have to give two to the opposing side.

Addendum: Justice Kavanaugh has written in favor of restricting peremptory challenges.

The Impact of Jury Race in Criminal Trials

In a great paper, The Impact of Jury Race in Criminal Trials, Shamena Anwar, Patrick Bayer and Randi Hjalmarsson exploit random variation in the jury pool to estimate the effect of race on criminal trials. The authors have data from nearly 800 trials in two Florida counties. On any given day, a jury pool is randomly drawn from a master list based on driver’s licenses. On some days, the pool of about 30 people contains some black members and on other days, purely for random reasons, it does not. The voir dire process–>For every $1 spent on legal aid, the savings can range from $1.60 to $30.removals, excuses and challenges–whittles down the jury pool to 6 jury members with typically 1 alternate.

The authors have data on the race, gender, and age of each member of the jury pool as well as each member of the ultimate jury. The authors also know the race and gender of the defendant and the charges. What the authors discover is that all white juries are 16% more likely to convict black defendants than white defendants but the presence of just a single black person in the jury pool equalizes conviction rates by race. The effect is large and remarkably it occurs even when the black person is not picked for the jury. The latter may not seem possible but the authors develop an elegant model of voir dire that shows how using up a veto on a black member of the pool shifts the characteristics of remaining pool members from which the lawyers must pick; that is, a diverse jury pool can make for a more “ideologically” balanced jury even when the jury is not racially balanced.

The author’s results show not only that blacks and whites are treated differently depending on the composition of the jury pool but also that random variation in the jury pool adds to the variability of sentences holding race constant. Like is not treated as like. The results also suggest that we don’t need racial quotas to increase fairness. We can increase fairness and reduce variability in a racially neutrally way by expanding the size of juries. Six-person juries have become common because they are cheap(er) but a return to twelve person juries would reduce the variability of sentences and greatly equalize conviction rates across race.

A Cost/Benefit Analysis of Clinical Trial Designs for COVID-19 Vaccine Candidates

I am very happy to see this new and urgently needed study.  They have heeded the stricture to show their work.  The authors are Donald A. Berry, Scott Berry, Peter Hale, Leah Isakov, Andrew W. Lo, Kien Wei Siah, and Chi Heem Wong, and here is the abstract:

We compare and contrast the expected duration and number of infections and deaths averted among several designs for clinical trials of COVID-19 vaccine candidates, including traditional randomized clinical trials and adaptive and human challenge trials. Using epidemiological models calibrated to the current pandemic, we simulate the time course of each clinical trial design for 504 unique combinations of parameters, allowing us to determine which trial design is most effective for a given scenario. A human challenge trial provides maximal net benefits—averting an additional 1.1M infections and 8,000 deaths in the U.S. compared to the next best clinical trial design—if its set-up time is short or the pandemic spreads slowly. In most of the other cases, an adaptive trial provides greater net benefits.

And what is an adapted trial you may be wondering?:

An adaptive version of the traditional vaccine efficacy RCT design (ARCT) is based on group sequential methods. Instead of a fixed study duration with a single final analysis at the end, we allow for early stopping for efficacy via periodic interim analyses of accumulating trial data…While this reduces the expected duration of the trial, we note that adaptive trials typically require more complex study protocols which can be operationally challenging to implement for test sites unfamiliar with this framework. In our simulations, we assume a maximum of six interim analyses spaced 30 days apart, with the first analysis performed when the first 10,000 subjects have been monitored for at least 30 days.

That means of course you might cut the trial short.  Kudos to the authors for producing one of the most important papers of this year.

New Emergent Ventures anti-Covid prize winners

The first new prize is to Anup Malani of the University of Chicago, with his team, for their serological research in India and Mumbia.  They showed rates of 57 percent seroprevalance in the Mumbai slums, a critical piece of information for future India policymaking.  Here is the research.

Professor Malani is now working in conjunction with Development Data Lab to extend the results by studying other parts of India.

The second new prize goes to 1Day Sooner, a 2020-initiated non-profit which has promoted the idea of Human Challenge Trials for vaccines and other biomedical treatments.  Alex here covers the pending HCTs in Britain, as well as providing links to previous MR coverage of the topic.

I am delighted to have them both as Emergent Ventures prize winners.

Here are the first, second, and third cohorts of winners of Emergent Ventures prizes against Covid-19.

Some doubts about medical ethics, and maybe that Russian vaccine is underrated

Most major questions in ethics are unsettled, though of course I have my own views, as do many other people.  I take that unsettledness as a fairly fundamental truth, I have been studying these matters for decades, and I even have several published articles in the top-ranked journal Ethics.

Now, if you take a whole group of people, give them medical licenses, teach them all more or less the same thing in graduate school, but not much other philosophy, and call it “medical ethics“…you have not actually gone much further.  Arguably you have retrogressed.

So when I hear people appeal to “medical ethics,” my intellectual warning bells go off.  To be sure, often I agree with those people, if only because I think contemporary American institutions often are not very flexible or able to execute effectively on innovations.  For instance, I didn’t think America could make a go at Robin Hanson’s variolation proposal, and so I opposed it.  “Medical ethics” seems to give the same instruction, though with less of a concrete institutional argument.

Still, the Lieutenant Colombo in me is bothered.  What about other nations?  Should we ever wish that they serve themselves up as medical ethics-violating guinea pigs, for the greater global good?

Medical ethics usually says no, or tries to avoid grappling with that question too directly.  But I wonder.

How about that Russian vaccine they will be trying in October?

To be clear, I won’t personally try it, and I don’t want the FDA to approve it for use in the United States.  But am I rooting for the Russians to try it this fall?  You betcha.  (Am I sure that is the correct ethical view?  No!  But I know the critics should not be sure either.)  I am happy to revise my views as further information comes in, but I see a good chance that  the attempt improves expected global welfare, and I think that is very often (but not always) a standard with strong and indeed decisive relevance.  And all the new results on cross-immunities imply that some pretty simple vaccines can have at least partial effectiveness.

Why exactly is “medical ethics” so sure this Russian vaccine is wrong other than that it violates “medical ethics”?  All relevant scenarios involve risk to millions of innocents, and I have not heard that Russians will be forced to take the vaccine.  The global benefits could be considerable, and I do note that the Russian vaccine scenario is the one that potentially spends down the reputational capital of various medical establishments.

Trying a not yet fully tested vaccine still seems wrong to many medical ethicists, even if the volunteers are compensated so they are better off in ex ante terms, as in some versions of Human Challenge Trials, an idea that (seemingly) has been elevated from “violating medical ethics” to a mere “problematic.”  Medical ethics claims priority over the ex ante Pareto principle, but I say we are back to the unsettled ethics questions on that one, but if anything with the truth leaning against medical ethics.

I find it especially strange when “medical ethics” is cited — often without further argumentation or explanation — on Twitter and other forms of social media as a kind of moral authority.  It then seems especially glaringly obvious that the moral consensus was never there in the first place, and that there is a gross and indeed now embarrassing unawareness of that underlying social fact.  It feels like citing Kant to the raccoon trying to claw through your roof.

I think medical ethics would not like this critique of medical ethics.  Yet I will be watching the Russian vaccine experiment closely.

Addendum: There is also biomedical ethics, but that would require a blog post of its own.  It is much more closely integrated with standard ethical philosophy, though it does not resolve any of the fundamental philosophical uncertainties.

AcceleratingHT

I’ve been working with Michael Kremer, Susan Athey, Chris Snyder and others to design incentives to speed vaccines and other health technologies. AcceleratingHT is our website and now features a detailed set of slides which explain the calculations behind our global plan. The global plan is similar in style to the US plan although on a larger scale. The key idea is that the global economy is losing $350 billion a month so speed pays. One way to speed a vaccine is to invest in capacity for 15-20 vaccine candidates before any candidates are approved, so that the moment a candidate is approved we can begin production (one can store doses in advance of approval). Most of the capacity will be wasted but that is a price worth paying. As Larry Summer says if you will die of starvation if you don’t get a pizza in two hours, order 5 pizzas. Human challenge trials are another way to speed the process.

A global plan is ideal since there are significant benefits to coordination. If each country invests in vaccines independently they will each choose the vaccine candidates most likely to succeed but that means all our eggs are a few baskets. There are over 100 vaccine candidates and they have different scientific and production risks so you want to choose the 15-20 which maximize the probability of success for the portfolio as a whole. To do that efficiently you need countries to agree that ‘I will invest in lots of capacity (more than I need) in candidate X if you invest in lots of capacity (more than you need) for candidate Y’, even knowing that the probability that X succeeds may be less than that of Y.

Vaccine nationalism is making a global plan look unlikely but if each country invests in multiple candidates around the world, as Operation Warp Speed is doing, and if each country guarantees to uphold contracts, we can reach a similar solution.

At AcceleratingHT you can also find our Incentive Design App which computes the optimal vaccine program given user chosen parameters. A big shout out to Juan Camilo Castillo, a newly graduated PhD student from Stanford, who put in a lot of heavy lifting on the app. We have been working on these models under time pressure and I will never forget the late night/early morning zoom calls where Michael Kremer would call out, “I think we need to take into account factor X. What effect would that have?” and Camilo would respond “Give me 5 minutes!” and, as we debated other factors Y and Z, Camilo would hack-away changing parameters and rewriting code till he had an answer. Hire a rising star while you can!

  • 1
  • 2