No Learning Without Risk
Here’s something from a paper that I am working on. The context is why first doses first makes more sense the greater the uncertainty but the point made is larger. No indent.
An important feature of First Doses First (FDF) and other policies such as fractional dosing is that they are reversible. In other words, FDF contains an option to switch back to Second Doses First (SDF). Options increase in value with uncertainty (Dixit and Pindyck 1994). Thus, contrary to many people’s intuitions, the greater the uncertainty the greater the value of moving to First Doses First. Indeed, the value of the option can be so high that one might want to move to First Doses First even if it were worse in expectation. For example, if the expected efficacy of the first dose were just 45% then in expectation it would be worse than Second Doses First (95% efficacy) but if there were lots uncertainty around the 45% expected efficacy it might still be better to switch to First Doses First. If there was a 75% chance that the efficacy of the first dose was 30%, for example, and a 25% chance that it was 90% (.75*.3+.25*.90=45%) then under reversibility one would still want to switch to First Doses First to learn whether the true efficacy was 30% or 90%.*
Put differently shifting away from the default strategy to an alternative such as FDF or fractional dosing might be considered to be “risky”. But in this context, learning requires risk. When learning is desirable, it is also desirable to take on risk. Risk aversion can prevent learning and thus can be dangerous.
If FDF is worse in expectation than SDF then it would be optimal to switch to the most minimal form of FDF necessary to learn about the true efficacy rate. In other words, to run an experiment. If FDF is superior in expectation to SDF then it might also be better to run an experiment before switching but not necessarily. If FDF is superior in expectation to SDF then the cost of running the experiment is keeping the policy with lower expected value while the experiment is running. If these costs are high then switching immediately is better.
It would take at least 16 weeks, for example, to run an experiment on extending dosing from 3 weeks to 12 weeks (including, optimistically, just 1 week to setup the experiment). As of early January 2021, confirmed cases in the United States are increasing at the rate of 200,000 per day or 1,400,000 per week. Thus there could be 22,400,000 new confirmed cases in the time it takes to run the experiment. At a case fatality rate of 1.7% that means 380,800 new deaths. If First Doses First reduces the infection rate in expectation by 10% that would imply that running the experiment has an expected cost of 38,080 lives.
At these rates, more lives could be saved in expectation by switching to the policy with higher expected value and simultaneously running experiments. Randomized trials that explicitly test the impact of dosing timing, fractional dosing and different timings of additional doses on severe, symptomatic and asymptomatic infections, and also on transmission should be incorporated as part of roll-out plans (Kominers and Tabarrok 2020, Bach 2021). However, roll-out of modified plans should not wait until these trial results are known; instead, plans should be adjusted as new information emerges. Most notably the British moved to First Doses First and they approved the AstaZeneca vaccine on December 30, 2020 and the consequences of both of these decisions should be monitored very closely to help improve decisions in other countries.
*This assumes that one could learn the true efficacy rate quickly enough relative to the ongoing pandemic to benefit from the new information. One might respond that in principle SDF also contains an option to switch to FDF but this option is valueless since Second Doses First provides no opportunity to learn. Only under First Doses First do we learn valuable new information.