Results for “invisible graveyard” 11 found
The Invisible Graveyard is Invisible No More
We called it the invisible graveyard, the place they buried people killed by FDA delay. Back then only a few of us–mostly libertarians long practiced in seeing the invisible hand–could see the invisible graveyard. Normal people looked at us oddly and quickly ran away when we frantically pointed to the dead. “There! there! Can’t you see the bodies?” Now, however, the veil has been lifted and even normal people see.
Here is Ezra Klein writing in the NYTimes:
The problem here is the Food and Drug Administration. They have been disastrously slow in approving these tests and have held them to a standard more appropriate to doctor’s offices than home testing. “The F.D.A. needs to catch up to the science,” Mina said, frustration evident in his voice. “They are inadvertently killing people by not following the science.” On my podcast, I asked Vivek Murthy, President Biden’s nominee for surgeon general, whether the F.D.A. had been too cautious. “I do think we’ve been too conservative,” he told me. Murthy went on to argue that there’s a difference between the diagnostic testing doctors do and the surveillance testing the public could do and that the F.D.A. had failed to appreciate the difference. Speeding the F.D.A. on this issue will be an early, and crucial, test for the Biden administration. In this case, Democrats need to deregulate.
Even back in December when I was tweeting from the rooftops things like:
Your daily reminder that 14,696 people have died from COVID in the United States since Pfizer applied for an EUA from the FDA.
people argued that I was exaggerating the simple math of FDA delay. Today, however, the reality of deadly delay is almost conventional wisdom. Here’s Klein again:
The new strains spread quickly. The speed of our countermeasures will decide our fate. What feel like reasonable delays in our normal experience of time — a few weeks here for Congress to debate a bill, a few weeks there for the F.D.A. to hold meetings — could lead to the kind of explosive infections that overwhelm our hospitals and fill our morgues.
Peltzman Revisited
Casey Mulligan has an excellent new paper, Peltzman Revisited: Quantifying 21st-Century Opportunity Costs of Food and Drug Administration Regulation. What are the costs of delaying a new drug or a vaccine? Longer and bigger clinical trials increase safety but I’ve often made the point that the people who would have lived had a good drug been approved sooner are buried in an invisible graveyard and thus these costs are typically undercounted–the failure to see the invisible graveyard biases decisions in favor of delay. Mulligan makes a different and rarely considered point about substitution effects. If a vaccine isn’t available there are substitutes but these substitutes are themselves potentially unsafe and ineffective. But who is testing the substitures?
Many of these substitute interventions, such as remote work, closing schools, and canceling normal medical appointments, are beyond the jurisdiction of the FDA and can be utilized without any attempt to demonstrate their safety or efficacy.
If the substitutes work, the costs of delay are reduced. The FDA, for example, is right to prioritize drugs for which there are few alternative treatments. But the standards for many vaccine or drug substitutes are completely different than those used to approve a vaccine:
Closing schools to in-person learning is an important example of a prevention activity that was available, was applied to tens of millions of children in the United States, and was outside the FDA’s jurisdiction…Obviously the FDA’s effectiveness standard for vaccines differs from the effectiveness standard (if any) that school districts applied in deciding to close schools.
Where were the randomized controlled trials for closing schools, shutting the parks and beaches, and delaying medical appointments? Thus, it’s quite possible that greater safety of vaccines comes at the expense of greater time under less safe and possibly unsafe substitutes. As Mulligan concludes:
Approval delays for pandemic tests and vaccines pushed tens of millions of individuals and businesses into preventions and treatments that were both outside FDA jurisdiction and hardly safe or effective. The pandemic experience raises the question of whether, on the whole, consumers engage in more unsafe and ineffective practices than they would if FDA approval were not a prerequisite for pharmaceutical sales.
Addendum: Much else of interest in the paper including a calculation of the value of the vaccines in the hundreds of billions and trillions very much in line with work done by the AHT team, including myself ,in the AER PP (especially the appendix) and Science.
Yglesias on Operation Warp Speed and the Republicans
Here’s Yglesias on Operation Warp Speed and the Republicans:
The debate over Operation Warp Speed wasn’t just a one-off policy dispute. Long before the pandemic, there was a conservative critique that the Food and Drug Administration is too slow and too risk-averse when it comes to authorizing new medications. Alex Tabarrok, a George Mason University economist, wrote about the “invisible graveyard” that could have been avoided if the FDA took expected value more seriously and considered the cost of delay in its authorization decisions.
The pandemic experience validated this criticism, which came to be embraced by some on the left as well — and it was about more than just vaccines. When it came to home Covid tests, Ezra Klein noted in the New York Times in 2021, “the problem here is the Food and Drug Administration. They have been disastrously slow in approving these tests and have held them to a standard more appropriate to doctor’s offices than home testing.”
And yet, just as the invisible graveyard was becoming seen and the debate was being won and just as a historical public-private partnership had sped vaccines to the public and saved millions, the Republicans abandoned the high ground:
…it’s not surprising that Democrats are comfortable with the bureaucratic status quo and hesitant to ruffle feathers at federal regulatory agencies. What’s shocking is that Republicans — the traditional party of deregulation, the party that argued for years that the FDA is too slow-footed, the party that saved untold lives by accelerating vaccine development under Trump — have abandoned these positions.
At the cusp of what should have been a huge policy victory, Republicans don’t brag about their success, and they have no FDA reform legislation to offer. Instead, they’ve taken up the old mantle of hard-left skepticism of modern science and the pharmaceutical industry.
It’s been painful to see all that has been gained now being lost. Libertarian economists and conservatives argued for decades that the FDA worried more about approving a drug that later turns out to be unsafe than about failing to approve a drug that could save lives; thus producing a deadly caution. But now the FDA is being attacked for what they did right, quickly approving safe vaccines. I hope that he is wrong but I fear that Yglesias is correct that the FDA may now get even slower and more cautious.
The irony of the present moment is that there is substantial backlash to the FDA’s approval of vaccines that haven’t turned out to be dangerous at all.
That’s only going to make regulators even more cautious. Right now the entire US regulatory state is taking essentially no heat for the slow progress on the next generation of vaccines, and an enormous amount of heat for the perfectly safe vaccines that it already approved. And the ex-president who pushed them to speed up their work on those vaccines is not only no longer defending them, he’s embarrassed to have ever been associated with the project.
Like I said, it’s a comical moment of Republican infighting. But it’s a very grim one for anyone concerned with the pace of scientific progress in America.
*The Messenger*
The author is Peter Loftus, and the subtitle is Moderna, the Vaccine, and the Business Gamble That Changed the World. An excellent book, here is one very short excerpt:
The FDA usually follows a rigid process of interacting with the drug companies it regulates. Normally, it can take months for a company to schedule an in-person meeting with the FDA.
Culture dies hard, here is Alex on the Invisible Graveyard. And this:
…Moderna executives expressed confidence they could hit the enrollment targets without significantly slowing down overall enrollment. But Fauci and Slaoui said they actually wanted Moderna to slow down overall enrollment in order to ensure they enrolled more minorities.
The book estimates the delay here at three weeks — how many lives was that in winter of 2020/2021?
The American Academy of Pediatrics Tells the FDA to Speed Up and Stop Endangering Patients
The American Academy of Pediatrics has written a stunning letter to the FDA:
We understand that the FDA has recently worked with Pfizer and Moderna to double the number of children ages 5-11 years included in clinical trials of their COVID-19 vaccines. While we appreciate this prudent step to gather more safety data, we urge FDA to carefully consider the impact of this decision on the timeline for authorizing a vaccine for this age group. In our view, the rise of the Delta variant changes the risk-benefit analysis for authorizing vaccines in children. The FDA should strongly consider authorizing these vaccines for children ages 5-11 years based on data from the initial enrolled cohort, which are already available, while continuing to follow safety data from the expanded cohort in the post-market setting. This approach would not slow down the time to authorization of these critically needed vaccines in the 5–11-year age group.
In addition, as FDA continues to evaluate clinical trial requirements for children under 5 years, we similarly urge FDA to carefully consider the impact of its regulatory decisions on further delays in the availability of vaccines for this age group. Based on scientific data currently available on COVID-19 vaccines, as well as on 70 years of vaccinology knowledge in the pediatric population, the Academy believes that clinical trials in these children can be safely conducted with a 2-month safety follow-up for participants. Assuming that the 2-month safety data does not raise any new safety concerns and that immunogenicity data are supportive of use, we believe that this is sufficient for authorization in this and any other age group. Waiting on a 6-month follow-up will significantly hinder the ability to reduce the spread of the hyper infectious COVID-19 Delta variant among this age group, since it would add 4 additional months before an authorization decision can be considered. Based on the evidence from the over 340 million doses of COVID-19 doses administered to adults and adolescents aged 12-17,as well as among adults 18 and older, there is no biological plausibility for serious adverse immunological or inflammatory events to occur more than two months after COVID-19 vaccine administration.
In my many years of writing about the FDA, I can’t recall a single instance in which a major medical organization told the FDA to use a smaller trial and speed up the process because FDA delay was endangering the safety of their patients. Wow.
The invisible graveyard is invisible no more.
News You Can Use
1. Popular wisdom says the second cheapest bottle of wine has the biggest markup because no one wants to look like a cheapskate and buy the cheapest bottle and restaurants take advantage of that tendency to markup the second cheapest bottle relatively more so it’s a poor value. Popular wisdom is wrong according to an analysis of wines at London restaurants. Peak markup is near the middle. Buying by the glass is also ok.
2. How to cook cicadas. An entire cookbook, albeit a short one. No word on what wine pairs well with cicada.
3. The Invisible Graveyard: A Conversation with Economist Alex Tabarrok a good podcast with with physicians in training Mitch and Daniel Belkin. You may also be interested in Metformin and the Biology of Aging with Nir Barzilai.
4. 1729–earn Bitcoin for completing tasks & tutorials. Balaji Srinivasan’s newsletter that pays readers. 1729, a very interesting number and an effort to bootstrap the community and technology to eventually deliver online countries. Here’s a good winning entry, a review of Balaji’s essay Founding vs. Inheriting.
Will Trump Appoint a Great FDA Commissioner?
As someone who has written about FDA reform for many years it’s gratifying that all of the people whose names have been floated for FDA Commissioner would be excellent, including Balaji Srinivasan, Jim O’Neill, Joseph Gulfo, and Scott Gottlieb. Each of these candidates understands two important facts about the FDA. First, that there is fundamental tradeoff–longer and larger clinical trials mean that the drugs that are approved are safer but at the price of increased drug lag and drug loss. Unsafe drugs create concrete deaths and palpable fear but drug lag and drug loss fill invisible graveyards. We need an FDA commissioner who sees the invisible graveyard.
Each of the leading candidates also understands that we are entering a new world of personalized medicine that will require changes in how the FDA approves medical devices and drugs. Today almost everyone carries in their pocket the processing power of a 1990s supercomputer. Smartphones equipped with sensors can monitor blood pressure, perform ECGs and even analyze DNA. Other devices being developed or available include contact lens that can track glucose levels and eye pressure, devices for monitoring and analyzing gait in real time and head bands that monitor and even adjust your brain waves.
The FDA has an inconsistent even schizophrenic attitude towards these new devices—some have been approved and yet at the same time the FDA has banned 23andMe and other direct-to-consumer genetic testing companies from offering some DNA tests because of “the risk that a test result may be used by a patient to self-manage”. To be sure, the FDA and other agencies have a role in ensuring that a device or test does what it says it does (the Theranos debacle shows the utility of that oversight). But the FDA should not be limiting the information that patients may discover about their own bodies or the advice that may be given based on that information. Interference of this kind violates the first amendment and the long-standing doctrine that the FDA does not control the practice of medicine.
Srinivisan is a computer scientist and electrical engineer who has also published in the New England Journal of Medicine, Nature Biotechnology, and Nature Reviews Genetics. He’s a co-founder of Counsyl, a genetic testing firm that now tests ~4% of all US births, so he understands the importance of the new world of personalized medicine.
The world of personalized medicine also impacts how new drugs and devices should be evaluated. The more we look at people and diseases the more we learn that both are radically heterogeneous. In the past, patients have been classified and drugs prescribed according to a handful of phenomenological characteristics such as age and gender and occasionally race or ethnic background. Today, however, genetic testing and on-the-fly examination of RNA transcripts, proteins, antibodies and metabolites can provide a more precise guide to the effect of pharmaceuticals in a particular person at a particular time.
Greater targeting is beneficial but as Peter Huber has emphasized it means that drug development becomes much less a question of does this drug work for the average patient and much more about, can we identify in this large group of people the subset who will benefit from the drug? If we stick to standard methods that means even larger and more expensive clinical trials and more drug lag and drug delay. Instead, personalized medicine suggests that we allow for more liberal approval decisions and improve our techniques for monitoring individual patients so that physicians can adjust prescribing in response to the body’s reaction. Give physicians a larger armory and let them decide which weapon is best for the task.
I also agree with Joseph Gulfo (writing with Briggeman and Roberts) that in an effort to be scientific the FDA has sometimes fallen victim to the fatal conceit. In particular, the ultimate goal of medical knowledge is increased life expectancy (and reducing morbidity) but that doesn’t mean that every drug should be evaluated on this basis. If a drug or device is safe and it shows activity against the disease as measured by symptoms, surrogate endpoints, biomarkers and so forth then it ought to be approved. It often happens, for example, that no single drug is a silver bullet but that combination therapies work well. But you don’t really discover combination therapies in FDA approved clinical trials–this requires the discovery process of medical practice. This is why Vincent DeVita, former director of the National Cancer Institute, writes in his excellent book, The Death of Cancer:
When you combine multidrug resistance and the Norton-Simon effect , the deck is stacked against any new drug. If the crude end point we look for is survival, it is not surprising that many new drugs seem ineffective. We need new ways to test new drugs in cancer patients, ways that allow testing at earlier stages of disease….
DeVita is correct. One of the reasons we see lots of trials for end-stage cancer, for example, is that you don’t have to wait long to count the dead. But no drug has ever been approved to prevent lung cancer (and only six have ever been approved to prevent any cancer) because the costs of running a clinical trial for long enough to count the dead are just too high to justify the expense. Preventing cancer would be better than trying to deal with it when it’s ravaging a body but we won’t get prevention trials without changing our standards of evaluation.
Jim O’Neill, managing director at Mithril Capital Management and a former HHS official, is an interesting candidate precisely because he also has an interest in regenerative medicine. With a greater understanding of how the body works we should be able to improve health and avoid disease rather than just treating disease but this will require new ways of thinking about drugs and evaluating them. A new and non-traditional head of the FDA could be just the thing to bring about the necessary change in mindset.
In addition, to these big ticket items there’s also a lot of simple changes that could be made at the FDA. Scott Alexander at Slate Star Codex has a superb post discussing reciprocity with Europe and Canada so we can get (at the very least) decent sunscreen and medicine for traveler’s diarrhea. Also, allowing any major pharmaceutical firm to produce any generic drug without going through a expensive approval process would be a relatively simply change that would shut down people like Martin Shkreli who exploit the regulatory morass for private gain.
The head of the FDA has tremendous power, literally the power of life and death. It’s exciting that we may get a new head of the FDA who understands both the peril and the promise of the position.
Making American Great Again–The FDA
Does Donald Trump want to streamline the FDA and speed new drugs to patients? The Washington Post thinks that it can read the tea leaves:
A single sentence in President-elect Donald Trump’s health-care platform sends a strong hint to the drug and medical device industry that they may have an easier time getting their products on the market under his administration.
“Reform the Food and Drug Administration, to put greater focus on the need of patients for new and innovative medical products,” his health plan states.
On the face of it, the bullet point may seem almost bland, but efforts to integrate patients’ preferences and encourage innovation often result in proposals aimed at speeding up the process for getting new medicines on the market by easing regulations. Critics argue that such efforts can erode standards that are in place to protect patients from drugs that don’t work and might even be harmful.
“The language … is industry code for deregulation and reducing of safety standards,” said Robert Weissman, president of Public Citizen, a consumer watchdog.
There is plenty of evidence that the FDA is too slow (see, for example, here, here, here and here) so I would support such a move. Senators Cruz and Lee proposed a reciprocity bill last term under which drugs approved in other developed countries would quickly be approved here; perhaps such a bill could find renewed interest in a Trump administration (Economists also support the idea of reciprocity.)
On the other hand, Trump has expressed support for Medicare being allowed to negotiate drug prices which is tantamount to price controls given the size of Medicare and that is potentially a disaster. Price controls could significantly reduce research and development in the pharmaceutical industry and end up greatly adding to the invisible graveyard. Trump’s advisers would seem to lean towards streamlining the FDA process rather than imposing price controls but it’s difficult to be certain.
Senators Cruz and Lee Introduce Reciprocity Bill
Senators Ted Cruz (R-Texas) and Mike Lee (R-Utah) have just introduced a bill that would implement an idea that I have long championed, making drugs, devices and biologics that are approved in other developed countries also approved for sale in the United States. Highlights of the “Reciprocity Ensures Streamlined Use of Lifesaving Treatments Act (S. 2388), or the RESULT Act,” include:
- Amending the Food, Drug and Cosmetic Act to allow for reciprocal approval of drugs, devices and biologics from foreign sponsors in certain trusted, developed countries including EU member countries, Israel, Australia, Canada and Japan.
- Encouraging the FDA to expeditiously review life-saving drug and device applications, this legislation would provide the FDA with a 30-day window to approve or deny a sponsor’s application….
- The HHS Secretary is instructed to approve a drug, device or biologic if the FDA confirms the product is:
- Lawfully approved for sale in one of the listed countries;
- Not a banned device by current FDA standards;
- There is a public health or unmet medical need for the product.
- If a promising application for a life-saving drug is declined Congress is granted the authority to disapprove of a denied application and override an FDA decision with a majority vote via a joint resolution.
In explaining why he introduced the bill Senator Cruz argued:
We continue to lose far too many of our loved ones to the “invisible graveyard,” as economist Alex Tabarrok has described: lives that could have been saved but for a bureaucratic barrier that rejects medical cures and innovation…The bill I am introducing takes the first step to reverse this trend. It provides for reciprocal drug approval, so that cures and medical devices that are already approved in other countries can more expeditiously come to the U.S.
Is the FDA Too Conservative or Too Aggressive?
I have long argued that the FDA has an incentive to delay the introduction of new drugs because approving a bad drug (Type I error) has more severe consequences for the FDA than does failing to approve a good drug (Type II error). In the former case at least some victims are identifiable and the New York Times writes stories about them and how they died because the FDA failed. In the latter case, when the FDA fails to approve a good drug, people die but the bodies are buried in an invisible graveyard.
In an excellent new paper (SSRN also here) Vahid Montazerhodjat and Andrew Lo use a Bayesian analysis to model the optimal tradeoff in clinical trials between sample size, Type I and Type II error. Failing to approve a good drug is more costly, for example, the more severe the disease. Thus, for a very serious disease, we might be willing to accept a greater Type I error in return for a lower Type II error. The number of people with the disease also matters. Holding severity constant, for example, the more people with the disease the more you want to increase sample size to reduce Type I error. All of these variables interact.
In an innovation the authors use the U.S. Burden of Disease Study to find the number of deaths and the disability severity caused by each major disease. Using this data they estimate the costs of failing to approve a good drug. Similarly, using data on the costs of adverse medical treatment they estimate the cost of approving a bad drug.
Putting all this together the authors find that the FDA is often dramatically too conservative:
…we show that the current standards of drug-approval are weighted more on avoiding a Type I error (approving ineffective therapies) rather than a Type II error (rejecting effective therapies). For example, the standard Type I error of 2.5% is too conservative for clinical trials of therapies for pancreatic cancer—a disease with a 5-year survival rate of 1% for stage IV patients (American Cancer Society estimate, last updated 3 February 2013). The BDA-optimal size for these clinical trials is 27.9%, reflecting the fact that, for these desperate patients, the cost of trying an ineffective drug is considerably less than the cost of not trying an effective one.
(The authors also find that the FDA is occasionally a little too aggressive but these errors are much smaller, for example, the authors find that for prostate cancer therapies the optimal significance level is 1.2% compared to a standard rule of 2.5%.)
The result is important especially because in a number of respects, Montazerhodjat and Lo underestimate the costs of FDA conservatism. Most importantly, the authors are optimizing at the clinical trial stage assuming that the supply of drugs available to be tested is fixed. Larger trials, however, are more expensive and the greater the expense of FDA trials the fewer new drugs will be developed. Thus, a conservative FDA reduces the flow of new drugs to be tested. In a sense, failing to approve a good drug has two costs, the opportunity cost of lives that could have been saved and the cost of reducing the incentive to invest in R&D. In contrast, approving a bad drug while still an error at least has the advantage of helping to incentivize R&D (similarly, a subsidy to R&D incentivizes R&D in a sense mostly by covering the costs of failed ventures).
The Montazerhodjat and Lo framework is also static, there is one test and then the story ends. In reality, drug approval has an interesting asymmetric dynamic. When a drug is approved for sale, testing doesn’t stop but moves into another stage, a combination of observational testing and sometimes more RCTs–this, after all, is how adverse events are discovered. Thus, Type I errors are corrected. On the other hand, for a drug that isn’t approved the story does end. With rare exceptions, Type II errors are never corrected. The Montazerhodjat and Lo framework could be interpreted as the reduced form of this dynamic process but it’s better to think about the dynamism explicitly because it suggests that approval can come in a range–for example, approval with a black label warning, approval with evidence grading and so forth. As these procedures tend to reduce the costs of Type I error they tend to increase the costs of FDA conservatism.
Montazerhodjat and Lo also don’t examine the implications of heterogeneity of preferences or of disease morbidity and mortality. Some people, for example, are severely disabled by diseases that on average aren’t very severe–the optimal tradeoff for these patients will be different than for the average patient. One size doesn’t fit all. In the standard framework it’s tough luck for these patients. But if the non-FDA reviewing apparatus (patients/physicians/hospitals/HMOs/USP/Consumer Reports and so forth) works relatively well, and this is debatable but my work on off-label prescribing suggests that it does, this weighs heavily in favor of relatively large samples but low thresholds for approval. What the FDA is really providing is information and we don’t need product bans to convey information. Thus, heterogeneity plus a reasonable effective post-testing choice process, mediates in favor of a Consumer Reports model for the FDA.
The bottom line, however, is that even without taking into account these further points, Montazerhodjat and Lo find that the FDA is far too conservative especially for severe diseases. FDA regulations may appear to be creating safe and effective drugs but they are also creating a deadly caution.
Hat tip: David Balan.
Saturday assorted links
2. Faster vaccines would have saved how many lives?
3. Yishan, on Twitter and Musk.
4. “Every chess movie, ranked.” On the list is an oddly under-publicized Kasparov-Thiel “documentary” of the two walking/driving around New York City for a day. Though it was only nine years ago, it feels like such a different world. You also get to watch Peter beat Frank Brady in speed chess, and Garry then comment on the proceedings.
5. From Hannes Malmberg (email): “Pre-1920, rapid natural population growth made high immigration consistent with a low share of foreign-born. Between 1880 and 1920, the number of foreign-born more than doubled, 6.7mn to 13.9mn, while the share of foreign-born didn’t budge, 13.3% to 13.2%.“