Results for “Tests”
771 found

Peltzman Revisited

Casey Mulligan has an excellent new paper, Peltzman Revisited: Quantifying 21st-Century Opportunity Costs of Food and Drug Administration Regulation. What are the costs of delaying a new drug or a vaccine? Longer and bigger clinical trials increase safety but I’ve often made the point that the people who would have lived had a good drug been approved sooner are buried in an invisible graveyard and thus these costs are typically undercounted–the failure to see the invisible graveyard biases decisions in favor of delay. Mulligan makes a different and rarely considered point about substitution effects. If a vaccine isn’t available there are substitutes but these substitutes are themselves potentially unsafe and ineffective. But who is testing the substitures?

Many of these substitute interventions, such as remote work, closing schools, and canceling normal medical appointments, are beyond the jurisdiction of the FDA and can be utilized without any attempt to demonstrate their safety or efficacy.

If the substitutes work, the costs of delay are reduced. The FDA, for example, is right to prioritize drugs for which there are few alternative treatments. But the standards for many vaccine or drug substitutes are completely different than those used to approve a vaccine:

Closing schools to in-person learning is an important example of a prevention activity that was available, was applied to tens of millions of children in the United States, and was outside the FDA’s jurisdiction…Obviously the FDA’s effectiveness standard for vaccines differs from the effectiveness standard (if any) that school districts applied in deciding to close schools.

Where were the randomized controlled trials for closing schools, shutting the parks and beaches, and delaying medical appointments? Thus, it’s quite possible that greater safety of vaccines comes at the expense of greater time under less safe and possibly unsafe substitutes. As Mulligan concludes:

Approval delays for pandemic tests and vaccines pushed tens of millions of individuals and businesses into preventions and treatments that were both outside FDA jurisdiction and hardly safe or effective. The pandemic experience raises the question of whether, on the whole, consumers engage in more unsafe and ineffective practices than they would if FDA approval were not a prerequisite for pharmaceutical sales.

Addendum: Much else of interest in the paper including a calculation of the value of the vaccines in the hundreds of billions and trillions very much in line with work done by the AHT team, including myself ,in the AER PP (especially the appendix) and Science.

How will AI change *other* policy debates?

Not debates about AI, no the other ones.  For one thing, more ideas and more intelligence will mean more projects.  That raises the value of permitting reform, and raises the value of YIMBY.  But perhaps the most surprising conclusion comes on the immigration issue, with apologies to Garett Jones:

Which policy issue might decrease in importance? My prediction is immigration. GPT-4 is already passing a wide swath of professional tests, ranging from bar exams to medical qualifiers to economics exams. The service still requires more integration with other software tools, but in the not-too-distant future, the US will have added the equivalent of many millions of intelligent minds (albeit not bodies) to its labor force.

I have long favored boosting America’s immigration flow by about three times, to give the US an intake roughly on a per-capita par with Canada and Australia. This is still a good idea, but it should be done in a different way. Rather than more high-skilled immigration, the new priority might be more lower-wage migrants. The US might want a “bar-belled” immigration policy, which gives priority to AI researchers and engineers on the high-wage end, and workers such as construction laborers on the low-wage end.

The AI researchers, by creating more and better AI, would serve as a substitute for many other potential high-skilled immigrants. But all those new ideas will need people to turn them into actual projects in the physical world. In contrast, importing additional humanities professors from Europe no longer seems so vital when you can ask the AI instead.

Here is my full Bloomberg column.  To be sure, we need the smart, ambitious carpenters!

Are social media making us miserable?

Stuart Richie rebuts some of the recent studies:

And here’s the thing: when the authors of the “Facebook arrival” study raised their standards in this way, running a correction for multiple comparisons, all the results they found for well-being were no longer statistically significant. That is, a somewhat more conservative way of looking at the data indicated that every result they found was statistically indistinguishable from a scenario where Facebook had no effect on well-being whatsoever.

Now let’s turn to the second study, which was a randomised controlled trial where 1,637 adults were randomly assigned to shut down their Facebook account for four weeks, or go on using it as normal. Let’s call it the “deactivating Facebook” study. This “famous” study has been described as “the most impressive by far” in this area, and was the only study cited in the Financial Times as an example of the “growing body of research showing that reducing time on social media improves mental health”.

The bottom-line result was that leaving Facebook for a month led to higher well-being, as measured on a questionnaire at the end of the month. But again, looking in a bit more detail raises some important questions.

First, the deactivation happened in the weeks leading up to the 2018 US midterm elections. This was quite deliberate, because the researchers also wanted to look at how Facebook affected people’s political polarisation. But it does mean that the results they found might not apply to deactivating Facebook at other, less fractious times – maybe it’s particularly good to be away from Facebook during an election, when you can avoid hearing other people’s daft political opinions.

Second, just like the other Facebook study, the researchers tested a lot of hypotheses – and again, they used a correction to reduce false-positives. This time, the results weren’t wiped out entirely – but almost. Of the four questionnaire items that showed statistically-significant results before the correction, only one – “how lonely are you?” – remained significant after correction.

It’s debatable whether even this result would survive the researchers corrected for all the other statistical tests they ran. Not only that, but they also ran a second model, controlling for the overall amount of time people used Facebook, and this found even fewer results than the first one.

Third, as well as the well-being questionnaire at the end of the study, the participants got daily text messages asking them how happy they were, among other questions. Oddly, these showed absolutely no effect of being off Facebook – and not even the slightest hint of a trend in that direction.

Here is the entire piece, which is well thought out.

What do I think of the economists’ Israel petition?

It is signed by many luminaries, and it opens with this:

The governing coalition in Israel is considering an array of legislative acts that would weaken the independence of the judiciary and its power to constrain governmental actions. Numerous Israeli economists, in an open letter that some of us joined, expressed concerns that such a reform would adversely affect the Israeli economy by weakening the rule of law and thereby moving Israel in the direction of Hungary and Poland. Although we significantly vary in our views on public policy and on the challenges facing Israeli society, we all share these concerns. A strong and independent judiciary is a critical part of a system of checks and balances. Undermining it would be detrimental not only to democracy but also to economic prosperity and growth.

I would say I haven’t made up my mind on the substantive issue, as I have seen credible (not saying they are true, I don’t know) arguments that the current Israeli judiciary has too much power.  The proposed reforms still might be a badly timed and significant overreach, but my intuition is that the arguments are more complicated than this petition is making them out to be.  As economists, are they not at least obliged to tell us what the relevant trade-off is?

I also wonder if these outside voices have influence in Israeli politics, or whether they might occasion backlash.  Again, I don’t know, but I do see an argument for reserving collective petitions for very clear cut cases when the transmitted signal will be positive.  Is the binding constraint here “not having enough elite academic foreigners in opposition to Netanyahu”?

More generally and perhaps most importantly, will this petition be effective?  Many kinds of petitions should be saved up for when they will change something.  If they are not going to matter, in essence the signers are signaling their weakness rather than their strength.  They are spending down their reputational capital, rather than building it up.  And in those cases, why have the petition at all?

Monday assorted links

1. Contrarian perspective on the current Israeli disputes.

2. Running cognitive pipelines on cheap hardware.  And additive prompting.

3. Maxims.  17th century, Hansonian.

4. Turkey fact of the day: “Turkey borders seven different countries all of which use different alphabets.”  Significant.

5. Machine learning as a tool for hypothesis generation.

6. “This sea slug cut off its own head — and lived to tell the tale.”  And readjusting our expectations about the Fed, future rate hikes, and presumably inflation as well.  Be ready people.

AGI is Coming

ARSTechnica: On Monday, researchers from Microsoft introduced Kosmos-1, a multimodal model that can reportedly analyze images for content, solve visual puzzles, perform visual text recognition, pass visual IQ tests, and understand natural language instructions. The researchers believe multimodal AI—which integrates different modes of input such as text, audio, images, and video—is a key step to building artificial general intelligence (AGI) that can perform general tasks at the level of a human.

In 2020 Metaculus forecasters were predicting weak general AI by around 2053. Now they are predicting weak general AI by 2028 and strong general AI which includes:

  • Has general robotic capabilities, of the type able to autonomously, when equipped with appropriate actuators and when given human-readable instructions, satisfactorily assemble a (or the equivalent of a) circa-2021 Ferrari 312 T4 1:8 scale automobile model. A single demonstration of this ability, or a sufficiently similar demonstration, will be considered sufficient.

by 2040.

I never expected to witness the birth of aliens. It is a very strange time to be alive. If you think the world isn’t changing in a very uncertain and discontinuous way you just aren’t paying attention.

New facts about the game theory of balloons

But it turns out that China’s effort has been underway for more than a decade. According to a declassified intelligence report issued Thursday by the State Department, it involves a “fleet of balloons developed to conduct surveillance operations” that have flown over 40 countries on five continents.

That is from the Washington Post.  And:

Balloon operations obviously make sense for the Chinese. The United States has military bases in Japan and elsewhere from which it can launch daily flights by P-8 and other surveillance planes that fly perilously close to Chinese airspace. China doesn’t have similar options.

The frequency of these American “Sensitive Reconnaissance Operations,” or SROs, has increased sharply from about 250 a year a decade ago to several thousand annually, or three or four a day, a former intelligence official told me. China wants to push back, and collect its own signals; it wants its own version of “freedom of navigation” operations. Balloons are a way to both show the flag and collect intelligence…

Let’s look at another tit-for-tat motivation: China claims in its internal media that the Pentagon has aggressive plans to use high-altitude balloons, in projects such as “Thunder Cloud.”

It turns out the Chinese are right. Thunder Cloud was the name for the U.S. Army’s September 2021 exercise in Norway to test its “Multidomain Operations” warfighting concept, following a similar test in the Pacific in 2018, according to the Pentagon’s Defense News.

Here is my previous post on the game theory of the balloons.  Worth a reread.

Yglesias on Operation Warp Speed and the Republicans

Here’s Yglesias on Operation Warp Speed and the Republicans:

The debate over Operation Warp Speed wasn’t just a one-off policy dispute. Long before the pandemic, there was a conservative critique that the Food and Drug Administration is too slow and too risk-averse when it comes to authorizing new medications. Alex Tabarrok, a George Mason University economist, wrote about the “invisible graveyard” that could have been avoided if the FDA took expected value more seriously and considered the cost of delay in its authorization decisions.

The pandemic experience validated this criticism, which came to be embraced by some on the left as well — and it was about more than just vaccines. When it came to home Covid tests, Ezra Klein noted in the New York Times in 2021, “the problem here is the Food and Drug Administration. They have been disastrously slow in approving these tests and have held them to a standard more appropriate to doctor’s offices than home testing.” 

And yet, just as the invisible graveyard was becoming seen and the debate was being won and just as a historical public-private partnership had sped vaccines to the public and saved millions, the Republicans abandoned the high ground:

…it’s not surprising that Democrats are comfortable with the bureaucratic status quo and hesitant to ruffle feathers at federal regulatory agencies. What’s shocking is that Republicans — the traditional party of deregulation, the party that argued for years that the FDA is too slow-footed, the party that saved untold lives by accelerating vaccine development under Trump — have abandoned these positions.

At the cusp of what should have been a huge policy victory, Republicans don’t brag about their success, and they have no FDA reform legislation to offer. Instead, they’ve taken up the old mantle of hard-left skepticism of modern science and the pharmaceutical industry. 

It’s been painful to see all that has been gained now being lost. Libertarian economists and conservatives argued for decades that the FDA worried more about approving a drug that later turns out to be unsafe than about failing to approve a drug that could save lives; thus producing a deadly caution. But now the FDA is being attacked for what they did right, quickly approving safe vaccines. I hope that he is wrong but I fear that Yglesias is correct that the FDA may now get even slower and more cautious.

The irony of the present moment is that there is substantial backlash to the FDA’s approval of vaccines that haven’t turned out to be dangerous at all.

That’s only going to make regulators even more cautious. Right now the entire US regulatory state is taking essentially no heat for the slow progress on the next generation of vaccines, and an enormous amount of heat for the perfectly safe vaccines that it already approved. And the ex-president who pushed them to speed up their work on those vaccines is not only no longer defending them, he’s embarrassed to have ever been associated with the project.

Like I said, it’s a comical moment of Republican infighting. But it’s a very grim one for anyone concerned with the pace of scientific progress in America.

A possible health surveillance breakthrough

Yale researchers have found that testing for the presence of a single immune system molecule on nasal swabs can help detect stealthy viruses not identified in standard tests, they report Jan. 1 in the journal Lancet Microbe.

Finding a dangerous new virus is like searching for a needle in a haystack,” said Ellen Foxman, associate professor of laboratory medicine and immunobiology and senior author of the study. “We found a way to significantly reduce the size of the haystack.”

…For the new study, Foxman and her team revisited an observation made in her lab in 2017, which they thought may provide a new way to monitor for unexpected pathogens. Nasal swabs are commonly taken from patients with suspected respiratory infections and are tested to detect specific signatures of 10 to 15 known viruses. Most tests come back negative. But as Foxman’s team observed in 2017, in a few cases the swabs of those who tested negative for the “usual suspect” viruses still exhibited signs that antiviral defenses were activated, indicating the presence of a virus. The telltale sign was a high level of a single antiviral protein made by the cells that line the nasal passages.

Here is further information, this is a project funded and accelerated by Fast Grants.

Indoor Air Quality and Learning

More on the surprisingly large effects of air pollution on cognition from Palacios, Eichholtz, Kok and Duran:

Governments devote a large share of public budgets to construct, repair, and modernize school facilities. However, evidence on whether investments in the physical state of schools translate into better student outcomes is scant. In this study, we report the results of a large field study on the implications of poor air quality inside classrooms − a key performance measure of school mechanical ventilation systems. We continuously monitor the air quality (i.e., CO2), together with a rich set of indoor environmental parameters in 216 classrooms in the Netherlands. We link indoor air quality conditions to the outcomes on semi-annual nationally standardized tests of 5,500 children, during a period of five school terms (from 2018 to 2020). Using a fixed-effects strategy, relying on within-pupil changes in air quality conditions and test results, we document that exposure to poor indoor air quality during the school term preceding a test is associated with significantly lower test results: a one standard deviation increase in the school-term average daily peak of CO2 leads to a 0.11 standard deviation decrease in subsequent test scores. The estimates based on plausibly exogenous variation driven by mechanical ventilation system breakdown events confirm the robustness of the results. Our results add to the ongoing debate on the determinants of student human capital accumulation, highlighting the role of school infrastructure in shaping learning outcomes.

Note that the authors have data on the same students in high and low pollution episodes, allowing them to control for a wide variety of other factors.

Here are previous MR posts on air pollution including Why the New Pollution Literature is Credible and our MRU video on the Hidden Costs of Air Pollution. Note that you can take lower bounds of these effects and still think we are not paying enough attention to the costs of air pollution.

Biometrics will become more important

One side effect of all the new AI developments is that biometrics will increase greatly in importance.

Captcha tests, for one thing, will become obsolete.  To prevent bots from flooding a web site, biometric tests will be a ready option.

Passwords also will seem too risky.  If you are using only a password to protect your relationships with your AIs, there will (over time) be too much value in there.  The AI will be involved in your work routines, and it will be suited for you and you alone.  You won’t want to risk it being harmed or destroyed or reprogrammed.  Biometrics will provide superior protection.

It also will be harder to protect your passwords, because users will be more vulnerable to written and even voiced scams, run by AI and passing various Turing tests.  Again, biometric entry tests are the natural alternative.

Here is the 2022 biometric year in review, there is much pending in the way of legislation.  I already use facial recognition to access my iPhone, for better or worse expect much more of this.

Testing Freedom

In the latest Discourse Magazine I discuss the FDA’s long-standing fear and antipathy toward personalized medical tests and how this violates the 1st Amendment.

In 1972, the FDA confiscated thousands of home pregnancy tests, declaring that they were “drugs” meant to diagnose a “disease” and thus fell under the FDA’s regulatory dominion. The case went to the U.S. District Court for the District of New Jersey, and Judge Vincent P. Biunno ruled that the FDA had overstepped. “Pregnancy,” he said, “is a normal physiological function of all mammals and cannot be considered a disease … a test for pregnancy, then, is not a test for the diagnosis of disease. It is no more than a test for news….” As a result of Judge Biunno’s ruling, home pregnancy tests are easily available today from pharmacies, grocery stores and online shops without a prescription.

These days, debates over home pregnancy tests from the 1970s seem anachronistic and paternalistic. Yet the same paternalistic arguments appear again and again with every new testing technology. In the late 1980s, for example, the FDA simply declared that it would not approve at-home HIV tests, regardless of their safety or efficacy. As with pregnancy tests, the concern was that people could not be trusted with information about their own bodies…the first rapid at-home HIV test was developed and submitted to the FDA in 1987 [but] it took 25 years before the FDA would approve these tests. (Now, you can easily buy such a test on Amazon.)

…The FDA has a vital role in ensuring that tests are clinically accurate—tests should do what they say they do. Tests don’t need to be perfectly accurate to be useful (think of thermometers, personality tests and tire pressure gauges), but if a test advertises that it measures HDL cholesterol, it should do that within the tolerances the firm promises. The FDA has the technical knowledge to ensure that tests work, and that’s a skill that Americans value from the agency.

What Americans don’t want is to be told they can’t handle the truth. Yet when it came to at-home tests such as pregnancy tests, HIV tests and genetic tests, that’s exactly the reasoning the FDA used—and continues to use—to suppress information. The FDA should ensure that tests are safe, but “safety” means physical safety. The FDA may not declare a product unsafe because it might produce dangerous knowledge. Patients have a right to know about their own bodies. Our antibodies, ourselves. The FDA has authority over drugs and devices but not over patients.

Judge Biunno had it right back in 1972 when he said that diagnostic tests produce “news.” Test results, therefore, are a type of speech that fall under the First Amendment right to freedom of speech. The Supreme Court has repeatedly rejected restrictions on freedom of speech based on “a fear that people would make bad decisions if given truthful information”; thus, FDA restrictions on tests based on such fears are unconstitutional. The question of whether consumers will respond “safely” to test results is no more relevant to the FDA’s regulatory authority than the question of whether readers will respond safely to political news published in The New York Times. The FDA does not have the constitutional authority to regulate news.

Retrospective look at rapid Covid testing

To be clear, I still favor rapid Covid tests, and I believe we were intolerably slow to get these underway.  The benefits far exceed the costs, and did earlier on in the pandemic as well.

That said, with a number of pandemic retrospectives underway, here is part of mine.  I don’t think the strong case for those tests came close to panning out.

I had raised some initial doubts in my podcasts with Paul Romer and also with Glen Weyl, mostly about the risk of an inadequate demand to take such tests.  I believe that such doubts have been validated.

Ideally what you want asymptomatic people in high-risk positions taking the tests on a frequent basis, and, if they become Covid-positive, learning they are infectious before symptoms set in (remember when the FDA basically shut down Curative for giving tests to the asymptomatic?  Criminal).  And then isolating themselves.  We had some of that.  But far more often I witnessed:

1. People with symptoms taking the tests to confirm they had Covid.  Nothing wrong with that, but it leads to a minimal gain, since in so many cases it was pretty clear without the test.

2. Various institutions requiring tests for meet-ups and the like.  These tests would catch some but not all cases, and the event still would turn into a spreader event, albeit at a probably lower level than otherwise would have been the case.

3. Nervous Nellies taking the test in low-risk situations mainly to reassure themselves and others.  Again, no problems there but not the highest value use either.

So the prospects for mass rapid testing — done in the most efficacious manner — were I think overrated.

I recall the summer of 2022 in Ireland, which by the way is when I caught Covid (I was fine, though decided to spend an extra week in Ireland rather than risk infecting my plane mates).  Rapid tests were available everywhere, and at much lower prices than in the United States.  Better than not!  But what really seemed to make the difference was vaccines.  The availability of all those tests did not do so much to prevent Covid from spreading like a wildfire during that Irish summer.  Fortunately, deaths rose but did not skyrocket.

The well-known Society for Healthcare Epidemiology just recommended that hospitals stop testing asymptomatic patients for Covid.  You may or may not agree, but that is a sign of how much status testing has lost.

Some commentators argue there are more false negatives on the rapid tests with Omicron than with earlier strains.  I haven’t seen proof of this claim, but it is itself noteworthy that we still are not sure how good the tests are currently.  That too reflects a lower status for testing.

Again, on a cost-benefit basis I’m all for such testing!  But I’ve been lowering my estimate of its efficacy.