Science

Kieran Healy has a new paper on that topic (pdf), by the way a paper with a very short title (but this is a family blog).  Here is his opening paragraph:

Nuance is not a virtue of good sociological theory. Sociologists typically use it as a term of praise, and almost without exception when nuance is mentioned is is because someone is asking for more of it.  I shall argue that, for the problems facing Sociology at present, demanding more nuance typically obstructs the development of theory that is intellectually interesting, empirically generative, or practically successful.
And yet I find this paper has a lot of…nuance.  But of course Healy is consistent, it is “Actually Existing Nuance” he is railing against…

This is from a recent working paper (pdf) by Miguel Morin:

When the adoption of a new labor-saving technology increases labor productivity, it is an open question whether the economy adjusts in the medium-term by decreasing employment or increasing output. This paper studies the effects of cheaper electricity on the labor market during the Great Depression. The first-stage of the identification strategy uses geography as an instrument for changes in the price of electricity and the second-stage uses labor market outcomes from the concrete industry—a non-traded industry whose location decisions are independent of the instrument. The paper finds that electricity was an important labor-saving technology and caused an increase in capital intensity and labor productivity, as well as a decrease in the labor share of income. The paper also finds that firms adjusted to higher labor productivity by decreasing employment instead of increasing output, which supports the theory of technological unemployment.

You will note of course that the short-, medium- and long-run effects here are quite different, and of course electricity is a major boon to mankind.  Still, technological unemployment is not just the fantasy of people who have failed to study Ricardo.

Here is a short summary of the paper, via Romesh Vaitilingam.

I have long argued that the FDA has an incentive to delay the introduction of new drugs because approving a bad drug (Type I error) has more severe consequences for the FDA than does failing to approve a good drug (Type II error). In the former case at least some victims are identifiable and the New York Times writes stories about them and how they died because the FDA failed. In the latter case, when the FDA fails to approve a good drug, people die but the bodies are buried in an invisible graveyard.

In an excellent new paper (SSRN also here) Vahid Montazerhodjat and Andrew Lo use a Bayesian analysis to model the optimal tradeoff in clinical trials between sample size, Type I and Type II error. Failing to approve a good drug is more costly, for example, the more severe the disease. Thus, for a very serious disease, we might be willing to accept a greater Type I error in return for a lower Type II error. The number of people with the disease also matters. Holding severity constant, for example, the more people with the disease the more you want to increase sample size to reduce Type I error. All of these variables interact.

In an innovation the authors use the U.S. Burden of Disease Study to find the number of deaths and the disability severity caused by each major disease. Using this data they estimate the costs of failing to approve a good drug. Similarly, using data on the costs of adverse medical treatment they estimate the cost of approving a bad drug.

Putting all this together the authors find that the FDA is often dramatically too conservative:

…we show that the current standards of drug-approval are weighted more on avoiding a Type I error (approving ineffective therapies) rather than a Type II error (rejecting effective therapies). For example, the standard Type I error of 2.5% is too conservative for clinical trials of therapies for pancreatic cancer—a disease with a 5-year survival rate of 1% for stage IV patients (American Cancer Society estimate, last updated 3 February 2013). The BDA-optimal size for these clinical trials is 27.9%, reflecting the fact that, for these desperate patients, the cost of trying an ineffective drug is considerably less than the cost of not trying an effective one.

(The authors also find that the FDA is occasionally a little too aggressive but these errors are much smaller, for example, the authors find that for prostate cancer therapies the optimal significance level is 1.2% compared to a standard rule of 2.5%.)

The result is important especially because in a number of respects, Montazerhodjat and Lo underestimate the costs of FDA conservatism. Most importantly, the authors are optimizing at the clinical trial stage assuming that the supply of drugs available to be tested is fixed. Larger trials, however, are more expensive and the greater the expense of FDA trials the fewer new drugs will be developed. Thus, a conservative FDA reduces the flow of new drugs to be tested. In a sense, failing to approve a good drug has two costs, the opportunity cost of lives that could have been saved and the cost of reducing the incentive to invest in R&D. In contrast, approving a bad drug while still an error at least has the advantage of helping to incentivize R&D (similarly, a subsidy to R&D incentivizes R&D in a sense mostly by covering the costs of failed ventures).

The Montazerhodjat and Lo framework is also static, there is one test and then the story ends. In reality, drug approval has an interesting asymmetric dynamic. When a drug is approved for sale, testing doesn’t stop but moves into another stage, a combination of observational testing and sometimes more RCTs–this, after all, is how adverse events are discovered. Thus, Type I errors are corrected. On the other hand, for a drug that isn’t approved the story does end. With rare exceptions, Type II errors are never corrected. The Montazerhodjat and Lo framework could be interpreted as the reduced form of this dynamic process but it’s better to think about the dynamism explicitly because it suggests that approval can come in a range–for example, approval with a black label warning, approval with evidence grading and so forth. As these procedures tend to reduce the costs of Type I error they tend to increase the costs of FDA conservatism.

Montazerhodjat and Lo also don’t examine the implications of heterogeneity of preferences or of disease morbidity and mortality. Some people, for example, are severely disabled by diseases that on average aren’t very severe–the optimal tradeoff for these patients will be different than for the average patient. One size doesn’t fit all. In the standard framework it’s tough luck for these patients. But if the non-FDA reviewing apparatus (patients/physicians/hospitals/HMOs/USP/Consumer Reports and so forth) works relatively well, and this is debatable but my work on off-label prescribing suggests that it does, this weighs heavily in favor of relatively large samples but low thresholds for approval. What the FDA is really providing is information and we don’t need product bans to convey information. Thus, heterogeneity plus a reasonable effective post-testing choice process, mediates in favor of a Consumer Reports model for the FDA.

The bottom line, however, is that even without taking into account these further points, Montazerhodjat and Lo find that the FDA is far too conservative especially for severe diseases. FDA regulations may appear to be creating safe and effective drugs but they are also creating a deadly caution.

Hat tip: David Balan.

Bowen and Casadevall have a new PNAS paper on this question:

The general public funds the vast majority of biomedical research and is also the major intended beneficiary of biomedical breakthroughs. We show that increasing research investments, resulting in an increasing knowledge base, have not yielded comparative gains in certain health outcomes over the last five decades. We demonstrate that monitoring scientific inputs, outputs, and outcomes can be used to estimate the productivity of the biomedical research enterprise and may be useful in assessing future reforms and policy changes. A wide variety of negative pressures on the scientific enterprise may be contributing to a relative slowing of biomedical therapeutic innovation. Slowed biomedical research outcomes have the potential to undermine confidence in science, with widespread implications for research funding and public health.

Carolyn Johnson summarizes the results of the paper:

Casadevall and graduate student Anthony Bowen used a pretty straightforward technique to try and answer the question. They compared the NIH budget, adjusted for inflation, with the number of new drugs approved by the Food and Drug Administration and the increases in life expectancy in the U.S. population over the same time period.

Those crude health measures didn’t keep pace with the research investment. Funding increased four-fold since 1965, but the number of drugs only doubled. Life expectancy increased steadily, by two months per year.

Johnson also covers some useful responses from the critics.  The result also may say more about the NIH than about progress per se.  And here is a more optimistic take from Allison Schraeger.

Observers seem to focus on the target event and not its complement.  Bagchi and Ince have a new paper on this question:

Consumers routinely rely on forecasters to make predictions about uncertain events (e.g., sporting contests, stock fluctuations). The authors demonstrate that when forecasts are higher versus lower (e.g., a 70% vs. 30% chance of team A winning a game) consumers infer that the forecaster is more confident in her prediction, has conducted more in-depth analyses, and is more trustworthy. The prediction is also judged as more accurate. This occurs because forecasts are evaluated based on how well they predict the target event occurring (team A winning). Higher forecasts indicate greater likelihood of the target event occurring, and signal a confident analyst, while lower forecasts indicate lower likelihood and lower confidence in the target event occurring. But because, with lower forecasts, consumers still focus on the target event (and not its complement), lower confidence in the target event occurring is erroneously interpreted as the forecaster being less confident in her overall prediction (instead of more confident in the complementary event occurring—team A losing). The authors identify boundary conditions, generalize to other prediction formats, and demonstrate consequences.

Of course this also has relevance for the evolutionary processes governing pundits.

Here is a related press release (pdf).  For the pointer I thank Charles Klingman.

Tim Urban at Wait but Why has a fascinating longform series on How and Why SpaceX Will Colonize Mars which itself is part of a longer series on Elon Musk and his companies. Here’s just one bit:

The Scary Thing About the Universe

Species extinctions are kind of like human deaths—they’re happening constantly, at a mild and steady rate. But a mass extinction event is, for species, like a war or a sweeping epidemic is for humans—an unusual event that kills off a large chunk of the population in one blow. Humans have never experienced a mass extinction event, and if one happened, there’s a reasonable chance it would end the human race—either because the event itself would kill us (like a collision with a large enough asteroid), or the effects of an event would (like something that decimates the food supply or dramatically changes the temperature or atmospheric composition). The extinction graph below shows animal extinction over time (using marine extinction as an indicator). I’ve labeled the five major extinction events and the percentage of total species lost during each one (not included on this graph is what many believe is becoming a new mass extinction, happening right now, caused by the impact of humans):1

1062px-Extinction_intensity

…Let’s imagine the Earth is a hard drive, and each species on Earth, including our own, is a Microsoft Excel document on the hard drive filled with trillions of rows of data. Using our shortened timescale, where 50 million years = one month, here’s what we know:

  • Right now, it’s August of 2015
  • The hard drive (i.e. the Earth) came into existence 7.5 years ago, in early 2008
  • A year ago, in August of 2014, the hard drive was loaded up with Excel documents (i.e. the origin of animals). Since then, new Excel docs have been continually created and others have developed an error message and stopped opening (i.e gone extinct).
  • Since August 2014, the hard drive has crashed five times—i.e. extinction events—in November 2014, in December 2014, in March 2015, April 2015, and July 2015. Each time the hard drive crashed, it rebooted a few hours later, but after rebooting, about 70% of the Excel docs were no longer there. Except the March 2015 crash, which erased 95% of the documents.
  • Now it’s mid-August 2015, and the homo sapiens Excel doc was created about two hours ago.

Now—if you owned a hard drive with an extraordinarily important Excel doc on it, and you knew that the hard drive pretty reliably tended to crash every month or two, with the last crash happening five weeks ago—what’s the very obvious thing you’d do?

You’d copy the document onto a second hard drive.

That’s why Elon Musk wants to put a million people on Mars.

On a related note the latest Planet Money podcast is How to Stop an Asteroid. It’s funny and informative and yours truly makes an appearance. Worth a listen.

That is the title of my current column at The Upshot.  I very much enjoyed my read of William MacCaskill’s Doing Good Better: How Effective Altruism Can Help You Make a Difference.  The point of course is to apply science, reason, and data analysis to our philanthropic giving.

I am more positive than negative on this movement and also the book, as you can see from the column.  Still, I think my more skeptical remarks are the most interesting part to excerpt:

Neither Professor MacAskill nor the effective-altruism movement has answered all the tough questions. Often the biggest gains come from innovation, yet how can a donor spur such advances? If you had a pile of money and the intent to make the world a better place in 1990, could you have usefully expected or encouraged the spread of cellphones to Africa? Probably not, yet this technology has improved the lives of many millions, and at a profit, so for the most part its introduction didn’t draw money from charities. Economists know frustratingly little about the drivers of innovation.

And as Prof. Angus Deaton of Princeton University has pointed out, many of the problems of poverty boil down to bad politics, and we don’t know how to use philanthropy to fix that. If corruption drains away donated funds, for example, charity could even be counterproductive by propping up bad governments.

Sometimes we simply can’t know in advance how important a donation will turn out to be. For example, the financier John A. Paulson’s recently announced $400 million gift to Harvard may be questioned on the grounds that Harvard already has more money than any university in the world, and surely is not in dire need of more. But do we really know that providing extra support for engineering and applied sciences at Harvard — the purpose of the donation — will not turn into globally worthwhile projects? Innovations from Harvard may end up helping developing economies substantially. And even if most of Mr. Paulson’s donation isn’t spent soon, the money is being invested in ways that could create jobs and bolster productivity.

In addition, donor motivation may place limits on the applicability of the effective-altruism precepts. Given that a lot of donors are driven by emotion, pushing them to be more reasonable might backfire. Excessively cerebral donors might respond with so much self-restraint that they end up giving less to charity. If they are no longer driven by emotion, they may earn and save less in the first place.

On Paulson, here is Ashok Rao’s recent post on compounding returns.

Here is my Washington Post review of that book, which I very much liked.  Here is one bit from the review:

My favorite parts of the book are about the military, an area where most other popular authors on automation and smart software have hesitated to tread. In this book you can read about how much of America’s military prowess comes from superior human performance and not just from technology. Future gains will result from how combat participants are trained, motivated, and taught to work together and trust each other, and from better after-action performance reviews. Militaries are inevitably hierarchical, but when they process and admit their mistakes, they can become rapidly more efficient.

The subtitle of the book is What High Achievers Know That Brilliant Machines Never Will.

FDA approval at what price?

by on August 13, 2015 at 11:33 am in Economics, Law, Medicine, Science | Permalink

There is plenty of debate over whether the FDA should be looser or tougher with new drug approval, but I rarely hear the question posed as “approval at what price?”

One option would be to approve relatively strong and safe drugs at full Medicare and Medicaid reimbursement rates, if not higher.  Drugs with lesser efficacy or higher risk could be approved at lower reimbursement prices.  It is possible or perhaps even likely, of course, that private insurance companies would follow the government’s lead.

Dr. Peter Bach has promoted one version of this idea, and produced a calculator for valuing these drugs.  In essence the government would be saying to lower quality producers “yes, you can continue to try to improve this drug, but not at public expense.”

I believe proposals of this kind deserve further attention, and in general the notion of regulatory approval need not be conceived in strictly binary, yes/no terms.

From Greg Ip:

Quantifying innovation is difficult: Government statistics don’t adequately measure activities that only recently came into existence. Mr. Mandel circumvents this problem by surmising that innovation leaves its mark in the sorts of skills employers demand. For example, the shale oil and gas revolution is apparent in the soaring numbers of mining, geological and petroleum engineers, whereas the ranks of biological, medical, chemical, and materials scientists have slipped since 2006-07.

Screening job postings on Indeed, a job website, Mr. Mandel finds that the proportion mentioning “Android” (Google’s mobile operating system), “fracking” and “robotics” has risen notably in the past four to six years. But the proportion mentioning “composite materials,” “biologist,” “gene” or “nanotechnology” has trended down. His conclusion: Today’s economy is “unevenly innovative.”

You can find the whole article here.  Does anyone have a link to the study itself?

Sentences to ponder

by on August 12, 2015 at 1:24 pm in Education, Medicine, Philosophy, Science | Permalink

We did not observe any individual protein-altering variants that are reproducibly associated with extremely high intelligence and within the entire distribution of intelligence.

That is from “a whole bunch of guys” writing in Molecular Psychiatry, via Michelle Dawson.  In other words, the prospect of straightforward genetic engineering for smarter babies probably won’t be a reality anytime soon.  Technology remains pretty far behind the matchmaker.

Let’s raise their status! Details here.

One Billion Club

Hat tip: Zac Gochenour.

One recurring problem in economics, and the other social sciences all the more, is that researchers will accept a lot of conventional wisdom on a topic if it suits their preexisting biases, especially if it is not an area which they have researched themselves.  Yet this entire question is — surprise, surprise — largely unstudied.  Social scientists love to talk about themselves, but critical self-scrutiny backed by data is less popular.

Jason Briggeman just wrote a GMU dissertation to investigate these and similar questions, here is his abstract:

In the United States and most other wealthy nations, all drugs are banned unless individually permitted. This policy, called pre-market approval, is controversial among economists; the preponderance of the economics literature that offers a judgment on pre-market approval is critical of the policy, but surveys of U.S. economists show that many, perhaps a majority, support pre-market approval. Here I analyze the results of a recent survey that asked economists who support pre-market approval to justify, with reference to the economic concept of market failure, their support of the policy. I find that, while almost all the economists surveyed could point to a market failure or failures that may plausibly exist and affect the market for pharmaceuticals, none were able to make a well reasoned connection between those market failures and the particular remedy of pre-market approval. None of the economists surveyed cited in support of their position any literature specific to pre-market approval. I supplement the survey findings with a review of relevant reading material assigned in health economics courses at top universities, searching that material for discussions of what may justify pre-market approval. I find a strong argument that the prospect of overt disasters being caused by avoidable mistakes can justify some intervention in pharmaceuticals; however, I find little to justify the other interventions that are part of pre-market approval. I suggest that future inquiry into possibilities for liberalizing reform concentrate on understanding matters such as the informational effects of product bans, the distinction between safety and efficacy, the nature of demand for drugs about which little is known, and the political economy of drug substitutes.

The upshot is that economists hold a lot of views whose justifications they cannot articulate very well.  I think you would find the same when it comes to the Ex-Im Bank (are you sure it fits the model of strategic trade theory?), the mortgage agencies (what was that externalities argument for home ownership again?) or all sorts of random regulations.  The relatively interventionist economists will pull some justification out of a hat, and the relatively pro-market economists will be pretty skeptical.

For the pointer I thank Daniel Klein.

This Leah Sottile WaPo piece is excellent in many ways.  Here are a few bits:

Bees are still dying at unacceptable rates…Ohio State University’s Honey Bee Update noted that losses among the state’s beekeepers over the past winter were as high as 80 percent.

…Researchers say innovative beekeepers will be critical to helping bees bounce back.

“People ask me, ‘The bees are going to be extinct soon?’ ” said Ramesh Sagili, principal investigator at the Oregon State University Honey Bee Lab. “I’m not worried about bees being extinct here. I’m worried about beekeepers being extinct.”

Commercial beekeepers are leaving the sector and innovative bee hobbyists are taking on a much larger role:

“I feel a social responsibility to provide good bees,” Prescott said. “It makes me happy to look at the part that I’m playing.”

…Obsessing over bee health was unheard of 50 years ago, said Marla Spivak, a University of Minnesota professor of entomology. “In the past, it was very easy to keep bees. Throw them in a box, and they make honey and survive. Now, it takes lots of management.”

The story has some excellent examples:

Henry Storch, 32, does it because he felt a calling to beekeeping. A farrier by trade, Storch said he could make more money shoeing horses. But five years ago, he became obsessed with the notion that he could build a better bee…He barely flinched as a bee stung him on the upper lip.

…Storch’s mountain-bred “survivor” bees are like open-range cows: tough, hardened and less in need of close management than the bees he trucks to the California almond fields. Storch compares the effort to growing organic, non-GMO food.

The good news is this:

Amid the die-off, beekeepers have been going to extraordinary lengths to save both their bees and their livelihoods.

That effort may finally be paying off. New data from the Agriculture Department show the number of managed honeybee colonies is on the rise, climbing to 2.7 million nationally in 2014, the highest in 20 years.

Recommended.  To trace the longer story, here are previous MR posts on bees.

The ghost in the machine

by on August 7, 2015 at 7:30 am in Film, Science, Travel | Permalink

I visited two wonderful churches in Barcelona. The first, of course, was La Sagrada Familia. Ramez Naam put it best, this is “the kind of church that Elves from the 22nd Century would build.” I can’t add to that, however, so let me turn to the second church.

The Chapel Torre Girona at the Polytechnic University of Catalonia in Barcelona is home to the MareNostrum, not the world’s fastest but certainly the world’s most beautiful supercomputer.

BSCC

Although off the usual tourist path, it’s possible to get a tour if you arrange in advance. As you walk around the nave, the hum of the supercomputer mixes with Gregorian chants. What is this computer thinking you wonder? Appropriately enough the MareNostrum is thinking about the secrets of life and the universe.

In this picture, I managed to capture within the cooling apparatus a saintly apparition from a stained glass window.

The ghost in the machine.

ComputerSaint

Hat tip: Atlas Obscura.