Innovation Breakdown

From my review today in the WSJ of Innovation Breakdown by Joseph Gulfo:

Yo is a smartphone app. MelaFind is a medical device. Yo sends one meaningless message: “Yo!” MelaFind tells you: “biopsy this and don’t biopsy that.” MelaFind saves lives. Yo does not. Guess which firm found it easier to put their product in consumers hands? Oy.

In “Innovation Breakdown: How the FDA and Wall Street Cripple Medical Advances,” Joseph Gulfo tells the tumultuous history of MELA Sciences, the company that invented MelaFind. When Dr. Gulfo joined the firm as president and CEO in 2004, the company’s brilliant team of scientists had spent many years and tens of millions of dollars to develop MelaFind, a “camera with a brain”—optical technology that would scan potential melanomas in multiple spectra and then, using sophisticated algorithms and large datasets, diagnose which were most likely to be cancerous.

MELA Sciences conducts an extensive clinical trial according to a protocol agreed on by the FDA and all looks good. After the clinical trial is completed, however, the FDA backs away from the protocol and comes out against MelaFind.

…The title of Dr. Gulfo’s book is “Innovation Breakdown” but “Innovator’s Breakdown” might have been more apt. The letter sent the author into survival mode. He battled the FDA, calmed investors, and defended against the lawsuit all while trying to keep the company afloat. Under stress, Dr. Gulfo’s health began to decline: He lost 29 pounds, his hair began to fall out, and the pain in his gut became so intense he needed an endoscopy. When his wife begged him to quit, he refused. They turned into roommates. “We were nothing more than cordial. I basically shut my wife out of my life,” he writes.

…The climax to this medical thriller comes when, in “the greatest 15 minutes of [his] life,” Dr. Gulfo delivers an impassioned speech, à la “Twelve Angry Men,” to the FDA’s advisory committee. The committee voted for approval, 8 to 7, and, perhaps with the congressional hearing in mind, the FDA approved MelaFind in September 2011.

It was a major triumph for the company, but Dr. Gulfo was beat. He retired from the company in June 2013—just in time to save his marriage.

Yet remarkably, given his experience, Mr. Gulfo writes that he still believes in a strong FDA. He argues in the book that better “leadership” and a few tweaks to existing rules can fix the problem. He’s wrong.

Compare MelaFind’s experience in the U.S. with its reception in Europe: MelaFind was submitted for marketing approval in Europe in May 2011. It was approved just five months later. One key reason for Europe’s efficient approval process is that European governments don’t review medical devices directly. Instead they certify independent “notified bodies” that specialize and compete to review new products. The European system works more quickly than the U.S. system, and there is no evidence that it results in reduced patient safety. Rather than tweak the current system, why doesn’t the U.S. just adopt the European model and call it a day? Our health and our economy would be better off for it.

Google’s Sergey Brin recently said that he didn’t want to be a health entrepreneur because “It’s just a painful business to be in . . . the regulatory burden in the U.S. is so high that I think it would dissuade a lot of entrepreneurs.” Mr. Brin won’t find anything in Dr. Gulfo’s book to persuade him otherwise. Until we get our regulatory system in order, expect a lot more Yo’s and not enough life-saving innovations.

Comments

'Yet remarkably, given his experience, Mr. Gulfo writes that he still believes in a strong FDA. He argues in the book that better “leadership” and a few tweaks to existing rules can fix the problem. He’s wrong.'

Man with intimate experience of the FDA and medical training is wrong, according to the Bartley J. Madden Chair in Economics at the Mercatus Center, who has neither.

Beyond satire, this time.

'MelaFind was submitted for marketing approval in Europe in May 2011. It was approved just five months later.'

And when were the FDA approved studies done? Quite a bit before May 2011, right? But let's just elide that point, shall we?

Nicely spotted, approval date in Bureaucratic America is Sep 2011 while for innovation friendly Europe is Oct 2011. One has to assume these events are completely independent for Alex's narrative to work, are they?

And it only took 7 years during which time no one was able to be helped by the innovation, not to mention the innovation wouldn't have happened at all had it not been for one persons push. How many innovations are we missing out on that are being pushed by businesses who only have 1 or 2 years of funds? Do good medical ideas only come from the very wealthy?

I wonder if those Americans with ebola would have been able to take experimental drugs if it wasn't for the media attention...

"Man with intimate experience of the FDA and medical training is wrong, according to the Bartley J. Madden Chair in Economics at the Mercatus Center, who has neither."

That's just an argument from authority.

To be fair, it's not just an argument from authority. It's also guilt by association.

Except that he is wrong.

My guess is Stockholm syndrome or first guy in line syndrome.

Argument from authority is only a logical fallacy when cited to an irrelevant source of authority. In field, it is a valid observation.

Any valid argument from authority should be able to provide supporting reasons beyond because expert 1 said so. You can't really trust people you don't know and anytime you do it's an opportunity for someone to take advantage or mislead you (intentionally or not).

Incorrect!

A number of books on logic and argumentation disagree with you. They seemed pretty authoritative.

What are your qualifications to make such a statement?

"better “leadership” ... can fix the problem": CEOs seem terribly keen on the Führerprinzip.

Of course a CEO thinks more CEOing is the answer.

I have no trouble believing someone not intimately involved knows more than someone intimately involved. I have imagination. I also see this every day. I tell the stories here all the time.

The doctor has on his website my procedure is not covered. Just to make sure I called insurance after 2 hours they still couldn't give a definitive answer.

The point is, that is just happening as I type. It happens to me almost every day where I tell people what their jobs mean.

Perhaps Mr. Gulfo isn't honest. After all, continuation of his business depends on keeping his allies in the FDA. After all, at the end of the day a majority voted for him. But if he alienates them by writing that their agency needs to be defanged - they will shut his business down in retaliation.

People everywhere have the good instinct to self-censor their expressions when dealing with a powerful organization that can crush them with little accountability. Step over the line, and you are dissident with all the 'fun' that ensues.

I am sympathetic to this cause, but the "Yo" lead falls flat, since it hand-waves away the real cost-benefit issues at play.

If there is anything that should ever be easy to release into the world, it's an app that just says "Yo."

Nice review, Alex.

'Twas ever so. If you want regulators to leave you alone, then devote yourself to something frivolous that can fail without anyone much caring. Produce soda pop, not milk.

Treating melanomas is why we have a health care crisis.

"“This technology should still be considered to be in the developmental stage,” said Dr. Roberta Lucas, an instructor of clinical dermatology at the Northwestern University Feinberg School of Medicine in Chicago. “We are better off when the system supports doctors who are thorough and unhurried; who examine and listen carefully and who empower patients to practice good surveillance and sun protection.”

In fact, some members of an expert medical panel asked to review MelaFind a few years ago for the Food and Drug Administration warned that the device had the potential to give doctors and patients a false sense of security. While MelaFind can analyze small pigmented spots identified by dermatologists as having signs of melanoma, it is not designed to evaluate other problems: large melanomas, colorless melanomas or two other types of skin cancer — basal and squamous cell carcinoma.

Dr. Amy E. Newburger, a dermatologist in Scarsdale, N.Y., who was a member of that F.D.A. panel, told me that she was concerned that a doctor could inadvertently use MelaFind on a non-melanoma skin cancer, receive a score indicating that the spot was not irregular, and erroneously decide not to biopsy it. She voted against recommending the device for F.D.A. approval.

Some biostatisticians are also critical of MelaFind, saying the device can recognize a high percentage of melanomas correctly because it also falsely scores as positive so many non-melanomas — potentially prompting doctors to perform unnecessary biopsies.

To help me visualize that issue, Jason Connor, a biostatistician at Berry Consultants, a biostatistics consulting firm, compared the accuracy of MelaFind in distinguishing non-melanomas to a hypothetical pregnancy test which, used on 100 nonpregnant women, would mistakenly conclude that 90 of them were pregnant.

“My concern with MelaFind is that it just says everything is positive,” Mr. Connor said. A member of the F.D.A. panel, he abstained on a vote about whether the device’s intended uses outweighed the risks.

“I don’t think this helps an aggressive doctor,” Mr. Connor told me, “and unaggressive doctors could do just as well if they were more diligent without the device.”

The concerns about the performance of the test are legitimate, but it's hard to evaluate them without knowing the performance of traditional melanoma diagnosis tests.

The other concerns are, in my opinion, red herrings. If the device offers more accurate diagnosis of melanoma, then it is a useful tool and should be approved. Whether doctors will "empower patients", advise sunscreen use, have a false sense of security, ignore other problems, etc., are simply irrelevant to the performance of the device. In my opinion the discussion you have quoted does not raise the stock of the FDA in this matter.

Agree.

That quote just made me think of buggy whip makers decrying the invention of the car.

If you ask the people who stand to lose the most what they think, what do you think they will say? Especially since we know that the field (and dermatology in particular) is a cartel designed to protect high wages.

In my experience, the incidence of false positives and unnecessary biopsies directly corresponds with my insurance provider. If the visit would be covered by the insurance provider in the incidence of a biopsy being performed they would perform one.

I had this same experience with multiple dermatologists, it would appear from my perspective doctors follow insurance incentives, who would have guessed?

See? All those concerns are dumb.

People on panels have to justify their seat by filling the air with words that usually aren't thought out very well.

The Roberta Lucas critique?

Yo!

Someone should figure out the salary of the health care regulators, express it as a percentage of the GDP and then send it to Krugman so that he can write an article showing that health care regulation is not a problem: "Just look at how small a percent of GDP the wages of the regulators are!"

Hitler's salary was a small percentage of GDP.

Yeah, and I say Krugman would say something really stupid like [________]. Just like that.

So does this test have a beter AUC than current practice tests or not? I did not understand. Also the EU vs US comparison seems very very flawed... Obviously it will take longer in the US since there was not yet any scientific data ....

I shit you not, I am right now getting a $2000 procedure that is not insured because it is better than the $10,000 procedure my insurance would pay for.

How, I wonder does it become not experimental? By experimenting on me. That's how.

Out of date. "Yo" has been superseded by "Holla!"

Now I get it: they compared the sensitivity/specificity of Melafind against a specialist dermatologist. The sensitivity is about 98% (quite good compared to dermatologists only 78%(?)!), but specificity (false positives) is 3 times higher (9% vs 3% dermatologists). I think a lot this "resistance" comes from the fact that Melafind is replacing dermatologists work. There is an obvious conflict of interest right? They are likely also represented in the decision panel(?). I would say then, that blaiming the regulator is perhaps a bit too simplistic...

"blaming the regulator is perhaps a bit too simplistic"

If the regulator has set things up so that people with a conflict of interest are making the decisions, it is correct to blame the regulator.

I suspect it is that way partly because false negatives are worse for the patient.

Especially when a false positive might yield " do it the old fashioned way on these ones."

One correction; specificity is not false positives, it is "true negatives", or the proportion of benign lesions that are correctly identified as non-melanomas in this case. They're complimentary, so 100% is ideal and a 9% vs 3% is 3x better, but really a 3% rate for the dermatologists is pretty terrible.

There is an error in the criticism of Yo and etc. You say that it is evidence of an innovation breakdown, but in fact it shows how pervasive and complete innovation has become.

I am an app developer. I can write apps. Whenever I think of a neat one though, I go search, and see ten offerings. The last time this happened I was talking to a kid and realized it would be fun to take pictures with the phone/tablet, and then insert ghosts ... quick search, a dozen "ghost cameras."

This is not *absence* of innovation, this is triumph. If you want an app, it was written in the last 5 years.

A little data:

According to latest data from Apple, there are now over 600,000 apps available on the App Store, 200,000 of which designed specifically for the iPad. On July 21, 2008, the App Store had 900 apps available for download; a year later, it had 65,000; on May 3, 2010, Apple announced 200,000 apps were available on the App Store (with 5,000 for iPad), and on October 4, 2011 (the day the iPhone 4S was officially unveiled), the company formalized the impressive milestone of half a million apps available on the iOS App Store. Today, Apple says over 600,000 apps are available for iOS devices, with website AppShopper reporting 616,196 as "total available apps".

How on earth can 0 to 600K apps in 4 years be called an "innovation breakdown?"

Because the world is bigger than that encompassed by a phone or tablet?

Can you write an app that performs endoscopic evaluations of tumors with high accuracy and no bias in the errors?

I can't even imagine an app that would collect internet tidbits for the feverish processing and analysis by a clever and well-educated commentariat. Such an app probably should not be developed anyway, it would only help clever people waste time.

The argument is about innovation. We don't get to divide that to our wish list, unless we are just engaging in personal fantasy. There are thousands of innovators out there, innovating things. 600K apps in 4 years is fast, very fast.

Now, I understand that we'd all like space age medicine now, but that is not really an "innovation breakdown" argument. That is an argument about how to target rampant innovation, to meet your preferences.

I guess you are arguing for a national industrial policy, right?

No, I'm not.

Just as space age medicine is not all of innovation, so are apps not all of innovation.

You are saying 600K apps in 4 years, an astounding number, are no good to you, because you want something else.

That would be like me saying that 100K new consumer applications of carbon fiber don't matter, because they aren't what I want, or aren't "all of innovation."

Oh, here's another big one. There have also been like 100K consumer applications of the new high efficiency LEDs. Not what you want?

CC: "The market was down today."
j: "But GE stock closed higher!"
CC: "Yes, but the market was down today."
j: "How can you say that? GE closed HIGHER."
CC: "Yes but the MARKET was down today."
j: "Intel closed higher, too!"

Apologies to Tom Stoppard.

Note that you are essentially making the "smart people should not go to Wall St." argument. Freedom?

Can't even fathom how you got that idea, sorry.

The arc of Alex's story is from "Yo" to "Where's my medicine." Of course that is an argument that innovators "should" be doing what he (or you) want.

Peter drucker's distinction between innovation and invention might be helpful. But I can't remember what it is.

If it's a market of morons, smart people will develop products to separate morons from their money. If it's a market of bourgeois, smart people will develop products to separate bourgeois from their money.

What if it is a market of people between morons and bourgeois? What then?

I should also say that I understand Alex's concern that FDA regulations might not be at an optimum trade-off for safety and speed of innovation. That's a valid concern, and can be discussed again, as we have in the past.

It's just silly to tie it to "Yo" as evidence for breakdown. Yo is not there because it's the best app that can be thought of, it's there because the App ecosystem was fill with astounding speed. You have a harder time now *not* finding a completed app for your needs. And so simple joke apps become the meme.

Besides Prior_Approval's excellent points upstream, I'd like to say that if we had stronger patent laws, you would not need an FDA. A patented device that works is incentive to be copied by infringers, which then empowers the patent holder to sue and get money in a court of law. That's how Dr. Raymond Vahan Damadian made money, after GE stole (and improved upon) his MRI invention, which initially was a binary "YES/NO" cancer detector based on a crude form of non-imaging MRI.

So AlexT's narrative would work to persuade me even more if AlexT was pro-patent rather than pro-infringer or pro-copier. Essentially AlexT wants no FDA and the right of people to usurp others inventions for free--the worse of all worlds.

We talk about private companies spending their money to create medical patents, but I worry that many come out of public spending. Sure, majority private funded projects should produce intellectual property, but conversely majority public funded projects should produce public knowledge.

Seems to me the key would be to innovate for the European and perhaps rest of the world. Sure, you go through the motions for FDA approval but waste a lot of time and effort fighting them. And don't promote selling into the US market to investors. Then if you get through the FDA, bonus, if not, your business plan still works. If it doesn't work without the potential of selling into the US market, then regardless of the medical benefits, the device isn't viable in the current regulatory environment.

I agree with Prof. Tabarrok. The Pareto-optimal solution (and equitable) solution would be for "Yo" to have been subjected to several years' worth of expensive testing before it was allowed onto the market.

"Nobody says 'yo' anymore"
"But 'holla' is still unproven!"
DENIED

"Then if you get through the FDA, bonus, if not, your business plan still works. If it doesn’t work without the potential of selling into the US market, then regardless of the medical benefits, the device isn’t viable in the current regulatory environment."

I not sure if that was a parody comment. But in case it wasn't, please understand that most medical devices and drugs (no matter where discovered) expect to make back their research costs in the US market. The European market doesn't pay enough to fund the research (especially in pharmaceuticals). Effectively the US market subsidizes European low prices. Now we can hope that the European market would cover the gap if prices came down here. But it is equally (or more) possible that research would drop significantly with no one paying for current levels.

This is Europe and NICE enacting 'Bugger thy Neighbor.'

It just occurred to me that economists already made that joke.

Thalidomide. The European system missed that.

The issue is not how many false positives and negatives, but their cost in lives and money.

Comments for this post are closed