Innovation Breakdown

by on August 12, 2014 at 7:47 am in Economics, Law, Medicine, Uncategorized | Permalink

From my review today in the WSJ of Innovation Breakdown by Joseph Gulfo:

Yo is a smartphone app. MelaFind is a medical device. Yo sends one meaningless message: “Yo!” MelaFind tells you: “biopsy this and don’t biopsy that.” MelaFind saves lives. Yo does not. Guess which firm found it easier to put their product in consumers hands? Oy.

In “Innovation Breakdown: How the FDA and Wall Street Cripple Medical Advances,” Joseph Gulfo tells the tumultuous history of MELA Sciences, the company that invented MelaFind. When Dr. Gulfo joined the firm as president and CEO in 2004, the company’s brilliant team of scientists had spent many years and tens of millions of dollars to develop MelaFind, a “camera with a brain”—optical technology that would scan potential melanomas in multiple spectra and then, using sophisticated algorithms and large datasets, diagnose which were most likely to be cancerous.

MELA Sciences conducts an extensive clinical trial according to a protocol agreed on by the FDA and all looks good. After the clinical trial is completed, however, the FDA backs away from the protocol and comes out against MelaFind.

…The title of Dr. Gulfo’s book is “Innovation Breakdown” but “Innovator’s Breakdown” might have been more apt. The letter sent the author into survival mode. He battled the FDA, calmed investors, and defended against the lawsuit all while trying to keep the company afloat. Under stress, Dr. Gulfo’s health began to decline: He lost 29 pounds, his hair began to fall out, and the pain in his gut became so intense he needed an endoscopy. When his wife begged him to quit, he refused. They turned into roommates. “We were nothing more than cordial. I basically shut my wife out of my life,” he writes.

…The climax to this medical thriller comes when, in “the greatest 15 minutes of [his] life,” Dr. Gulfo delivers an impassioned speech, à la “Twelve Angry Men,” to the FDA’s advisory committee. The committee voted for approval, 8 to 7, and, perhaps with the congressional hearing in mind, the FDA approved MelaFind in September 2011.

It was a major triumph for the company, but Dr. Gulfo was beat. He retired from the company in June 2013—just in time to save his marriage.

Yet remarkably, given his experience, Mr. Gulfo writes that he still believes in a strong FDA. He argues in the book that better “leadership” and a few tweaks to existing rules can fix the problem. He’s wrong.

Compare MelaFind’s experience in the U.S. with its reception in Europe: MelaFind was submitted for marketing approval in Europe in May 2011. It was approved just five months later. One key reason for Europe’s efficient approval process is that European governments don’t review medical devices directly. Instead they certify independent “notified bodies” that specialize and compete to review new products. The European system works more quickly than the U.S. system, and there is no evidence that it results in reduced patient safety. Rather than tweak the current system, why doesn’t the U.S. just adopt the European model and call it a day? Our health and our economy would be better off for it.

Google’s Sergey Brin recently said that he didn’t want to be a health entrepreneur because “It’s just a painful business to be in . . . the regulatory burden in the U.S. is so high that I think it would dissuade a lot of entrepreneurs.” Mr. Brin won’t find anything in Dr. Gulfo’s book to persuade him otherwise. Until we get our regulatory system in order, expect a lot more Yo’s and not enough life-saving innovations.

prior_approval August 12, 2014 at 8:04 am

‘Yet remarkably, given his experience, Mr. Gulfo writes that he still believes in a strong FDA. He argues in the book that better “leadership” and a few tweaks to existing rules can fix the problem. He’s wrong.’

Man with intimate experience of the FDA and medical training is wrong, according to the Bartley J. Madden Chair in Economics at the Mercatus Center, who has neither.

Beyond satire, this time.

‘MelaFind was submitted for marketing approval in Europe in May 2011. It was approved just five months later.’

And when were the FDA approved studies done? Quite a bit before May 2011, right? But let’s just elide that point, shall we?

Axa August 12, 2014 at 8:19 am

Nicely spotted, approval date in Bureaucratic America is Sep 2011 while for innovation friendly Europe is Oct 2011. One has to assume these events are completely independent for Alex’s narrative to work, are they?

XVO August 12, 2014 at 8:40 am

And it only took 7 years during which time no one was able to be helped by the innovation, not to mention the innovation wouldn’t have happened at all had it not been for one persons push. How many innovations are we missing out on that are being pushed by businesses who only have 1 or 2 years of funds? Do good medical ideas only come from the very wealthy?

I wonder if those Americans with ebola would have been able to take experimental drugs if it wasn’t for the media attention…

Jeff August 12, 2014 at 10:19 am

“Man with intimate experience of the FDA and medical training is wrong, according to the Bartley J. Madden Chair in Economics at the Mercatus Center, who has neither.”

That’s just an argument from authority.

dan1111 August 12, 2014 at 10:32 am

To be fair, it’s not just an argument from authority. It’s also guilt by association.

andrew' August 12, 2014 at 10:53 am

Except that he is wrong.

My guess is Stockholm syndrome or first guy in line syndrome.

Colin August 12, 2014 at 11:02 am

Argument from authority is only a logical fallacy when cited to an irrelevant source of authority. In field, it is a valid observation.

XVO August 12, 2014 at 11:05 am

Any valid argument from authority should be able to provide supporting reasons beyond because expert 1 said so. You can’t really trust people you don’t know and anytime you do it’s an opportunity for someone to take advantage or mislead you (intentionally or not).

Logical Authority August 12, 2014 at 11:59 am

Incorrect!

(Not That) Bill O'Reilly August 12, 2014 at 12:10 pm

A number of books on logic and argumentation disagree with you. They seemed pretty authoritative.

Jeff August 12, 2014 at 1:52 pm

What are your qualifications to make such a statement?

dearieme August 12, 2014 at 10:49 am

“better “leadership” … can fix the problem”: CEOs seem terribly keen on the Führerprinzip.

dan1111 August 12, 2014 at 10:52 am

Of course a CEO thinks more CEOing is the answer.

andrew' August 12, 2014 at 10:55 am

I have no trouble believing someone not intimately involved knows more than someone intimately involved. I have imagination. I also see this every day. I tell the stories here all the time.

andrew' August 12, 2014 at 10:58 am

The doctor has on his website my procedure is not covered. Just to make sure I called insurance after 2 hours they still couldn’t give a definitive answer.

The point is, that is just happening as I type. It happens to me almost every day where I tell people what their jobs mean.

Marian Kechlibar August 14, 2014 at 4:11 am

Perhaps Mr. Gulfo isn’t honest. After all, continuation of his business depends on keeping his allies in the FDA. After all, at the end of the day a majority voted for him. But if he alienates them by writing that their agency needs to be defanged – they will shut his business down in retaliation.

People everywhere have the good instinct to self-censor their expressions when dealing with a powerful organization that can crush them with little accountability. Step over the line, and you are dissident with all the ‘fun’ that ensues.

dan1111 August 12, 2014 at 8:06 am

I am sympathetic to this cause, but the “Yo” lead falls flat, since it hand-waves away the real cost-benefit issues at play.

Dan Weber August 12, 2014 at 11:26 am

If there is anything that should ever be easy to release into the world, it’s an app that just says “Yo.”

David R. Henderson August 12, 2014 at 8:10 am

Nice review, Alex.

John Mansfield August 12, 2014 at 8:48 am

‘Twas ever so. If you want regulators to leave you alone, then devote yourself to something frivolous that can fail without anyone much caring. Produce soda pop, not milk.

charlie August 12, 2014 at 8:57 am

Treating melanomas is why we have a health care crisis.

““This technology should still be considered to be in the developmental stage,” said Dr. Roberta Lucas, an instructor of clinical dermatology at the Northwestern University Feinberg School of Medicine in Chicago. “We are better off when the system supports doctors who are thorough and unhurried; who examine and listen carefully and who empower patients to practice good surveillance and sun protection.”

In fact, some members of an expert medical panel asked to review MelaFind a few years ago for the Food and Drug Administration warned that the device had the potential to give doctors and patients a false sense of security. While MelaFind can analyze small pigmented spots identified by dermatologists as having signs of melanoma, it is not designed to evaluate other problems: large melanomas, colorless melanomas or two other types of skin cancer — basal and squamous cell carcinoma.

Dr. Amy E. Newburger, a dermatologist in Scarsdale, N.Y., who was a member of that F.D.A. panel, told me that she was concerned that a doctor could inadvertently use MelaFind on a non-melanoma skin cancer, receive a score indicating that the spot was not irregular, and erroneously decide not to biopsy it. She voted against recommending the device for F.D.A. approval.

Some biostatisticians are also critical of MelaFind, saying the device can recognize a high percentage of melanomas correctly because it also falsely scores as positive so many non-melanomas — potentially prompting doctors to perform unnecessary biopsies.

To help me visualize that issue, Jason Connor, a biostatistician at Berry Consultants, a biostatistics consulting firm, compared the accuracy of MelaFind in distinguishing non-melanomas to a hypothetical pregnancy test which, used on 100 nonpregnant women, would mistakenly conclude that 90 of them were pregnant.

“My concern with MelaFind is that it just says everything is positive,” Mr. Connor said. A member of the F.D.A. panel, he abstained on a vote about whether the device’s intended uses outweighed the risks.

“I don’t think this helps an aggressive doctor,” Mr. Connor told me, “and unaggressive doctors could do just as well if they were more diligent without the device.”

dan1111 August 12, 2014 at 10:08 am

The concerns about the performance of the test are legitimate, but it’s hard to evaluate them without knowing the performance of traditional melanoma diagnosis tests.

The other concerns are, in my opinion, red herrings. If the device offers more accurate diagnosis of melanoma, then it is a useful tool and should be approved. Whether doctors will “empower patients”, advise sunscreen use, have a false sense of security, ignore other problems, etc., are simply irrelevant to the performance of the device. In my opinion the discussion you have quoted does not raise the stock of the FDA in this matter.

fwiw August 12, 2014 at 10:21 am

Agree.

That quote just made me think of buggy whip makers decrying the invention of the car.

If you ask the people who stand to lose the most what they think, what do you think they will say? Especially since we know that the field (and dermatology in particular) is a cartel designed to protect high wages.

mbutu o malley August 12, 2014 at 10:25 am

In my experience, the incidence of false positives and unnecessary biopsies directly corresponds with my insurance provider. If the visit would be covered by the insurance provider in the incidence of a biopsy being performed they would perform one.

I had this same experience with multiple dermatologists, it would appear from my perspective doctors follow insurance incentives, who would have guessed?

andrew' August 12, 2014 at 11:03 am

See? All those concerns are dumb.

People on panels have to justify their seat by filling the air with words that usually aren’t thought out very well.

J August 12, 2014 at 11:43 am

The Roberta Lucas critique?

Ed August 12, 2014 at 9:33 am

Yo!

Alex K. August 12, 2014 at 10:10 am

Someone should figure out the salary of the health care regulators, express it as a percentage of the GDP and then send it to Krugman so that he can write an article showing that health care regulation is not a problem: “Just look at how small a percent of GDP the wages of the regulators are!”

Steve-O August 12, 2014 at 3:48 pm

Hitler’s salary was a small percentage of GDP.

QZP August 17, 2014 at 6:24 am

Yeah, and I say Krugman would say something really stupid like [________]. Just like that.

Moreno Klaus August 12, 2014 at 10:46 am

So does this test have a beter AUC than current practice tests or not? I did not understand. Also the EU vs US comparison seems very very flawed… Obviously it will take longer in the US since there was not yet any scientific data ….

andrew' August 12, 2014 at 10:51 am

I shit you not, I am right now getting a $2000 procedure that is not insured because it is better than the $10,000 procedure my insurance would pay for.

How, I wonder does it become not experimental? By experimenting on me. That’s how.

andrew' August 12, 2014 at 10:49 am

Out of date. “Yo” has been superseded by “Holla!”

Moreno Klaus August 12, 2014 at 10:59 am

Now I get it: they compared the sensitivity/specificity of Melafind against a specialist dermatologist. The sensitivity is about 98% (quite good compared to dermatologists only 78%(?)!), but specificity (false positives) is 3 times higher (9% vs 3% dermatologists). I think a lot this “resistance” comes from the fact that Melafind is replacing dermatologists work. There is an obvious conflict of interest right? They are likely also represented in the decision panel(?). I would say then, that blaiming the regulator is perhaps a bit too simplistic…

dan1111 August 12, 2014 at 11:31 am

“blaming the regulator is perhaps a bit too simplistic”

If the regulator has set things up so that people with a conflict of interest are making the decisions, it is correct to blame the regulator.

andrew' August 12, 2014 at 8:42 pm

I suspect it is that way partly because false negatives are worse for the patient.

Especially when a false positive might yield ” do it the old fashioned way on these ones.”

J August 13, 2014 at 12:54 pm

One correction; specificity is not false positives, it is “true negatives”, or the proportion of benign lesions that are correctly identified as non-melanomas in this case. They’re complimentary, so 100% is ideal and a 9% vs 3% is 3x better, but really a 3% rate for the dermatologists is pretty terrible.

john August 12, 2014 at 10:59 am

There is an error in the criticism of Yo and etc. You say that it is evidence of an innovation breakdown, but in fact it shows how pervasive and complete innovation has become.

I am an app developer. I can write apps. Whenever I think of a neat one though, I go search, and see ten offerings. The last time this happened I was talking to a kid and realized it would be fun to take pictures with the phone/tablet, and then insert ghosts … quick search, a dozen “ghost cameras.”

This is not *absence* of innovation, this is triumph. If you want an app, it was written in the last 5 years.

john August 12, 2014 at 11:05 am

A little data:

According to latest data from Apple, there are now over 600,000 apps available on the App Store, 200,000 of which designed specifically for the iPad. On July 21, 2008, the App Store had 900 apps available for download; a year later, it had 65,000; on May 3, 2010, Apple announced 200,000 apps were available on the App Store (with 5,000 for iPad), and on October 4, 2011 (the day the iPhone 4S was officially unveiled), the company formalized the impressive milestone of half a million apps available on the iOS App Store. Today, Apple says over 600,000 apps are available for iOS devices, with website AppShopper reporting 616,196 as “total available apps”.

How on earth can 0 to 600K apps in 4 years be called an “innovation breakdown?”

China Cat August 12, 2014 at 11:16 am

Because the world is bigger than that encompassed by a phone or tablet?

Can you write an app that performs endoscopic evaluations of tumors with high accuracy and no bias in the errors?

I can’t even imagine an app that would collect internet tidbits for the feverish processing and analysis by a clever and well-educated commentariat. Such an app probably should not be developed anyway, it would only help clever people waste time.

john August 12, 2014 at 11:38 am

The argument is about innovation. We don’t get to divide that to our wish list, unless we are just engaging in personal fantasy. There are thousands of innovators out there, innovating things. 600K apps in 4 years is fast, very fast.

Now, I understand that we’d all like space age medicine now, but that is not really an “innovation breakdown” argument. That is an argument about how to target rampant innovation, to meet your preferences.

I guess you are arguing for a national industrial policy, right?

China Cat August 12, 2014 at 11:52 am

No, I’m not.

Just as space age medicine is not all of innovation, so are apps not all of innovation.

john August 12, 2014 at 12:02 pm

You are saying 600K apps in 4 years, an astounding number, are no good to you, because you want something else.

That would be like me saying that 100K new consumer applications of carbon fiber don’t matter, because they aren’t what I want, or aren’t “all of innovation.”

john August 12, 2014 at 12:04 pm

Oh, here’s another big one. There have also been like 100K consumer applications of the new high efficiency LEDs. Not what you want?

China Cat August 12, 2014 at 5:58 pm

CC: “The market was down today.”
j: “But GE stock closed higher!”
CC: “Yes, but the market was down today.”
j: “How can you say that? GE closed HIGHER.”
CC: “Yes but the MARKET was down today.”
j: “Intel closed higher, too!”

Apologies to Tom Stoppard.

john August 12, 2014 at 11:39 am

Note that you are essentially making the “smart people should not go to Wall St.” argument. Freedom?

China Cat August 12, 2014 at 11:52 am

Can’t even fathom how you got that idea, sorry.

john August 12, 2014 at 12:03 pm

The arc of Alex’s story is from “Yo” to “Where’s my medicine.” Of course that is an argument that innovators “should” be doing what he (or you) want.

andrew' August 12, 2014 at 8:45 pm

Peter drucker’s distinction between innovation and invention might be helpful. But I can’t remember what it is.

The Anti-Gnostic August 12, 2014 at 11:33 am

If it’s a market of morons, smart people will develop products to separate morons from their money. If it’s a market of bourgeois, smart people will develop products to separate bourgeois from their money.

andrew' August 12, 2014 at 8:53 pm

What if it is a market of people between morons and bourgeois? What then?

john August 12, 2014 at 11:44 am

I should also say that I understand Alex’s concern that FDA regulations might not be at an optimum trade-off for safety and speed of innovation. That’s a valid concern, and can be discussed again, as we have in the past.

It’s just silly to tie it to “Yo” as evidence for breakdown. Yo is not there because it’s the best app that can be thought of, it’s there because the App ecosystem was fill with astounding speed. You have a harder time now *not* finding a completed app for your needs. And so simple joke apps become the meme.

Ray Lopez August 12, 2014 at 12:37 pm

Besides Prior_Approval’s excellent points upstream, I’d like to say that if we had stronger patent laws, you would not need an FDA. A patented device that works is incentive to be copied by infringers, which then empowers the patent holder to sue and get money in a court of law. That’s how Dr. Raymond Vahan Damadian made money, after GE stole (and improved upon) his MRI invention, which initially was a binary “YES/NO” cancer detector based on a crude form of non-imaging MRI.

So AlexT’s narrative would work to persuade me even more if AlexT was pro-patent rather than pro-infringer or pro-copier. Essentially AlexT wants no FDA and the right of people to usurp others inventions for free–the worse of all worlds.

john August 12, 2014 at 12:49 pm

We talk about private companies spending their money to create medical patents, but I worry that many come out of public spending. Sure, majority private funded projects should produce intellectual property, but conversely majority public funded projects should produce public knowledge.

JKB August 12, 2014 at 12:44 pm

Seems to me the key would be to innovate for the European and perhaps rest of the world. Sure, you go through the motions for FDA approval but waste a lot of time and effort fighting them. And don’t promote selling into the US market to investors. Then if you get through the FDA, bonus, if not, your business plan still works. If it doesn’t work without the potential of selling into the US market, then regardless of the medical benefits, the device isn’t viable in the current regulatory environment.

Urso August 12, 2014 at 12:58 pm

I agree with Prof. Tabarrok. The Pareto-optimal solution (and equitable) solution would be for “Yo” to have been subjected to several years’ worth of expensive testing before it was allowed onto the market.

andrew' August 12, 2014 at 8:56 pm

“Nobody says ‘yo’ anymore”
“But ‘holla’ is still unproven!”
DENIED

Sebastian h August 12, 2014 at 6:26 pm

“Then if you get through the FDA, bonus, if not, your business plan still works. If it doesn’t work without the potential of selling into the US market, then regardless of the medical benefits, the device isn’t viable in the current regulatory environment.”

I not sure if that was a parody comment. But in case it wasn’t, please understand that most medical devices and drugs (no matter where discovered) expect to make back their research costs in the US market. The European market doesn’t pay enough to fund the research (especially in pharmaceuticals). Effectively the US market subsidizes European low prices. Now we can hope that the European market would cover the gap if prices came down here. But it is equally (or more) possible that research would drop significantly with no one paying for current levels.

andrew' August 12, 2014 at 8:58 pm

This is Europe and NICE enacting ‘Bugger thy Neighbor.’

It just occurred to me that economists already made that joke.

Eli Rabett August 20, 2014 at 1:32 pm

Thalidomide. The European system missed that.

The issue is not how many false positives and negatives, but their cost in lives and money.

Comments on this entry are closed.

Previous post:

Next post: