Sarah Constantin replies on MetaMed

Not long ago I linked to this Robin Hanson blog post on MetaMed.  I was sent this reply, which I will put under the fold:

I noticed you linked Robin Hanson’s article on MetaMed on Marginal Revolution.  I’m the VP of research at MetaMed, and I just wanted to tell you a little bit more about us, because if all you know about us is the Overcoming Bias article you might get some misleading impressions.

Medical practice is basically a mass-produced product. Professional and regulatory bodies (like the AMA) put out guidelines for treatment.  At their best, these guidelines follow the standards of evidence-based medicine, which means that on average they will produce the best health outcomes in the general population.  (Of course, in practice they often fall short of that standard.  For example, checklists are overwhelmingly beneficial by an evidence-based medicine standard, and yet are not universally used.)

But even at their best, the guidelines that are best from a population-health standpoint need not be optimal for an individual patient.  If you have the interest and the willingness to pay, investigating your condition in depth, in the context of your entire medical history, genetic data, and personal priorities, may well turn up opportunities to do better than the standardized medical guidelines which at best maximize average health outcomes.

That’s basically MetaMed’s raison d’etre.  And it’s a pretty conservative hypothesis, in fact.  We may harbor a few grander ambitions (for example, I come from a mathematical background and I’m working on some longer-term projects related to algorithmically automating parts of the diagnostic process, and using machine learning principles on biochemical networks in novel ways) but fundamentally the thing we claim to be able to do is give you finer-grained information than your doctor will.  We’re, of course, as yet unproven in the sense that we haven’t had enough clients to provide empirical evidence of how we improve health outcomes, but we’re not making extraordinary claims.

Robin Hanson seems to be implying that MetaMed is claiming to be useful only because we’re members of the “rationalist community.”  This isn’t true.  We think we’re useful because we give our clients personalized attention, because we’re more statistically literate than most doctors, because we don’t have some of the misaligned incentives that the medical profession does (e.g. we don’t have an incentive to talk up the benefits of procedures/drugs that are reimbursable by insurance), because we have a variety of experts and specialists on our team, etc.

The “rationalist” sensibility is important, to some degree, because, for instance, we’re willing to tell clients that incomplete evidence is evidence in the Bayesian sense, whereas the evidence-based medicine paradigm says that anything that yet hasn’t been tested in clinical trials and found a 5% p-value is completely unknown. For instance, we’re willing to count reasoning from chemical mechanisms as (weak) evidence. There’s a difference in philosophy between “minimize risk of saying a falsehood” and “be as close to accurate as possible”; we strive to do the latter.  So there’s a sense in which our epistemic culture allows us to be more flexible and pragmatic.  But we certainly aren’t basing our business model on a blanket claim of being better than the establishment just because we come from the rationalist community.


Comments for this post are closed