RAND Hits Back

Joseph Newhouse and the other RAND researchers have responded to Nyman’s paper arguing that attrition bias biased their results.  The RAND researchers were aware of these issues and in fact designed the experiment to avoid incentives for non-random attrition.   Most importantly, the basic RAND findings have now been replicated in many other studies (smaller and not always experiments but the results are solid).  I call it a knockout for RAND.

It’s a credit to the many insightful commentators on Marginal Revolution that many of these points were made already in the comments on my original post.

Thanks to Jason Furman for the pointer.


Hurrah for the BLOGOSPHERE.

I guess I am just dull. If they were getting side payments to minimize the differences between the plans, what was the difference between the plans? Didn't this undermine the whole point?

Also, I have to say that at least here in DC they have a different method of reducing doctor visits. No family doctor that I have found will take new patients or schedule a visit in under 8 months. I have an excellent plan with a $5 copay, but the only people that are dredging for patients are the dentists!

If they were getting side payments to minimize the differences between the plans, what was the difference between the plans?

As I understand it, the participants in the cost-sharing plan got $1000 per year plus a plan with a $1000 annual out-of-pocket maximum, whereas the other group got a plan with no co-pays (or something like that). So if they both use a lot of medical care, there's no difference between the plans. But if they use no medical care, the participants in the cost-sharing group get $1,000 to take home, and the people in the other group get nothing. So there's still an incentive to reduce consumption of medical care.

One obvious explanation for why attrition rates were so different is that there was a significant percentage of people who found, after enrolling, that they just didn't like being in a cost-sharing (co-pay) plan -- because they had to think about how much they were paying for every visit or treatment, etc. -- and wanted to go back to a more traditional insurance scheme. This is hardly surprising: it's obviously easier to be in a traditional insurance plan rather than have to shell out $20 (or whatever) every time you go to the doctor. Cost-sharing plans involve more work than traditional insurance, so all things being equal, you'd expect more to drop out of the former than the latter.

That kind of attrition, though, would not change RAND's broader conclusion, which had nothing to do with how much people did or didn't like cost-sharing, but instead argued that cost-sharing did reduce consumption of health care, and that this did not have an impact on overall health outcomes (people in the cost-sharing plan ended up as healthy as people in the traditional plan, even though they consumed less health care).

@K. Williams,

But if a participant soon realized they would likely out spend their 'bonus' then they'd leave the study.

Comments for this post are closed