The editor of this truly excellent book is Timothy N. Ogden, the subtitle is Perspectives on Randomized Trials in Development Economics, and the contributors include Angus Deaton, Dean Karlan, Lant Pritchett, David McKenzie, Judy Gueron, Rachel Glennerster, Chris Blattman, and yours truly, with a focus on randomized control trials and other experiment-related methods. Here is one bit from the interview with me:
I would say that just about every reputable RCT has shifted my priors. Literally every one. That’s what’s wonderful about them, but it’s also the trick. You might ask, “why do they shift your priors?” They shift your priors because on the questions that are chosen, and ones that ought to be chosen, theory doesn’t tell us so much. “How good is microcredit?” or “What’s the elasticity of demand for mosquito nets?” Because theory doesn’t tell you much about questions like that, of course an RCT should shift your priors. But at the same time, because theory hasn’t told you much, you don’t know how generalizable the results of those studies are. So each one should shift your priors, and that’s the great strength and weakness of the method.
Now, you asked if any of the results surprised me. I think the same reasoning applies. No, none of them have surprised me because I saw the main RCT topics to date as not resolvable by theory. So they’ve altered my priors but in a sense that can’t shake you up that much. If you offer a mother a bag of lentils to bring her child in to be vaccinated, how much will that help? Turns out, at least in one part of India, that helps a lot. I believe that result. But 10 years ago did I really think that if you offered a mother in some parts of India a bag of lentils to induce them to bring in their kids to vaccination that it wouldn’t work so well? Of course not. So in that sense, I’m never really surprised.
One of my worries is RCTs that surprise some people. Take the RAND study from the 1970s that healthcare doesn’t actually make people much healthier. You replicate that, more or less, in the recent Oregon Medicaid study. When you have something that surprises people, they often don’t want to listen to it. So it gets dismissed. It seems to me that’s quite wrong. We ought to work much more carefully on the cases where RCTs are surprising many of us, but we don’t want to do that. So we kind of go RCT-lite. We’re willing to soak up whatever we learn about mothers and lentils and vaccinations, but when it comes to our core being under attack, we get defensive.
I very much recommend the book, which you can purchase here. Interviews are so often so much better than just letting everyone be a blowhard, and Ogden did a great job.