Caroline Hoxby is mad, and rightly so. In August, the American Federation of Teachers released a study attacking charter schools because charter school students performed worse than their public school “peers.” The study got huge media attention, including a front page article and editiorial in the NYTimes, despite the fact that it is not a very good study – lagging far behind its peers in the academic literature.
The main problem is that the study doesn’t do a very good job at comparing peers. The most credible studies look at the achievement differences between randomly assigned students (as did the study on private schools in Colombia I discussed earlier). When charter schools are over-subscribed (which often occurs – a sure sign that parents think they are superior to more traditional public schools) students are sometimes selected by lottery. Using data on randomly assigned students in Chicago, Hoxby and co-author Jonah Rockoff find significant achievement gains for the charter school students (paper, executive summary). (Surprise! When given the opportunity, parents can pick good schools.).
Another problem with the AFT study is that it uses a relatively small sample, about 3% of charter students in the fourth and eight grades. In another paper, Hoxby examined tests from 99% of 4th grade charter students. It’s not possible to use a randomized study when you look at nearly all charter school students so instead Hoxby compares charter students to students in the nearest regular public school and the nearest regular public school with a similar racial composition. For the latter comparison she found that charter students were 5% more likely to to be proficient in reading and 2.8% more likely to be proficient in math – small but meaningful improvements. And in places where the regular public schools are especially bad, like Washington DC, charter students were about 36% more likely than their peers to be proficient in reading and math!
Despite the fact that Hoxby’s studies are of far higher quality than those of the ATF and other groups you don’t see her work trumpeted across the front page of the NYTimes. And it’s not as if Hoxby isn’t well known, she is a Harvard professor whom several years ago The Economist listed as one of the best young economists in the world. As Brad DeLong might ask in another context, Why can’t we have a better media?
I have drawn from an op-ed by Hoxby in the Wall Street Journal from Wed. Sept. 29, 04 (sorry I don’t have the link).