A new RCT look at educational vouchers

by on August 24, 2012 at 3:32 pm in Data Source, Education | Permalink

From Matthew M. Chingos and Paul E. Peterson (pdf):

In the first study using a randomized experiment to measure the impact of school vouchers on college enrollment, we examine the college-going behavior through 2011 of students who participated in a voucher experiment as elementary school students in the late 1990s. We find no overall impacts on college enrollments but we do find large, statistically significant positive impacts on the college going of African American students who participated in the study. Our estimates indicate that using a voucher to attend private school increased the overall college enrollment rate among African Americans by 24 percent.

Hat tip goes to Michael Petrilli, via ModeledBehavior.

1 dearieme August 24, 2012 at 3:49 pm

Surely vouchers ought to be reckoned a success if they result in 18-year-olds being better educated. To assume that that corresponds to more going to college seems to me to be unwarranted.

2 derek August 24, 2012 at 4:01 pm

I would guess that all else being equal a certain percentage of students will go on to college, and we would see those numbers represented in white students. I would also guess that african american students would be under represented in the numbers who go on to college, and these voucher programs make all else equal, hence the increase in those students going on.

There is an indictment of someone in these numbers.

3 dearieme August 24, 2012 at 5:21 pm

No, no, I wasn’t commenting on the black students: good luck to them. It’s the “overall impacts” that I was querying: it’s not obvious to me that more students going to college must be uncritically accepted as a good thing.

Still, if “no overall impacts” were observed, but more black students are going to college, which racial group is sending fewer to college? And am I meant to think that a good thing, a bad thing, or just one of those things?

4 Edward Burke August 25, 2012 at 9:41 am

Then of course most students of whatever color or background arriving at the gates of any American college or university with crisp high school diplomas in hand will have to enroll in post-secondary English and/or math remediation . . . .

5 gwern August 24, 2012 at 4:30 pm

If the RCT had found that vouchers decreased college attendance (perhaps the schools ripped them off or used harsh methods etc), would you be here solicitously commenting that ‘surely they ought to be a success only if they result in being better educated, to assume that fewer going to college means that seems to be to be unwarranted’?

6 Kyle August 24, 2012 at 5:06 pm

Interesting that it is phrased as “using a voucher” rather than “being awarded a voucher.” I have not read the study, but I would wonder whether it isnt measuring hidden variables, and what the difference within the selected group between those who chose to use or not use the vouchers was.

7 Andrew August 24, 2012 at 5:46 pm

Kyle,

I just checked the study. They used instrumental variables estimation with “being awarded a voucher” as an instrument, and instrumented for the number of years that the voucher was used. So your concern about measuring the impact of hidden variables (endogeneity) should be allayed. On the other hand, some might wonder who the “LATE compliers” are–but this is a question of which subpopulation the voucher effect applies to, not whether this effect is measuring something other than the effect of the voucher.

It looks like a well-designed study and I’m looking forward to reading it further, especially after some peer review.

8 Kyle August 24, 2012 at 10:35 pm

Thanks for reading the document. I am not an economist and don’t have my copy of mostly harmless econometrics handy. That said, unless I misunderstand you (which is likely), I’m not sure my concerns are allayed.

Being awarded a voucher was random. The decision to use it, once awarded, was not. It seems likely to me that those who chose to use vouchers more years likely came from families that placed more value on education, or that felt that education could help their child more, as compared with families who were awarded vouchers and didn’t use them.

If the study said that just being awarded vouchers made African Americans more likely to go to college (and maybe it does, but that isn’t how the excerpt reads), that would make for a much different conclusion than saying their use of vouchers leads to enrollment. IMO, the latter doesn’t seem that surprising, because voucher use could be a proxy for high family value of education (or whatever).

Now, maybe what you just said is the regression methodology corrected for this, but I was just too stupid to understand how. That seems likely.

9 Kyle August 24, 2012 at 10:41 pm

Ok, I read the study, and it does say that merely offering a voucher leads to increased enrollment. Use of that voucher further increases enrollment. I would guess you can attribute the first part to the positive impact of private schools and the second part to self selection by high value beneficiaries. From the study:

“A voucher offer is shown to have increased the overall (part- time and full-time) enrollment rate of African Americans by 7.1 percentage points, an increase of 20 percent. If the offered scholarship was actually used to attend private school, the impact on African American college enrollment is estimated to be 8.7 percentage points, a 24 percent increase.”

10 Andrew August 25, 2012 at 12:29 am

Not stupid at all… but instrumental variables does correct for the problem you describe. It takes the effect from the random part only (the award) and scales it up to reflect the effect of attendance rather than the award itself. Briefly, here’s how it works. Say the award effect was 0.4. But say the award only induced 1/2 of the awardees to attend, so that 0.4 effect had to have come only from that 1/2 of awardees. The instrumental variables approach just takes that 0.4 and multiplies it by 2, giving you a 0.8 average effect per attendee. This is the impact of attendance, totally free of the selection problem you worry about, but it’s just for the kind of person who would be induced to attend a private school by a voucher. So the concern isn’t a selection problem, it’s that the effect is for the kind of student who will actually take advantage of a voucher. In this case, that’s a useful subpopulation to be looking at.

11 Heedless August 26, 2012 at 1:03 am

I wish they had included a measure for graduating college, as I consider that a much better measure of educational quality (in both the colleges themselves and in grades 1-12). After all, the only requirements for entering college are a GED and a willingness to take out a loan.

12 Other Kyle August 24, 2012 at 6:34 pm

I am wondering, isn’t the process of “using a voucher” to attend a private school akin to skimming from the crop? In other words, “problem” students from public schools that demonstrate limited aptitude for education or test-taking are unlikely to receive a voucher, as the private school is looking to protect its reputation and sterling educational outputs (i.e., test results, etc.) If this is the case, wouldn’t it be a fair assumption to make that the African-American students that excel in the public school environment and make good selections for private-school vouchers have the intelligence and aptitude to attend college?

I’m just trying to understand what the point is being measured here. Would it be safe to assume that *everyone* who attends a private school has a better chance to attend college, and not just African-Americans? Or is it more of a representation that traditionally, bright African-American students would be lost in underfunded, low-performance public schools?

13 Willitts August 24, 2012 at 9:35 pm

Let’s assume that your skimming theory is true.

It still means that black students who had school choice and, ostensibly, chose to be among brighter than average peers were more likely to attend college.

The vouchers separated the wheat from the chaff. Don’t overlook the fact that some students might be held back from their full potential by dumber than average or more unruly than average peers.

Vouchers enabled people who can escape to do so. And that isn’t even assuming the private schools are any better at teaching. My kids go to Catholic school and I wouldn’t have it any other way. And I was the product of the New York City public school system.

14 joshua August 24, 2012 at 6:37 pm

Does this mean Democrats will warm up to school vouchers?

15 Jayson Virissimo August 25, 2012 at 3:48 am

Unlikely; minimum wage laws disproportionately harm blacks, but Democrats aren’t about to stop advocating them.

16 Edward Burke August 25, 2012 at 9:51 am

–which raises a larger issue of “injurious amelioration” working throughout our political economy: “helpful” minimum wage laws hurt the poor, ditto for public education. What other “remedies” and palliatives advertised as addressing the plight of the downtrodden consistently and pervasively do more harm than good? (I haven’t had drop one of coffee yet this morning.)

17 economist1 August 26, 2012 at 2:07 am

Evidence? You need to show that minimum wages in the USA both produce significant unemployment (Evidence is highly mixed) and that this doesn’t outweigh the huge benefit of higher wages to minimum wage earners. Good Luck….

18 Eric H August 26, 2012 at 11:18 pm

No, he just needs to show that minimum wages produce a differential between black and non-black potential employees. And also that Democrats would not stop their advocacy once confronted with that evidence.

19 allen August 25, 2012 at 9:01 am

The already have. In part if not in total and dividing the Democratic party along the lines of substantive education reform erodes one of the primary political impediments to reform.

To Jayson Virissimo’s point, much of the interest of Democrats in reform is driven by one of the consitutencies without which the Democratic party becomes nationally-irrelevant – blacks. Blacks support vouchers, charters, tax credits and anything else that might get their kids a decent education at a significantly higher rate then whites. Some Democrats have chosen to stick with the status quo, some not.

It would be interesting to see whether Democratic party support for vouchers divides along racial lines.

20 Charlie August 24, 2012 at 8:16 pm

If there is no change in students going to college, and a positive effect on black students going to college, does that mean their is a negative effect on non-black students going to college? Or is their a positive, but not statistically significant effect on all students? If the former is the case, I really have to rethink my mental model of education.

21 Willitts August 24, 2012 at 9:40 pm

It depends on what proportion of the total was black.

If they are a minority, then you can have a significant change in the subset without a significant change in the broader sample. The rest could have had declining rates of college because blacks took seats away from them in the better schools. Or The rates could have been unchanged or increased by an insignificant amount.

Comments on this entry are closed.

Previous post:

Next post: