Results for “school vouchers test scores” 15 found
This paper evaluates the Zones of Choice (ZOC) program in Los Angeles, a school choice initiative that created small high school markets in some neighborhoods but left traditional attendance zone boundaries in place throughout the rest of the district. We leverage the design of the program to study the impact of neighborhood school choice on student achievement, college enrollment, and other outcomes using a matched difference-in-differences de-sign. Our findings reveal that the ZOC program boosted test scores and college enrollment markedly, closing achievement and college enrollment gaps between ZOC neighborhoods and the rest of the district. These gains are explained by general improvements in school effectiveness rather than changes in student match quality, and school-specific gains are concentrated among the lowest-performing schools. We interpret these findings through the lens of a model of school demand in which schools exert costly effort to improve quality.The model allows us to measure the increase in competition facing each ZOC school based on household preferences and the spatial distribution of schools. We demonstrate that the effects of ZOC were larger for schools exposed to more competition, supporting the notion that competition is a key channel driving the impacts of ZOC. In addition, demand estimates suggest families place a larger weight on school quality compared to peer quality, providing schools the right competitive incentives. An analysis using randomized admission lotteries shows that the treatment effects of admission to preferred schools declined after the introduction of ZOC, a pattern that is explained by the relative improvements of less-preferred schools. Our findings demonstrate the potential for public school choice to improve student outcomes while also underscoring the importance of studying market-level impacts when evaluating school choice programs.
Here is a link to the paper, note that Christopher is on the job market this year from UC Berkeley.
Diether W. Beuermann and C. Kirabo Jackson report:
Recent studies document that, in many cases, sought after schools do not improve student test scores. Three explanations are that (i) existing studies identify local average treatment effects that do not generalize to the average student, (ii) parents cannot discern schools’ causal impacts, and (iii) parents value schools that improve outcomes not well measured by test scores. To shed light on this, we employ administrative and survey data from Barbados. Using discrete choice models, we document that most parents have strong preferences for the same schools. Using a regression-discontinuity design, we estimate the causal impact of attending a preferred school on a broad array of outcomes. As found in other settings, preferred schools have better peers, but do not improve short-run test scores. We implement a new statistical test and find that this null effect is not due to school impacts being different for marginal students than for the average student. Looking at longer-run outcomes, for girls, preferred schools reduce teen motherhood, increase educational attainment, increase earnings, and improve health. In contrast, for boys, the results are mixed. The pattern for girls is consistent with parents valuing school impacts on outcomes not well measured by test scores, while the pattern for boys is consistent with parents being unable to identify schools’ causal impacts. Our results indicate that impacts on test scores may be an incomplete measure of school quality.
A neglected point…Arnold Kling, telephone!
There is a new NBER working paper by Richard J. Murnane, Marcus R. Waldman, John B. Willett, Maria Soledad Bos, and Emiliana Vegas. I have not had a chance to read it, but here is the key part of the abstract:
We found that:
1. On average, student test scores increased markedly and income-based gaps in those scores declined by one-third in the five years after the passage of SEP.
2. The combination of increased support of schools and accountability was the critical mechanism through which the implementation of SEP increased student scores, especially in schools serving high concentrations of low-income students. Migration of low-income students from public schools to private voucher schools played a small role.
We interpret these findings as more supportive of improved student performance than other recent research on the Chilean policy reform.
That is not exactly the Milton Friedman story, but it is essentially a positive report for vouchers.
Since Donald Trump has picked Betsy DeVos to be education secretary, many commentators have been pulling out their anti-school choice arguments from the closet, and for the most part it isn’t a pretty sight. To insist on a single government-run school and trash school choice, while out of the other side of one’s mouth criticizing Trump for “authoritarianism,” and other times proclaiming “Black Lives Matter” is from my point of view a pretty poor mix.
To be sure, we’re still not sure how well vouchers work, and I would suggest continuing experimentation rather than full-on commitment. Frankly, I find a lot of the voucher advocates unconvincing, but let’s not forget the single most overwhelming (yet neglected) empirical fact about vouchers: they improve parent satisfaction.
That result is not much contested. For instance:
Universally, school choice parents are highly satisfied with choice schools, reporting greater discipline, more responsive staff and better educational environments than the public schools they left. That parents are satisfied with their choice schools is a valuable indicator that school choice delivers real benefits. As University of Wisconsin professor John Witte, the official evaluator of the Milwaukee choice program, recently commented on school choice research: “There’s one very consistent finding: Parental involvement is very positive, and parental satisfaction is very positive…parents are happier. The people using vouchers are mostly black and Hispanic and very poor…they deserve the same kind of options that middle-class white people have.”
Patrick J. Wolf’s survey of twelve voucher programs (pdf) supports this interpretation. And here are strongly positive results on parental satisfaction Indiana. I could go on, but I don’t think there is much need.
Of course parents may like school choice for reasons other than test scores. To draw from the first link above, parents may like the academic programs, teacher skills, school discipline, safety, student respect for teachers, moral values, class size, teacher-parent relations, parental involvement, and freedom to observe religious traditions, among other facets of school choice.
Perhaps now is the time to remind you that how the buyers like the product is the fundamental standard used by economists for judging public policy? That is not to say it is the final standard all things considered, but surely economists should at least start here and report positive parental satisfaction as a major feature of school choice programs. In fact, I’ll say this: if you’re reading a critique of vouchers and the critic isn’t willing to tell you up front that parents typically like this form of school choice, I suspect the critic isn’t really trying to inform you.
To be sure, you still might not favor school vouchers. You might think they cost too much, you might think they will politicize private schools too much, or you might think they weaken national unity too much, to cite a few possibilities. (Although please, on that latter matter you can’t just say something silly like “public schools and the army made America what it is today.” You need some actual evidence. Won’t parents who are happy with the schooling of their children also contribute to national unity and push us away from polarization? That effect might outweigh whatever more negative mechanism you have in mind. Evidence please, not just sentiment.)
And as for test scores, the evidence there is still unclear. Here are a few earlier MR posts, no cherry- or lemon-picking, please.
Scott Alexander has some excellent comments on vouchers and school choice.
Fisman doesn’t cite any plausible mechanism through which private schools could have dragged down the test scores of the 86 percent of Swedish pupils who attend public schools. The explanation cannot be that private schools have drained public schools of resources, as private schools on average get 4 percent less funding than public schools.
One-third of Sweden’s municipalities still have no private schools. Social Democratic strongholds in northern Sweden in particular were less enthusiastic about licensing such institutions, and if private schools were causing the Swedish school crisis, we would expect municipalities with no privatization to outperform the rest of the country. Two studies by Böhlmark och Lindahl suggest that school results, if anything, fell more in regions with no private schools.
He also argues:
…in my view, the main culprit was the experiment with radically new pedagogical methods. The Swedish school system used to rely on traditional teaching methods. In recent decades, modern “individualist” or “progressive” pedagogic ideas took hold. The idea is that pupils should not be forced to learn using external incentives such as grades, and children should take responsibility for their own learning, driven by internal motivation. Rote memorization and repetition are viewed as old-fashioned relics. Teacher-led lectures have increasingly been replaced by group work and “research projects.”
The private Swedish schools are not really allowed to innovate where it matters, with their pedagogic methods. The curriculum and rules in the classroom are determined by the state, which also trains teachers in the so called “modern” pedagogic theories. “Swedish schools have comparatively low levels of autonomy over curricula and assessments,” PISA notes.
In practice, what private Swedish schools have control over is management and cost control, and this is where they have directed their efforts. But since the public Swedish schools were pretty well managed to start with, productivity gains from privatization were limited.
Interesting op-ed in the Washington Post on schools in New Orleans.
…the levees broke and the city was devastated, and out of that destruction came the need to build a new system, one that today is accompanied by buoyant optimism. Since 2006, New Orleans students have halved the achievement gap with their state counterparts. They are on track to, in the next five years, make this the first urban city in the country to exceed its state’s average test scores. The share of students proficient on state tests rose from 35 percent in 2005 to 56 percent in 2011; 40 percent of students attended schools identified by the state as “academically unacceptable” in 2011, down from 78 percent in 2005.
….Most of the buzz about the city’s reforms focuses on the banishment of organized labor and the proliferation of charter schools, which enroll nearly 80 percent of public school students, up from 1.5 percent pre-Katrina. But what really distinguishes New Orleans is how government has redefined its role in education: stepping back from directly running schools and empowering educators to make the decisions about hours, curriculum and school culture that best drive student learning. Now, state and school-district officials mostly regulate and monitor — setting standards, ensuring equity and closing failing schools. Instead of a traditional school system, there is a system of schools in what officials liken to a fenced-in free market. Families have more choice about where their children can best succeed, they say, and educators have more opportunity to choose a school that best aligns with their approach.
The population of New Orleans changed pre and post-Katrina so it’s difficult to compare pre and post-Katrina test scores; although given the state of the schools pre-Katrina it’s hard to believe that the schools have not greatly improved. What really drives innovation, however, is not a simple substitution of private for public but a system substitution of competition for monopoly. The key therefore is to expand charters and voucher programs.
The state of Louisiana just passed a voucher program that although limited to poor and middle class students in failing schools will offer as many as 380,000 vouchers to be used at private schools or apprenticeships. Indiana has passed a potentially even larger program that would make about 500,000 students voucher-eligible. Keep in mind that at present there are 50 million public school students and only 220,000 voucher students nationwide.
My ideal program would fund students not schools and would make vouchers available to all students on a non-discriminatory basis. We are far from that ideal but we are slowly moving in the right direction. Charters and the expansion of voucher programs around the country are starting to bring more competition, dynamism and evolutionary experimentation to the field of education.
Many studies of education vouchers have looked at the achievement of children who are given vouchers and who transfer to private schools. Generally these studies have found small but meaningful improvements (e.g. here and here). A voucher program, however, is about much more than transferring students from lousy public schools to better private schools it’s about creating incentives to improve the public schools.
Florida’s Opportunity Scholarship Program rated schools. Students at schools that received an F in multiple years became eligible for a voucher that allowed them to attend a private or higher-rated public school. In Feeling the Florida Heat? (ungated version) a paper sponsored by the Urban Institute Rouse et al. look at what happened at failing schools.
…we find that schools that received a grade of “F” in summer 2002 immediately improved the test scores of the next cohort of students, and that these test score improvements were not transitory, but rather remained in the longer term. We also find that “F”-graded schools engaged in systematically different changes in instructional policies and practices as a consequence of school accountability pressure, and that these policy changes may explain a significant share of the test score improvements (in some subject areas) associated with “F”-grade receipt.
Thus, this paper shows two things. First, that the test scores of the students in the public schools improved when vouchers gave the schools better incentives to perform. Second, at least some of the improvement comes from changes in how students are taught. The author’s note, for example:
…we find that schools receiving an “F” grade are more likely to focus on low-performing students, lengthen the amount of time devoted to instruction, adopt different ways to organize the day and learning environment of the students and teachers, increase resources available to teachers…
It is not true that "nothing can be done to improve the schools." Incentives matter.
Notice that Florida’s program worked even though the program was very weak. It offered vouchers only to students in the worst schools and only after those schools received F grades in multiple years. The vouchers were relatively small and could not be topped up. In addition, the program lasted only a few years before it was declared unconstitutional by Florida’s supreme court.
A true voucher program would be national, would not discriminate among students, would offer funding equal to that spent on students in public schools and would be permanent. Competition in such a system would be more intense and even more productive than in Florida’s program.
Is Caroline Hoxby’s defense of school vouchers hopelessly flawed in its data work? I doubt it, but I have not been following this debate. Here is the recent and excellent WSJ article on the spat; the link does not require a subscription. Excerpt:
Five years ago Harvard’s Caroline Hoxby, a rising star in economics, wrote a paper that reached an unusual conclusion: Cities with more streams tended to have schools with higher test scores.
Today her work is a widely cited landmark in the fierce national debate over free-market competition in public schools. And it’s at the center of a bitter dispute with another economist that is riveting social scientists across the country.
Her adversary is Jesse Rothstein, a young professor at Princeton, who says her study is full of flaws. In a rebuttal to her critic, Dr. Hoxby wrote of his work: "Every claim is wrong." She has also accused him of ideological bias. Dr. Rothstein, in turn, says she resorts to "name-calling" and "ad hominem attacks" on him.
Tyler mentioned, following a depressed Brad DeLong, a new paper on education vouchers in Chile that does not find large achievement gains. I have some criticisms of the paper (see below) but I was surprised that neither mentioned the most important recent paper on vouchers, Vouchers for Private Schooling in Colombia by Angrist, Bettinger, Bloom, King and Kremer in the Dec. 2002 AER.
Using data from a randomized experiment, Angrist et al. estimate that attending private school increased the probability of finishing eighth grade by 13-15 percentage points or 25 percent. Test scores increased by .29 standard deviations which is equivalent to about an extra year’s worth of schooling which has been estimated to increase yearly wages by 10 percent. Other markers such as teen cohabitation also improved.
Is this just a case of dueling papers? No, first, unlike Hsieh and Urquiola (HU), the Angrist et al. results are consistent with results found elsewhere. See in particular those found for Catholic schooling in the United States . Second, Hsieh and Urquiola (HU) are good researchers, judging by their paper, but Angrist et al. have a much more convincing research design – results from a randomized trial beat econometric identification any day. Cheer up Brad!
I shouldn’t give the impression that the results are directly comparable, however, as HU are trying to get at the general equilibrium effect of a voucher experiment and Angrist et al. are after the partial equilibrium effect of private schooling. Given the large gains found in the partial equilibrium literature, however, the GE results from HU are not plausible in my view.
Now regarding the HU paper some information is in order. First, there were no vouchers in Chile. Instead, there was public funding of some private schools on a per-student basis. Parents could not apply their voucher to the tuition at a private school of their choice.
Second, HU do not test whether students who transferred to private schools did better than other students – they tested whether aggregate scores (public and private) increased over time as more students attended private schools. Their evidence seems consistent with a nationwide decline in public school quality over time. More generally, I would have liked to have seen some information in their paper on the power of their tests. Given the size of the private sector what sort of gains could would we have expected to see in the aggregate scores and is their technique powerful enough to pick up such gains?
Third, HU claim that “cream skimming” was extensive but I find this difficult to believe because there is no price difference between public and private (voucher-accepting) schools since each was paid the same per-student amount. There are some non-pecuniary barriers but no limits on entry that HU mention.
Fourth, why did private enrollment increase if parents did not perceive a quality improvement? HU mention “freshly painted walls” which I thought was a bit flip – we ought to take revealed preference more seriously.
I do think that the HU study of Chile provides useful information about designing a good voucher program and my priors would have been that the program instituted in Chile, even though not a true voucher program, would have produced a larger effect – thus I learned something from the paper.
Co-blogger Alex and I had been having a debate over school vouchers, here is Alex’s last word, with links to the debate and my earlier posts, click here and here. I am skeptical about vouchers, although not from an anti-market point of view. We have seen from the electricity and water sectors that mixed public-private systems often create bad incentives, and do not always improve performance.
Brad Delong now cites NBER research (the paper itself costs $5) that school vouchers have not improved educational performance in Chile.
Here is a quotation from the paper:
In 1981, Chile introduced nationwide school choice by providing vouchers to any student wishing to attend private school. As a result, more than 1,000 private schools entered the market, and the private enrollment rate increased by 20 percentage points, with greater impacts in larger, more urban, and wealthier communities. We use this differential impact to measure the effects of unrestricted choice on educational outcomes. Using panel data for about 150 municipalities, we find no evidence that choice improved average educational outcomes as measured by test scores, repetition rates, and years of schooling. However, we find evidence that the voucher program led to increased sorting, as the best public school students left for the private sector.
My take: I am still willing to experiment with vouchers, mainly because they would give many inner city kids a chance they don’t currently have. But sometimes I wonder how much schooling, in the formal sense, matters at all. The United States has mediocre schooling, by international standards, but still produces highly productive individuals. Maybe a school is really just a collection of kids, in which case you can only get so far by reshuffling the mix.
Addendum: Here is a version of the paper.
That is the topic of my latest Bloomberg column, here is one excerpt of some super-simple (but neglected) arguments:
Education is another area where Friedman’s ideas seem newly relevant. Friedman was a strong supporter of school choice, but over time the movement stalled, as a variety of studies showed scholastic gains from school-voucher programs that were either modest, zero or negative. Advocates for school choice then moved on to the argument that vouchers allow parents to choose the kind of education they want for their children, whether or not test scores go up. That argument, too, went nowhere.
Then came the pandemic, when millions of American parents encountered a public school system that didn’t seem to care too much about educating their children. Schools stayed closed or offered inferior remote instruction, and generally followed their own bureaucratic imperatives. All of a sudden, home schooling, charter schools, private schools, micro-schools — in short, an entire host of “school choice” alternatives — rose in popularity. It remains to be seen how much those trends will stick, but Friedman may yet win this intellectual battle, at least partially.
And it’s not just the bureaucracy, it’s what’s taught in the classroom. Consider critical race theory and other instructional practices affiliated with wokeism. Whatever your views on this movement, it seems clear that it provokes strong and perhaps irresolvable differences among parents, teachers and administrators. Within a single public school district, those matters will probably never be settled to everyone’s satisfaction. Rather than pursuing a polarizing “fight to the death,” perhaps all sides can see that the case for school choice is stronger and more compelling than they had thought.
There are periodic attempts to knock Milton Friedman off his pedestal. For the most part, however, his legacy remains strong.
And who was the guy who predicted the recent problems with the FDA?
Intestinal helminths—including hookworm, roundworm, whipworm, and schistosomiasis—infect more than one-quarter of the world’s population. Studies in which medical treatment is randomized at the individual level potentially doubly underestimate the benefits of treatment, missing externality benefits to the comparison group from reduced disease transmission, and therefore also underestimating benefits for the treatment group. We evaluate a Kenyan project in which school-based mass treatment with deworming drugs was randomly phased into schools, rather than to individuals, allowing estimation of overall program effects. The program reduced school absenteeism in treatment schools by one-quarter, and was far cheaper than alternative ways of boosting school participation. Deworming substantially improved health and school participation among untreated children in both treatment schools and neighboring schools, and these externalities are large enough to justify fully subsidizing treatment. Yet we do not find evidence that deworming improved academic test scores.
If you do not today have a worm, there is some chance you have Michael Kremer to thank!
With Blanchard, Kremer also has an excellent and these days somewhat neglected piece on central planning and complexity:
Under central planning, many firms relied on a single supplier for critical inputs. Transition has led to decentralized bargaining between suppliers and buyers. Under incomplete contracts or asymmetric information, bargaining may inefficiently break down, and if chains of production link many specialized producers, output will decline sharply. Mechanisms that mitigate these problems in the West, such as reputation, can only play a limited role in transition. The empirical evidence suggests that output has fallen farthest for the goods with the most complex production process, and that disorganization has been more important in the former Soviet Union than in Central Europe.
Kremer with co-authors also did excellent work on the benefits of school vouchers in Colombia. And here is Kremer’s work on teacher incentives — incentives matter! His early piece on wage inequality with Maskin, from 1996, was way ahead of its time. And don’t forget his piece on peer effects and alcohol use: many college students think the others are drinking more than in fact they are, and publicizing the lower actual level of drinking can diminish alcohol abuse problems. The Hajj has an impact on the views of its participants, and “… these results suggest that students become more empathetic with the social groups to which their roommates belong,.” link here.
And don’t forget his famous paper titled “Elephants.” Under some assumptions, the government should buy up a large stock of ivory tusks, and dump them on the market strategically, to ruin the returns of elephant speculators at just the right time. No one has ever worked through the issue before of how to stop speculation in such forbidden and undesirable commodities.
Michael Kremer has produced a truly amazing set of papers.
San Francisco, population 865,000, has roughly the same number of dogs as children: 120,000. In many areas of the city, pet grooming shops seem more common than schools.
There is also this:
San Francisco’s public school system has around 53,000 students, a sharp drop from 90,000 in 1970.
The decline is a reflection both of families leaving the city and wealthier parents sending their children to private schools. Around 30 percent of San Francisco children attend private school, the highest rate among large American cities.
More than 10 private schools have opened in San Francisco since 2009, according to a tally by Elizabeth Weise, a journalist who writes a blog on the subject.
What percentage of those people also support the idea of further experimentation with school vouchers? And do they require proof that private schools will raise their children’s test scores, before becoming emotionally committed to sending them there?
Neighborhoods and Academic Achievement: Results from the Moving to Opportunity Experiment
Families originally living in public housing were assigned housing vouchers by lottery, encouraging moves to neighborhoods with lower poverty rates. Although we had hypothesized that reading and math test scores would be higher among children in families offered vouchers (with larger effects among younger children), the results show no significant effects on test scores for any age group among over 5000 children ages 6 to 20 in 2002 who were assessed four to seven years after randomization. Program impacts on school environments were considerably smaller than impacts on neighborhoods, suggesting that achievement-related benefits from improved neighborhood environments are alone small.
Here is the paper, and thanks to Sven Wilson for the pointer.
Long-time readers will recall my discussion of Vouchers for Private Schooling in Colombia by Angrist, Bettinger, Bloom, King and Kremer in the Dec. 2002 AER. The paper is especially important because it uses data from a randomized experiment.
Angrist et al. estimate that attending private school increased the probability of finishing eighth grade by 13-15 percentage points or 25 percent. Test scores increased by .29 standard deviations which is equivalent to about an extra year’s worth of schooling which has been estimated to increase yearly wages by 10 percent. Other markers such as teen cohabitation also improved.
Angrist, Bettiner and Kremer are back with a follow-up study that looks at high-school graduation rates and test scores on college-entrance exams.
The results of our follow-up study point to lasting benefits for voucher winners, with substantially higher high school graduation rates and, after adjusting for selection bias, higher test scores among those who took the ICFES exam [a college entrance test, Alex]….The size and persistence of the impact suggests PACES was a cost-effective intervention … there is substantial economic return to high school graduation in Colombia.