How much do charter schools really matter?

by on December 7, 2012 at 3:44 am in Education | Permalink

I’ve seen so many people discuss this topic, but Yusuke Jinnai seems to be making progress on the question.  Here is part of his abstract:

In this paper, I propose a new empirical approach to identify the impact of charter schools on local traditional schools. Specifically, I define direct impact as the effect of introducing charter schools on traditional-school students in grades that overlap with charter schools’ grades, while indirect impact is defined as the effect on students in non-overlapping grades. Unlike prior research work, which estimates the effects of charter school entry at the school level, I examine the impact at the grade level by exploiting the variation in gaps between grades offered by charter schools and grades at nearby traditional schools.

Using student-level panel data from North Carolina, this paper shows that the introduction of charter schools does not induce any significant indirect impact but generates a positive and significant direct impact on student achievement. Distinguishing between the two distinct impacts and taking into consideration both traditional-school and charter-school students, my study finds overall positive effects of introducing charter schools on student achievement. I also demonstrate that such overall effects would have been underestimated by 85% in the literature, since previous work identifies the impact of charter school entry at a moment when the direct and indirect impacts are likely to be mixed.

Finally, I argue that the direct impact consists of student sorting effects and competitive effects and, by controlling for unobserved peer characteristics, demonstrate one-quarter of the positive direct impact is driven by the former while three-quarters result from the latter.

The paper is here, and Yusuke is on the job market from Rochester this year.  His entire portfolio of papers on education appears to be quite interesting.

Here is a related post on school choice in Sweden, from Modeled Behavior.

prior_approval December 7, 2012 at 6:38 am

I’m curious if general director Cowen has any opinion about such FCPS schools as Thomas Jefferson (anything but a mediocre institution), George Mason High School, George C. Marshall High School, McLean High School., or even W.T. Woodson High School (which is quite literally just down Main Street from GMU).

Because I doubt that a charter school would ever have anywhere near the claim on resources that the above named schools do, not to mention the institutional connections to the entire sprectrum of American higher education, where, it is sadly true, GMU is at best a bit rate school for those unable to meet the higher standards which Fairfax County strives for.

And strangely, that is exactly what the tax payers of Fairfax County expect – some of the nation’s finest high schools, available for their children’s education, able to attend institutions f higher learning which make GMU look like the commuter school it remains.

Something that may only be apparent in the future, when sending a child to some of the finest public schools, not only in the nation, but the world. An option which is still open, assuming that general director Cowen lives in Fairfax County, a municipality notably lacking anything resembling a charter school.

One might just wonder why that is.

Andrew' December 7, 2012 at 8:21 am

This is even less understandable than that abstract!

prior_approval December 7, 2012 at 9:33 am

Fairfax County, which I believe is where both general director of the Mercatus Center Cowen and research director of the Independent Institute Tabarrok (just a guess – I’m not that interested in where Tabarrok lives, to be honest, not even to the extent of typing his name into google and reading the results from any of the commercial private data providers so common in the U.S.), has no charter schools, and no interest (that I’m aware of) in them.

Which just might have something to do with the budget of Fairfax County Public Schools – let’s quote them – ‘The Fairfax County School Board has adopted the Fairfax County Public Schools (FCPS) FY 2013 Approved Budget of $2.4 billion’ ( http://commweb.fcps.edu/newsreleases/newsrelease.cfm?newsid=2010 )

Yep, that is a budget which uses the word billions, for a school system with ca. 181,536 students covered by that budget.

Charter schools are just an excuse to poorly fund public schools, something that the residents of Fairfax County simply don’t accept.

But the residents of Fairfax County also don’t accept the idea of sharing revenue with poorer school systems. Which is one of the reasons that the Commonwealth of Virginia has the 3rd highest inequality rate for per student funding in the nation (for the non-PDF reading readership – http://webcache.googleusercontent.com/search?hl=de&q=cache:nhxHwBNg1zEJ:http://cte.rockhurst.edu/s/945/images/editor_documents/content/PROJECT%2520INEQUALITY%2520STUDENT%2520PAPERS(Listed%2520Alphabetically%2520by%2520P/cordes.pdf%2Bvirginia+inequality+school+funding&gbv=1&ct=clnk )

In other words, general director Cowen has no apparent interest in actually describing the situation where he lives.

Tracy W December 7, 2012 at 10:54 am

Charter schools are public schools.

Michael December 7, 2012 at 1:16 pm

In a de jure sense they are, but in a de facto sense, they can behave quite a bit like private schools. Many have selective acceptance and strongly pressure parents to pay a “donation” that looks and acts very much like tuition at a private school.

Bender Bending Rodriguez December 10, 2012 at 9:12 am

Replying to Michael, not Tracy W.

In a de jure sense they are, but in a de facto sense, they can behave quite a bit like private schools. Many have selective acceptance

Most states have laws that prohibit selective acceptance in charter schools. I was also under the impression that the Feds weren’t allowing selective admission in charter schools via NCLB. Do you have any anecdata to share?

Andrew' December 7, 2012 at 12:11 pm

So, are you saying they shouldn’t be lavishly wasting that much money or that their money should be taken and spread around the state or country so that people don’t have to look for any innovative alternatives to the public school status quo?

prior_approval December 7, 2012 at 12:47 pm

Fairfax County’s schools are some of the finest available in the entire world – which is why no one, to my knowledge, is talking about charter schools there. The budget is just a clue why that is – or do any commenter here honestly think that the massive resources available to the student body of McLean high school could be matched by a charter school? Or the resouces of TJ? (after my time admittedly – Pine Ridge was the superior learner school fo my youth) Or even second tier high schools, like Woodson or Annandale?

Because if so, well, I’m amused.

Andrew' December 7, 2012 at 1:04 pm

How much of the money will they get?

Using your numbers, $2.4B / 181,536 students / year = $13k / year

I think I could do damn well with about a quarter million a year for a class full of kids.

So, you are arguing that everyone should spend an astronomical amount that they can’t afford to avoid innovation?

prior_approval December 7, 2012 at 1:44 pm

‘So, you are arguing that everyone should spend an astronomical amount that they can’t afford to avoid innovation?’

Obviously, the citizens of Fairfax County think they can afford their public schools just fine, which may explain the lack of interest in charter schools. I’ll hazard a guess – of the richest 10 school districts in the entire U.S., there is not a single charter school to be found.

And I’ll further wager the citizens of those 10 richest school districts are eminently satisfied with the status quo. I’ll even hazard a further guess – none of those residents care the least about your education, or that of your chidren (potential or actual).

I know that that the results of my parent’s foresight have fully met their standards, even if I no longer live in North America. (I’m not kidding about the FCPS syste being among the best available in the entire world, by the way.)

Andrew' December 7, 2012 at 2:19 pm

Who have you asked if they think they can afford the schools?

If the teachers and parents are racking in obscene cash, of course they wouldn’t want to upset that apple cart.

Careless December 7, 2012 at 2:54 pm

PA has uncovered another conspiracy: people already happy with their existing public schools don’t try to get new, different schools opened.

Andrew' December 9, 2012 at 5:28 am

Even more nefarious! People happy with the taxpayer funding they get out of them don’t try to save taxpayers money in ways similar to people who have to because they don’t get the same amount!

Andrew' December 9, 2012 at 5:30 am

In other words, if you took this $13,000 per student per year and promised to give half (or all, or 1/3) of the savings to any parent who went to a cheaper school what would happen?

sourcreamus December 9, 2012 at 4:09 pm

According to test scores as compiled on schooldigger.com Fairfax ranks twentieth in Virginia, despite being the richest county in America. Whatever the reason for lack of charter schools is, it is not because Fairfax schools cannot be improved on.

Andrew' December 7, 2012 at 8:45 am

You expect universities to jack crap for anyone other than themselves?

I figured that one out when I learned my college sold twice as many parking passes as they had parking spaces.

prior_approval December 7, 2012 at 9:38 am

Well, I’m quite certain that GMU used to sell at least 3x times as many parking passes as spaces (and why yes, faculty and staff also had to pay, though the CUE bus was free), but then, GMU was a commuter school back in the 1980s and 90s.

Still is, actually, to the best of my knowledge. It just pretends to be a serious institution, sort of like how certain members of its faculty pretend to be more associated with GMU than what likely takes up the majority of their working day – something that was also apparent back in the 1980s and 1990s..

Ray Lopez December 7, 2012 at 10:41 am

You sound like a frustrated GMU reject. Did you go to NOVA? As for “director Cowen” and TJ high school, that is plainly wrong. The director of ‘Remember the Titans’ (yes, it’s based on TJ HS) is not Cowen but Boaz Yakin, and he has another movie out this year called “Safe”. You’re OUTTA HERE! Next.

prior_approval December 7, 2012 at 11:29 am

Oh no, my diploma from GMU comes from before when Prof. Cowen even arrived in the Econ dept.

And Tyler Cowen is the General Director of the Mercatus Center, which is all too amusingly located in Hazel Hall in Arlington (I can even remember when GMU didn’t even have an Arlington campus), Til Hazel being one of the titans that has made GMU what it is today.

My first pay check from GMU dates back to 1978, by the way – just six years after GMU split off from UVA.

Never had anything much to do with NOVA, I might add – not that there is anything wrong with NVCC in my extremely limited experience of its main campus on 236 (which I used to cut through to avoid light).

bluto December 7, 2012 at 11:49 am

I’m pretty sure it’s TC Williams in Alexandria, TJHS is the science and math magnet school that seems more likely to host a heartwarming story about races coming together to win an Intel Science Fair award, set up a cartel to sell perfect SAT scores, prank Caltech, or beat the house at blackjack.

Tracy W December 7, 2012 at 10:52 am

You assume that more resources means better schools. That’s not the case.

prior_approval December 7, 2012 at 11:37 am

In Fairfax County, that is a demonstrably false statement. Though I’ll admit the planetarium at WT Woodson (not to mention the 10,000 stadium) was a bit over the top – though watching Pele play there (and at RFK) remains a Northern Virginia memory of my youth.

Is there where I’m supposed to write something like go Cavs? Because really, varsity letter and all, I just don’t care anymore.

Andrew' December 7, 2012 at 2:49 pm

I’m not really following you. You point out a correlation and claim it proves causation. It’s really easy to propose a model where that correlation does not prove causation. People tend not to turn down free money. Why would they want a cheaper school if they get the extravagant school for free? They wouldn’t. On the other hand, people who don’t live near Washington D.C. and all the tax-funded institutions, for example, will look for ways to be efficient with resources.

Cliff December 7, 2012 at 12:09 pm

Thomas Jefferson actually receives fewer resources than other schools from the county and is in a (literally) crumbling structure. Of course it does get a lot of support from alumni so I’m sure it uses more resources overall, but surely you understand that resources don’t make that school what it is. You could devote the same resources to a D.C. school and you would have nothing.

And that’s the bottom line, resources have no effect on student performance. Just look at the D.C. school system. They get more money per pupil than NoVA with dramatically worse results. That’s a story repeated all over the country.

As for a lack of interest in charter schools here, obviously the reason is that the schools are doing well and the teacher’s unions steadfastly resist any charter schools anyway, so there is no impetus for change. I’m sure they would work just fine. Why wouldn’t they? This study and others suggest they would be an improvement. Which is why it is very puzzling to say charter schools are an “excuse” for spending less money. Who cares how much money gets spent?!? Results are what matter.

This is just another of your classic arguments by innuendo. Everyone involved in the side I don’t like must be evil! Let me cast dispersions on them instead of discussing the issue at hand!

prior_approval December 7, 2012 at 12:53 pm

‘This is just another of your classic arguments by innuendo.’

Except that the results of an education in Fairfax County’s schools are a matter of public record – and very few students from FCPS would ever consider stooping low enough to go to GMU (OK, my 1390 on the old style SATs was an exception – and what an exception – for a GMU student of a generation ago, though admittedly, this was post Karl Rove. And yes, the math section at 640 was my weak score.)

Andrew' December 7, 2012 at 2:22 pm

Again, I don’t understand what you are saying.

Rich people will spend more on education than they need to. Since they are rich, the got that way by being smart, on average. They’ll want to get the most out of the credentialling and signaling education system and any human capital improvements will be a nice bonus. And the parents and teachers’ unions who are benefitting from this irrationality won’t be interested in cheaper options any more than people getting a heart bypass will be interested in saving the insurance company money.

jeff December 7, 2012 at 9:17 am

someone should study the employment impact of having tyler cowen blog positively about your job market paper as a phd student

Brian December 7, 2012 at 10:34 am

I’ll be happy to help increase the sample size next year. Although I’m sure my advisor has widely circulated early drafts of my brilliant work already…

c December 7, 2012 at 9:26 am

Not sure what the previous comment was about, but I went to grad school with Yusuke, and he’s an excellent jm candidate, very friendly, intellectually curious guy. Glad to see his research getting play.

Bill December 7, 2012 at 9:31 am

Self-selection bias (note that students were not randomnly assigned to a charter school) and the ability to exclude more costly or underachieving students….guess what, I could achieve the same result in my garage.

And, many charters do. They literally are inventions of real estate developers who have stip mall, warehouse, or abandoned office space which are then renamed as serve as “College Prep” or “Hmong College Prep” etc. Parents who pay to keep their kids out of the “other” school, though, do welcome vouchers.

From what I’ve seen, controlled studies show no or little difference.

Tim Thomas December 7, 2012 at 9:32 am

Actually in Texas they’ve been shown to pretty much uniformly do worse and cost more. There are a couple good schools in the mix, but they’re the exception.

Michael December 7, 2012 at 1:01 pm

In some places, Charter schools aren’t any better, but because they have more ability to weed out the “bad” students, their averages are better. In other places, they may be good. In some cases, they’re just outright scams to transfer taxpayer money into private accounts. Those in the latter category probably won’t last as they’ll develop bad reputation, but I imagine someone could make a pretty decent sum of money before the scam is realized. I also imagine that such scams would be easier to pull off here in Texas where it doesn’t take much to convince people that public schools are liberal indoctrination institutions run by evil unions who force evolution, global warming, and diversity down the children’s throats.

Brent Buckner December 7, 2012 at 11:42 am

The “direct impact” in the linked study is “traditional-school students in grades that overlap with charter schools’ grades” (i.e. those who *don’t* go to the charter school).

Cliff December 7, 2012 at 12:14 pm

And is it just possible that separating underperforming and “overperforming” students could be good for both of them? That some students may learn more with a different style of teaching than others? That freeing teachers from trying to accommodate both types of learners and students from the fruitless prospect of competition with those far ahead of them might better take to the task?

Not saying for sure of course, maybe there would be expectations effects. But it certainly seems plausible.

Tim December 7, 2012 at 9:31 am

So he was able to figure out that the outcomes of charter schools are similar to those of private schools, both being schools that students self-select for? How insightful.

Even worse he’s saying the main benefit of charter schools are only achieved because of self-selection which means they’ll encounter diminishing returns.

Uninformed Observer December 7, 2012 at 10:11 am

But isn’t self-selection enough?

We’ve pretty well proven that you can’t produce good outcomes for students who don’t give a damn. If a charter school’s only barrier to entry is self-selection (unlike a private school, which can also use cost as a barrier), I’d say that’s a pretty sweet deal. You don’t have to be rich or the right color or come from the right family, you just have to put in a little effort. Sure, you can say selecting for merit or test scores is another barrier to entry, but that’s rather confusing the point, don’t you think?

Does eliminating self-selection improve outcomes? Historical examples?

DocMerlin December 7, 2012 at 10:29 am

+1

Bill December 7, 2012 at 5:44 pm

Uniformed,

Here is something, with footnotes, on self-selection sampling bias:

“Sampling bias

Sampling bias is systematic error due to a non-random sample of a population,[2] causing some members of the population to be less likely to be included than others, resulting in a biased sample, defined as a statistical sample of a population (or non-human factors) in which all participants are not equally balanced or objectively represented.[3] It is mostly classified as a subtype of selection bias,[4] sometimes specifically termed sample selection bias,[5][6] but some classify it as a separate type of bias.[7]

A distinction, albeit not universally accepted, of sampling bias is that it undermines the external validity of a test (the ability of its results to be generalized to the rest of the population), while selection bias mainly addresses internal validity for differences or similarities found in the sample at hand. In this sense, errors occurring in the process of gathering the sample or cohort cause sampling bias, while errors in any process thereafter cause selection bias.

Examples of sampling bias include self-selection, pre-screening of trial participants, discounting trial subjects/tests that did not run to completion and migration bias by excluding subjects who have recently moved into or out of the study area.”

Look up self-selection sampling bias in Wiki at http://en.wikipedia.org/wiki/Selection_bias

Footnotes have the history you crave.

Brent Buckner December 7, 2012 at 11:51 am

No, he’s not saying that at all. He’s looking at the impact of charter schools on the achievement of those students who attend the traditional schools.

Red December 7, 2012 at 10:29 am

Charter schools are the sole reason I choose to buy a house and move my family to Washington, D.C. The school – Washington Yu Ying – offers a bilingual Chinese program (one day fully in Chinese with native speaker teachers/one day in English. There is nothing that even comes close to this in the ‘burbs, even in Fairfax county.

Talking with other parents, I’ve learned I’m not the only one who moved to dc specifically for this school. So I’d say yes, charter schools offer immense educational benefits for the kids; they also may have secondary benefits in the real estate market and in raising the bar for all city schools generally.

prior_approval December 7, 2012 at 11:12 am

‘even in Fairfax county’

I’m not surprised – Fairfax county public schools still follow the old fashioned idea that it is their responsibility to educate students in English. Unlike some school systems, Fairfax has never been particularly trendy. Especially when it comes to any idea of idea of bilingual education – that is what weekends are for, for example, next to the Annandale volunteer fire department on Little River Turnpike for those learning Hebrew, if decades old memory serves. I don’t have any specific recollection of other bilingual education – it is just an example.

Cliff December 7, 2012 at 12:16 pm

God forbid you educate your students in two languages. That’s like, half the learning, right?

prior_approval December 7, 2012 at 12:58 pm

Well, my sister is a FCPS teacher who is paid more, due to her taking Spanish courses (fittingly enough, at GMU). But her Spanish skills are not used to increase the Spanish skills of her students (and not so ironically, she finds her Spanish speaking students to be much more willing to study and master English), but to help her teach her students better in English.

And yes, I still have some connections to the actual reality of Fairfax County schools. Schools that remain some of the best available in the world.

I recommend them for anyone who has children, to be honest.

Steve Sailer December 7, 2012 at 5:16 pm

How much of the increased competition -> increased test scores effect is really increased competition -> increased cheating?

Stephen December 7, 2012 at 5:59 pm

What do you guys think about this criticism from someone at a message forum I frequent?

“The data actually do not support any conclusions whatever about competitive effects. The author simply makes baseless unscientific conclusions from his shoddy methods.

What he did was to use a couple of highly limited, error filled measures of student characteristics. He threw them into the regression and examined the reduction in the positive effect. He then irrationally concluded that all variance not explained by his limited error filled control measures must be due to competitive effects. He has no measures of competitive effects themselves.
Thus, his conclusion rest on two definitively false assumptions:
1. That sorting effects and competitive effects are the only two possible sources of a positive improvement to traditional school achievement.
2. That his measures of sorting effects are error free and account for 100% of all variance in all possible ways in which the students could differ.

Unless both of these are true (and they are both very false), there is no basis whatever to infer that any of the 75% variance unexplained by his sorting measures is due to competitiveness effects. In fact, since he does have evidence that sorting effects impact the effect and has no evidence that competitive impacts the effect, it is far more rational to assume that the remaining unexplained 75% of the effect is due to untapped variance in sorting variables rather that competitiveness.

Basically, the author is committing the error that plagues much of economics research and other fields lacking any real experimental data and trying to use complex statistics to draw causal conclusions form purely correlational data.
That is to throw in some control measures of a variable they are trying to discount, then infer that all left over variance is due to their preferred but usually unmeasured causal variable. ”

http://www.freeratio.org/showpost.php?p=7341046&postcount=18

Keith December 7, 2012 at 8:11 pm

Stephen, that’s a very interesting critique, but as Prior Approval showed, you’re missing the two really important questions here:

1. What did you get on your SAT?

2. Did you know Tyler Cowen might live in Fairfax?

Ryan December 8, 2012 at 9:47 am

“By encouraging school choice, policymakers hope to induce more effort from public schools.”

Jinnai is overreaching to suggest that traditional public schools put in little effort. There is no discussion of how the effects he sees are reached, but there should be. The tendency is to teach to the test or narrow of curriculum (three hours of drill and kill in math and English, for example) should be considered before reaching a conclusion. The notion that traditional public schools are full of lazy teachers is a myth. Florida’s teacher evaluation has just concluded that the majority of its teachers are effective (74.6%) or highly effective (21.9%).

“However, this positive impact could be resulting not from competition but from student
sorting, because charter schools are likely to draw low-performing students in high-performing school districts, leaving relatively high-performing students at local traditional schools. In par- ticular, as illustrated in Table 3, parent education is higher and free-lunch eligibility is lower for charter-school students compared to traditional-school students. Thus, after the departure of low-performing students to charter schools, traditional-school students in overlapping grades benefit from better peers. In Section 7, I examine to what extent this positive direct impact can be explained by student sorting.”

Free lunch eligibility and parent education levels are higher in charter schools, so the charters are teaching harder to reach students? I don’t follow the logic here. It’s well established that traditional public schools teach “harder to reach students” than charters. Most charters have high attrition rates to ensure they have high graduation rates, so the departure of “low-performing students” is, if anything, from charters to traditional schools. Perhaps what Jinnai discusses is unique to North Carolina.

“At the same time, my study does not find any empirical evidence that charter schools harm nearby traditional public schools.”

It would not have been difficult to find empirical evidence if Jinnai had looked beyond test scores. Jinnai might have looked at the programs traditional schools offered and the cuts they made after losing funding. Usually, the result of a loss of funding is cutting the arts, electives, and extra curricular programs. He also might have looked at turnover rates in teaching staff at charter and public schools. He also might have looked at how unreliable school turnaround programs are. There’s a lot he might have looked at but does not.

The irony that is not mentioned in this study is that increasing school choice tends to lead to a lack of choice in programs as the curriculum becomes narrower.

Timothy Zak December 9, 2012 at 9:04 am

There was a lot of demonizing and unmerited speculation and innuendo when this issue came up here in Alberta. I found it disconcerting to observe how eagerly some people deride parents’ stake in education. I generally don’t like busybodies, particularly those who simultaneously claim to be the best decision maker when it comes to everybody else’s children.

Michael,
” liberal indoctrination institutions run by evil unions who force evolution, global warming, and diversity down the children’s throats”

Parents, and even children are well aware of the sort of ideological campaigning that goes in some classrooms. People who are adamantly opposed to charter schools are a perfect case in point. Schools are not built to employ teachers, or to provide a forum for their opinions. They are built for education. Since the parents are the primary educators, I find it interesting their authority is so hard for some schools to accept.

Ryan,
” increasing school choice tends to lead to a lack of choice in programs”
Ha, well said! Very sly… I hope. (Unless both of us are being obtuse.)

Steve Sailor,
“> increased cheating?”
Such innuendo comes from an ideological tilt, reflected in how implicitly, habitually people substitute the state for parenthood. I am not surprised any more to hear explicit comments of bigotry against those who are neither rich nor complacent, or who hold relatively benign religious beliefs. My impression: other than the obvious stake unions have in preventing competition, the contentious aspect of school choice is not the sufficiency of education but of control.

Comments on this entry are closed.

Previous post:

Next post: