The Small Schools Myth

by on September 2, 2010 at 7:30 am in Books, Data Source, Economics, Education | Permalink

Did Bill Gates waste a billion dollars because he failed to understand the formula for the standard deviation of the mean?  Howard Wainer makes the case in the entertaining Picturing the Uncertain World (first chapter with the Gates story free here). The Gates Foundation certainly spent a lot of money, along with many others, pushing for smaller schools and a lot of the push came because people jumped to the wrong conclusion when they discovered that the smallest schools were consistently among the best performing schools.

Schools1The chart at left, for example, shows by size the percentage of schools in North Carolina which were ever ranked in the top 25 of schools for performance.  Notice that nearly 30% of the smallest decile (10%) of schools were in the top 25 at some point during 1997-2000 but only 1.2% of the schools in the largest decile ever made the top 25.

Seeing this data many people concluded that small schools were better and so they began to push to build smaller schools and break up larger schools. Can you see the problem?

The problem is that because small school don’t have a lot of students, scores are much more variable.  If for random reasons a few geniuses happen to enroll one year in a small school scores jump up and if a few extra dullards enroll the next year scores fall.

Thus, for purely random reasons we would expect small schools to be among the best performing schools in any givenyear.  Of course we would also expect small schools to be among the worst performing schools in any given year!  And in fact, once we look at all the data this is exactly what we see.  The figure below shows changes in fourth grade math scores against school size.  Note that small schools have more variable scores but there is no evidence at all that scores on average decrease with school size.

States like North Carolina which reward schools for big performance gains without correcting for size end up rewarding small schools for random reasons.  Worst yet, the focus on small schools may actually be counter-productive because large schools do have important advantages such as
being able to offer more advanced classes and better facilities.

Schools2All of this was laid out in 2002 in a wonderful paper I teach my students every year, Thomas Kane and Douglas Staiger’s The Promise and Pitfalls of Using Imprecise School Accountability Measures.

In recent years Bill Gates and the Gates Foundation have acknowledged that their earlier emphasis on small schools was misplaced. Perhaps not coincidentally the Foundation recently hired Thomas Kane to be deputy director of its education programs.

Ignoring variance and how it relates to group size is a simple but common error. As Wainer notes, building on a discussion in Gelman and Nolan, counties with low cancer rates tend to be rural counties in the south, mid-west and west.  Is it the clean country air or some other factor peculiar to rural counties which accounts for this fact?  Probably not. The countries with the highest cancer rates also tend to be rural counties in the south, mid-west and west!  Once again, small size and random variation appear to be the main culprit.

Ken Rhodes September 2, 2010 at 8:09 am

“Ignoring variance and how it relates to group size is a simple but common error.”

Well DUUUUH!

Ignoring variance and how it relates to SAMPLE size is a simple but common error in ANY syllogism based on statistics.

pbhj September 2, 2010 at 8:58 am

>Note that small schools have more variable scores but there is no evidence at all that scores on average increase with school size.

Didn’t he mean decrease?

I’ve never heard this argument before. In my country the focus is on class sizes. One teacher to 30+ students is too many to properly educate them IMO.

Bill September 2, 2010 at 9:13 am

Experimentation is not waste of money. Good for Gates in conducting the experient.

Only those who seek to confirm their beliefs are wasting money. Empricists and factfinders have no trouble with false starts. Science is 98% false starts.

Ren September 2, 2010 at 9:45 am

I was at the beach in Morehead City, NC last week. Reading the local newspaper, I saw a story about how a charter school was in danger of being closed because of a high dropout rate. However, this school is one to take in many students who have already left other schools and this was not taken into account in the evaluation. The school is going to enact a policy that it will not enroll anyone after Sept 1 to avoid enrolling prior dropouts. Not a great system for helping troubled teens find their way.

I don’t know anything more about the charter school than what I read in the paper, but the article, if correct, did raise some concerns about how NC evaluates schools.

boji September 2, 2010 at 9:58 am

Small schools are simply better than large schools, especially urban schools. This fact is often hard to prove because so little education research has been done in small schools. Even basic numbers from standardized tests show that our small school would stack up against the best elite prep schools.

As far as the variance goes, that can be corrected by longitudinal data. As a former legislator and an active blogger, I get really sick of professional pundits who seem to take for granted that bigger is better.

As far as classes go, I think some of your observations are just wrong or short sighted. Some advanced classes that are avaiable in larger districts can be available to smaller districts through a new thing called the internet. Nearly all of our seniors take some distant learning classes, and more than 2/3 of our seniors have some college credits when they graduate from high school. Our top third usually get a semester or more. Our students go on to perform well in elite schools, regent universities, and private colleges.

For our struggling students, we can provide detailed instruction, caring attitudes, and no opt out policy that has given us a graduation rate of more than 99%. (We have had one drop out the past three years with an average graduiation class of 75) When most urban districts struggle to graduate more than 60% of their students, and suburban districts 80%, small schools rarely have a variance under 90%.

The prejudice against small schools is startling to me. It is one of the few givens in education reform that small schools just are not very good, well at least not as good as larger districts. Education reform has a lot of beliefs that studies show are not true. However, the beliefs about small schools are not even tested. The evidence that does exist does not indicate small schools perform poorly until the size of the school gets under 600 students K-12. Most pundits are shocked at the quality of education that is available all over Northwest Iowa in schools where the high schools have 200-400 students.

As a coach of forensics, my students successfully compete against the top students from elite prep schools and the top suburban schools in the nation. Our graduates work world wide from India, to Africa, to London, and also include a neurosurgen at the Cleveland clinic and one of microsoft’s new os developers.

Small schools can and do educate at a very high level. We have the advantage of quality students, parents, and administrators that allows discipline, so teachers just get to teach. When safety is not a concern, learning is a lot easier. We may not offer as many classes as our urban counterparts, but we have fewer slacker classes, and the availability of distant learning means even three of our freshman can take Latin.

As a statitician, maybe you should look at areas outside of North Carolina. Iowa has a lot of reports worth studying, and the strongest findings is that small schools on a national level or average schools in Iowa, have the best test scores and are the most efficient. Real reform will happen when we stop trying to fix all schools in the same way. The Race to the Top is a necessary program for Urban schools, but the problems it addresses are not the problems of rural districts. Forcing rural districts to adopt the same reforms is short sighted and counter productive.

Pablo H. September 2, 2010 at 10:20 am

Agree with all of the above, though you might be interested in the new MDRC research which praises the performance of small schools in NYC

http://www.mdrc.org/publications/560/overview.html

http://www.nytimes.com/2010/07/01/opinion/01thu3.html?_r=1&scp=1&sq=small%20schools%20mdrc&st=cse

Frog Leg September 2, 2010 at 10:41 am

Let’s look at a more specific, but still back-of the envelope, calculation. Standard deviation typically scales with 1/sqrt(n), where n is the sample size. If the variation were purely due to random chance, the ratio of number of students in the largest decile to that in the smallest decile would be ((27.7/1.2)^2 = about 533 to 1. This ratio is not reasonable. Therefore, there might be something there, even taking into account the difference in sampling size.

Bill September 2, 2010 at 10:59 am

Lake Wobegon has small schools, and all of its students are above average.

I would be interested in performance measures other than just testing scores.

For example, do students in smaller schools have a better self image, because they are more likely to be number 1 among fewer students than number 1 among many; what is the minimum size for enrichment or other extracurricular programs; what classes are frequently not offered; what classes are frequently offered because they do not involve capital costs, such as chemistry labs, etc.; what is the frequency of AP classes by school size; what is the percentage of student involvement in after school extracurricular activities, etc.

Testing is a one dimensional measure. I would fault Gates if they did not measure across other dimensions of student performance or attainment. But, I don’t know, so if someone does, let me know.

David R. Henderson September 2, 2010 at 11:18 am

Nice post, Alex.

Eliza September 2, 2010 at 12:16 pm

The author has a point about the poor use of statistics but it doesn’t exactly make the case for large schools (not sure if you’re trying to). It does make the case for reconsidering how government tinkers with schools.

Dropout rates might be another metric to look at. Large schools could have higher scores because more of their low achieving students dropout whereas in smaller schools lower achieving students are retained and graduate, albeit bringing the school’s average test scores down.

http://www.smallschoolsproject.org/PDFS/realgains.PDF

As for a bigger menu of course offerings, in my school of 3000 kids we managed to have 5 kids in an AP European history course. I really don’t think it’s worth sacrificing the whole school’s academic progress so five over-achieving college-bound kids (I was one of them) can learn something they could have read in a book or done as an independent study. And there were plenty of other AP courses I could have taken instead, AP Chemistry or AP American History. Even in my 3000 kid school, I ended up being bundled with 3rd year french students when i was taking 4th year french because there weren’t enough students at that level. If motivated, high-achieving kids are desperate for odd classes, how about partnering with a local community college or university. I also spent time tutoring other kids informally in classes that were mixed, rather than tracked. I’m glad I wasn’t in a magnet school where I wouldn’t have had that opportunity because I think that informal cross-student education is also important.

There are other non-academic advantages to small schools related to community building and city planning. Large High Schools are a “lulu,” locally unwanted land use, so they end up being isolated far from where students live in most new growth areas. That means they’re harder to get to for students and parents which discourage participation and makes it difficult for students too young to drive to participate in activities without parental chauffeurs.

Norm September 2, 2010 at 12:40 pm

Not too many years ago I followed this research more closely. There were consistent findings supporting small advantages, on average and accounting for many social factors, to small schools (down to roughly 400 students) as measured by tests. I never saw anything that convinced me there was a systematic advantage to large schools or, for that matter, a big advantage to small schools.

Several commenters note many factors are important regarding schools like “staying in school” and which particular programs a school, of any size, actually offers. So tests are not the only issue, especially when test results differ only slightly.

Choice is vital. Open enrollment, charters, private and home schooling allow kids and families to find a good situation for them. Small does help provide a variety to choose from.

Ted Craig September 2, 2010 at 1:21 pm

Smaller schools that allow parents to choose them, either through tuition or an enrollment program, muddle the issue. Any time a parent is actively choosing a school for their children, they are showing an increased involvement in their children’s academic careers. This doesn’t prove the simplistic “smaller schools are better schools” argument. Simply breaking up larger schools won’t necessarily work. My point, again, is school size matters less than school atmosphere and I have yet to see any hard facts produced that counter this point. Iowa is not representative of the more challenged areas of the nation.

Rahul September 2, 2010 at 3:46 pm

Maybe small school deserve to be rewarded exactly due to the variance argument. Small schools can be exceptionally good and exceptionally bad without being swamped out by the median approaching tendencies of large numbers.

Small implies accountable. You can easily see if they are being good or bad.

joan September 2, 2010 at 5:00 pm

Are people who are pushing merit pay for teachers based on performance gains also “failing to understand the formula for the standard deviation of the mean”?

Careless September 2, 2010 at 6:09 pm

How many students from neighboring districts would try to get in to New Trier if allowed?

Not that many from the neighboring districts, which all have excellent schools, but quite a number from Chicago. It’s two years to the day since a protest sent hundreds of Chicago students to New Trier to (futilely) try to enroll on the first day of school

vibram five fingers on sale September 2, 2010 at 11:30 pm

 I really loved reading your blog. It was very well authored and easy to understand. Unlike additional blogs I have read which are really not good. I also found your posts very interesting. In fact after reading, I had to go show it to my friend and he enjoyed it as well!

Matt Flipago September 3, 2010 at 12:35 am

I always thought the argument was around Class Size, as in the Student to teacher ratios.

Troy Camplin September 3, 2010 at 2:52 am

What? An increase in population size results in convergence toward the mean? Whoda thunk it?

Clay Boggess September 3, 2010 at 12:37 pm

Of course you will see greater fluctuation in performance with smaller schools; however smaller schools typically have a smaller student to teacher ratio which allows for more personalized instruction. Of course the most important determinant is to have strong and effective leadership at the top with the principal.

sony laptop battery September 7, 2010 at 4:37 am

The alliteration of these allowance laptop ac adapter bung and sometimes sony laptop battery charger abortive abandon ends consistently to alpha authoritative the motherboard

Digital Photo Frame October 18, 2010 at 6:08 am

Digital photo frames are common in 7 inch (17.8 cm) to 20 inch (50.8 cm) sizes. Some digital photo frames can only display JPEG pictures.Digital Photo Frame Most digital photo frames display the photos as a slideshow and usually with an adjustable time interval. They may also be able send photos to a printer. Digital photo frames typically display the pictures directly from a camera’s memory card,Hiphone though certain frames also provide internal memory storage. Some allow users to upload pictures to the frame’s memory via a USB connection,USB Flash Disk wirelessly via bluetooth technology. Few are able to send photos with cellular connectivity. Some frames allow photos to be shared from a frame to another.

club bbq November 23, 2010 at 3:00 am

Now,the portablebarbecue club and smokers for Barbeque at barbecuesclub are good helpers for almost everyone. If you having barbecue outdoor, thesebbq sauce clubsare your necessities.

Comments on this entry are closed.

Previous post:

Next post: