The Impact of No Child Left Behind on Student Achievement

by on November 25, 2009 at 7:20 am in Education | Permalink

Preliminary results are in, and they suggest it has helped with math skills but not with reading achievement, as measured in the 4th and 8th grades.  Via Thomas Dee and Brian Jacob:

The No Child Left Behind (NCLB) Act compelled states to design
school-accountability systems based on annual student assessments. The
effect of this Federal legislation on the distribution of student
achievement is a highly controversial but centrally important question.
This study presents evidence on whether NCLB has influenced student
achievement based on an analysis of state-level panel data on student
test scores from the National Assessment of Educational Progress
(NAEP). The impact of NCLB is identified using a comparative
interrupted time series analysis that relies on comparisons of the
test-score changes across states that already had school-accountability
policies in place prior to NCLB and those that did not. Our results
indicate that NCLB generated statistically significant increases in the
average math performance of 4th graders (effect size = 0.22 by 2007) as
well as improvements at the lower and top percentiles. There is also
evidence of improvements in 8th grade math achievement, particularly
among traditionally low-achieving groups and at the lower percentiles.
However, we find no evidence that NCLB increased reading achievement in
either 4th or 8th grade.

That is from an NBER paper, I do not yet see an ungated copy on-line.  To my skewed perspective, this is an intuitive result.  Math skills are more the result of drill, whereas you have to learn how to love to read and much of that happens within the family, not at school.  Math is therefore easier to "teach by central planning," so to speak.

1 Andrew November 25, 2009 at 7:36 am

“The impact of NCLB is identified using a comparative interrupted time series analysis that relies on comparisons of the test-score changes across states that already had school-accountability policies in place prior to NCLB and those that did not.”

Simplification FAIL (“…comparative interrupted time series analysis…”)

Attribution FAIL (a good Federal idea is almost always a copy of a good state idea, and to centralize everything is to kill future state innovation. In a sense, the control is not actually the schools that already implemented the idea.)

Plain English FAIL (So, what are they saying?)

Cost-benefit FAIL (http://www.heritage.org/research/education/wm1406.cfm)

Control for Hawthorne effect FAIL (where is the placebo group, oh yeah, Tyler told us all education is the placebo group!)

Opportunity cost/Free lunch FAIL (Free lunches don’t exist, what was the cost? From where did the teaching-to-the-test time come from, was that area assessed? Surely it wouldn’t come from hard to measure things, like subjects that boost creativity, would it?)

Recognition of analogy to healthcare reform FAIL (Math panels? Is healthcare like math or is healthcare like reading/communication?)

All-in-all, good effort.

2 Andrew November 25, 2009 at 8:06 am

Duly noted.

If I ever decide to be an economist I’ll keep it in mind.

3 Kevin Miller November 25, 2009 at 8:13 am

There’s reason to think that math is in general more closely tied to schooling than is reading. A big part of this is the massive difference in out-of-school practice in reading among different students (7x variation in the one good estimate I saw between students at the 25th and 75th percentile), whereas I don’t think you have the same extreme distribution in math.

If this is right (I don’t know of a similar estimate for math, so that’s speculation), then reading would be more buffered against the effects (good or bad) of interventions by schools.

4 John Farragut November 25, 2009 at 9:32 am

“Math is therefore easier to “teach by central planning,” so to speak.”

This explains the large number of brilliant Soviet mathematicians!

5 JSK November 25, 2009 at 10:12 am

@John:
Russians were good at math during tsarist times as well.

6 Zoe November 25, 2009 at 12:56 pm

Steven, that’s an excellent point about comprehension – of all the grown-ups who read The Secret most couldn’t tell you why it’s shit. Reading without reflection is useful – it will teach you how to copy and paste in the workplace – but it won’t help form critical minds. Comprehension practice and reflective reading take far more classroom time and are far more variable to teach even given small differences between kids.

Whereas math is memorization – you can get through most pre-calculus math by just memorizing rules and following them, but still not know what the heck you’re doing, or why. Seems to me that teaching to the test simply makes it less likely students will ask for clarification or that teachers will give it to them properly.

7 kranky kritter November 25, 2009 at 2:35 pm

Math achievement is a pretty well-defined concept at the middle school level. After editing math book for 15 years, I’m comfortable saying that.

Reading comprehension? Not so much. Vocabulary can be a red herring even if its an indicator. Fact is, what most folks really want is critical thinking skills…the ability to digest information, put pieces of info together, and communicate that to other people. And that’s something which #1 develops cognitively later on in a person’s development and #2 is seldom explicitly focused upon in the classroom as a subject or goal in and of itself. Students may be asked to apply critical thinking within some domain (literature, history, etc), but there’s no place where the education is itself critical-thinking-centric.

I think that should change. You know what would be good CT practice?Make kids argue on blogs about ideas they have passion for, and then grade them on things like using data, incorporating facts and ideas from other domains, grace in not tossing out insults, sticking to the subject, and so on.

8 kranky kritter November 25, 2009 at 2:39 pm

Math achievement is a pretty well-defined concept at the middle school level. After editing math book for 15 years, I’m comfortable saying that.

Reading comprehension? Not so much. Vocabulary can be a red herring even if its an indicator. Fact is, what most folks really want is critical thinking skills…the ability to digest information, put pieces of info together, and communicate that to other people. And that’s something which #1 develops cognitively later on in a person’s development and #2 is seldom explicitly focused upon in the classroom as a subject or goal in and of itself. Students may be asked to apply critical thinking within some domain (literature, history, etc), but there’s no place where the education is itself critical-thinking-centric.

I think that should change. You know what would be good CT practice?Make kids argue on blogs about ideas they have passion for, and then grade them on things like using data, incorporating facts and ideas from other domains, grace in not tossing out insults, sticking to the subject, and so on.

9 kurious kritter November 25, 2009 at 4:03 pm

Kranky Kritter is right on. I am wondering, though, why Tyler Cowen thinks love of reading comes from the family. Is their data to support this?

10 Steve Sailer November 25, 2009 at 8:13 pm

Reading is something that a sizable fraction of people do outside of school hours because they like it. Only a tiny fraction do math when they don’t have to because they like it.

I doubt if nurture effects are as big as nature effects in terms of loving to read.

11 doctorpat November 26, 2009 at 10:25 pm

So, how’s the NCLB doing on it’s legally-mandated goal of making 100.00% of raising all students out of “Below Basic” or “Basic” on test scores to “Proficient” or “Advanced” by the 2013-2014 school year? Are they going to make it?

It’s not REALLY legally-mandated unless there is a legislated penalty for the administrators and the originators of the idea. Preferably corporal punishment. On live TV.

Please?

12 Mick November 28, 2009 at 8:15 pm


“Math is therefore easier to “teach by central planning,” so to speak.”

This explains the large number of brilliant Soviet mathematicians!

Another explanation could be that Russia had a large number of high IQ Ashkenazi Jews whose career options in Hedge Funds or law firms were somewhat limited, as USSR did not have any.

Ethnic Russians have a decent IQ, comparable to other Europeans and there were 50 times more of them than Jews. So they produced a fair number of top mathematicians as well.

13 Cynthia B November 29, 2009 at 4:01 pm

“Math is therefore easier to ‘teach by central planning,’ so to speak.”

The statement is worthwhile as far as it goes; however, there is more to be considered from a pedagogical perspective. Computation skills can be sharpened with drill. Math, though, includes not only computation, but also numeracy, which can be seen as the mathematical parallel to literacy. In the same way that reading involves not only decoding of phonemes but also comprehension, sharp mathematical thinking requires not merely good calculation, but also thoughtful problem-solving and the ability to articulate it, whether through sentences or algorithms. While the higher-level mathematical thinking involved in creative problem solving does require good computation, it cannot be fully appreciated on standardized tests.

A teacher who can articulate what s/he is looking for in both numeracy and literacy can teach it and measure it, but not on a multiple choice (standardized) test.

While perhaps math computation can be “taught by central planning,” I am not convinced that all higher-level mathematical thinking can. I think both literacy and numeracy fall into the same pedogagical category in this regard.

14 Call Center March 6, 2010 at 7:24 am

TFA is good at increasing math tests scores and much less effective with reading.

Comments on this entry are closed.

Previous post:

Next post: