A new RCT study of preschool

by on November 22, 2013 at 9:28 am in Data Source, Education | Permalink

From Grover J. “Russ” Whitehurst:

I see these findings as devastating for advocates of the expansion of state pre-k programs.  This is the first large scale randomized trial of a present-day state pre-k program.  Its methodology soundly trumps the quasi-experimental approaches that have heretofore been the only source of data on which to infer the impact of these programs.  And its results align almost perfectly with those of the Head Start Impact Study, the only other large randomized trial that examines the longitudinal effects of having attended a public pre-k program.  Based on what we have learned from these studies, the most defensible conclusion is that these statewide programs are not working to meaningfully increase the academic achievement or social/emotional skills and dispositions of children from low-income families.

There is much more at the link at the Brookings blog, including the major details of the Vanderbilt study, with some very useful pictures.  For the pointer I thank Michelle Dawson.

john personna November 22, 2013 at 9:33 am

I thought it was established that any benefits, if they are there, are in later life outcomes, job success, lower rates of incarceration.

Is it really “devastating” that another study looks at low immediate gains?

I think you need to attack those later associations.

Jeff J November 22, 2013 at 10:07 am

This “devastating” report says exactly the same thing:

“The non‐cognitive effects of TN‐VPK are important because of their potential long‐term influence on children’s academic careers and the findings in other studies of early childhood education that show this to be the domain in which the largest effects occur. Further, the literature identifies these non‐cognitive outcomes as those with the biggest cost saving implications for schools and communities. It is too early to expect such effects to appear with any consequential magnitude for TN‐VPK, but there are early promising signs in the positive findings so far for kindergarten grade retention and first grade attendance.”

john personna November 22, 2013 at 10:49 am

Yikes.

Careless November 22, 2013 at 3:21 pm

Is it really “devastating” that another study looks at low immediate gains?

When a trillion a year of our country is running on that being false? Yes.

Careless November 22, 2013 at 3:23 pm

It would be another thing if the lack of evidence and all the evidence against it had any effect at stopping the movement, but it hasn’t, so we’re getting closer and closer to universal preschool to fail at the same things.

Careless November 22, 2013 at 3:25 pm

Distilled further, we’re all going to be setting a $100 bill on fire (adjust for your income) because people can’t accept unpleasant results of education studies on pre-teens

dearieme November 22, 2013 at 9:34 am

It’s a general rule that small-scale experiments in education always work, and that large-scale extensions almost always fail. If RCTs mimic large-scale extensions, that’s very handy.

Steve Sailer November 22, 2013 at 3:19 pm

Teaching is one of the performing arts. There are radical differences in the effectiveness of performers, whether in theater, movies, music, or teaching. Successful actors sometimes start up schools of acting where they train young actors. Some of them are very effective, but seldom does a new technique revolutionize the history of acting instruction. Stanislavski’s “method” is of course the exception to that rule, but the enduring fame of Method Acting instruction 65 years after it became a big deal on Broadway and in movies is testimony to how hard it is to come up with an effective new form of acting instruction.

john_d November 22, 2013 at 9:39 am

Universal pre-K is about free daycare so that parents can return to the workforce, not about the educational benefits to the children.

Yancey Ward November 22, 2013 at 10:31 am

It is not even this- it is simply another big government job group to be formed and unionized by the public employee unions.

stubbs November 22, 2013 at 11:18 am

I don’t know. Someone needs to do a study of the effects of teachers’ strikes. My bet that the biggest opponents of these strikes will be parents outraged at being denied the usual childcare provided. But the growth of union membership is certainly a goal of any democrat.

Melanie November 23, 2013 at 3:22 pm

This is limited to PA teacher strikes, but they’re not allowed in many states so this is probably the best you’ll get:
http://www.contrib.andrew.cmu.edu/~aclay/Turner_Strikes_Feb2013.pdf

Steve Sailer November 22, 2013 at 3:21 pm

Universal pre-K is about getting children away from bad mothers and putting them in the hands of nice people with nice degrees. One side effect of reducing the work load for bad mothers is it gives them more time, energy, and money to hit the clubs and make more babies to be dumped on universal pre-K.

Careless November 22, 2013 at 4:02 pm

Probably the one thing where I’m in synch with Sailer: temporarily sterilize people taking advantage of free preschool

Mister X November 22, 2013 at 9:40 am

I thought it was understood that much of the ‘benefit’ to these programs was to provide state-sponsored daycare for underprivileged families.

The utilization of the state-sponsored schools as a mechanism of wealth-redistribution has been on a solid march forward for several decades (I.e. the so-called ‘school lunch’ programs that provide 2-3 meals a day even when school is not in session).

anon November 22, 2013 at 1:31 pm

much of the ‘benefit’ to these programs was to provide state-sponsored daycare for underprivileged families.

I suspect that used to be the case, but suspect that now the pro-state indoctrination aspects are increasingly important.

Albigensian November 22, 2013 at 2:50 pm

Free daycare can (and in many states is) provided at far lower cost than preschool. Nor is it entirely a gift- it exists in part to enable parents (OK, single mothers) on welfare to transition from welfare to work.

This free daycare is often provided by small businesses which are then paid by the state.

“Preschool” may in fact be just daycare, but once it’s called “school” it becomes another public school, to be owned and operated directly by government and staffed by union teachers.

So, yes, I think it’s mostly about increasing the number of union teachers; any benefit to children or their parents is mostly incidental. Those unions provide a lot for their friends at election time.

It’s hard not to be cynical when it comes to preschool. After all, Head Start doesn’t work either yet it will probably last as long as the Republic.

Marie November 22, 2013 at 4:09 pm

Yes.

Government financed child care for the poor is a huge chunk of market, particularly as making sure those abandoned mothers with no resources hand their kids to someone during the day making $10 an hour so they can go out and make $8 an hour is popular with constituencies of both parties (or at least not something any publicly disagrees with).

The schools certainly want to make sure they get their part of the take.

I would suggest, though, that the teachers’ unions are probably only a little voice in this direction. When you have a preschool full of “high risk” kids in your building, that’s a ton of federal and state dollars coming in. Sure, it means hiring a teacher or two, but it also means a lot of skimming to pad salaries of administrators. If you have a school without a preschool, it’s not the teachers or their union that are going to lobby for the addition — it’s going to be the admin.

Mark November 22, 2013 at 9:48 am

This is the kind of thing we need more of: not just randomized trials, but commentary from people who are willing to review evidence and let it shape their view of policy instead of the other way around.

Jan November 22, 2013 at 2:14 pm

Are you kidding? Agenda, Whitehurst has it.

This blog post may give readers an idea how open minded Russ Whitehurst is on education policy issues: http://dianeravitch.net/2012/06/11/the-day-i-was-terminated/

Michael November 22, 2013 at 9:49 am

To some extent, the possibility of improved educational outcomes may be more a PR move used to sell the idea of free babysitting for poor mothers. Free babysitting can be a really great thing for unemployed low wage single mothers. Without it, the costs of child care may actually exceed any wages earned by the mother were she to go out and get a job. Granted, you could just give the mothers lump sum payments, but that sounds too much like welfare. This emphasis on educational outcome shifts the focus to the children (“Won’t somebody think of the children!”), and away from the poor single mother getting yet another gov’t handout (or however it’d get described by those opposed to gov’t help).

Furthermore, I think small gov’t types know this, which is why they seem unusually interested in finding evidence that there aren’t any educational benefits. They want to kill that PR aspect so they can start attacking it for what it really is, free babysitting for poor people.

eric November 22, 2013 at 10:05 am

he be trollin’

anon November 22, 2013 at 1:37 pm

+1

anon November 22, 2013 at 1:40 pm

a +1 for eric’s trollin’ comment

NPW November 22, 2013 at 10:11 am

What about single poor fathers?

cliff November 22, 2013 at 10:58 am

But why do we want to pay for daycare so a low wage mother can go back to work? Isn’t she more valuable at home? If anything we should be subsidizing high wage mothers to return to work, or at least stop discouraging them, by making child care costs deductible. Its a win-win-win for the mother/state/child.

AlanW November 22, 2013 at 12:27 pm

Child care costs are deductible.

Finch November 22, 2013 at 1:51 pm

The child and dependent care tax credit is both tiny and subject to phase-downs. It’s not really a meaningful help. If you get $1000 out of it, you’re doing well.

Marie November 22, 2013 at 4:12 pm

Yes, but if you were pretty much going to do it anyway, it’s a free $1000. Tax subsidies to get people to do what they want to do anyway don’t have to be as big.

Finch November 22, 2013 at 4:29 pm

I think cliff generally argues that childcare costs are a cost of doing business and therefore should be fully deductible. AlanW said childcare costs are deductible, which is very far from the truth. There exists a credit which provides maybe 6 or 7 percent of the cost of childcare for an infant.

JWatts November 22, 2013 at 3:58 pm

“To some extent, the possibility of improved educational outcomes may be more a PR move used to sell the idea of free babysitting for poor mothers”

You mean to some extent the advocates are lying.

Marie November 22, 2013 at 4:13 pm

For our own good, for our own good. You can’t forget that part.

john personna November 22, 2013 at 9:52 am

Again, one argument I’ve heard is that a dollar spent on pre-school saves more than a dollar in the prison system. If that’s true, pragmatists and “small government types” should be all over it. It represents a net reduction.

Mark November 22, 2013 at 10:11 am

That argument probably stems from the Perry Preschool Program, and Heckman et al’s subsequent 2010 re-analysis of the data. Costs from crime were significantly reduced. (p<0.1, which I guess is an okay threshold since we're looking at social outcomes decades later. Also, who am I to argue with Heckman?) So there may really be a benefit from giving free babysitting to poor people. But from the evidence, you can't argue for universal pre-K on the basis that recipients will have a better education.

What we need is more longitudinal studies, but they're hard and expensive. Educational intervention may be something like new medical/surgical procedures. We won't know what works and what doesn't work without trying some things, even if the benefits may not materialize for the study's subjects.

john personna November 22, 2013 at 10:51 am

I would not be opposed to a “big data” project identifying those zip codes which most need per-K to improve long-term outcomes.

P November 22, 2013 at 5:47 pm

36 percent of the members of the treatment group in the Perry program had been arrested 5 or more times by age 40, compared to 55 percent of the control group. Perhaps Heckman’s calculations are correct and this small reduction in criminal offending and other small effects made the program cost-effective. However, the idea that the program could be replicated on a large scale today with similar effects is a pipe dream. Not only is it very difficult to scale up such programs, but there’s little reason to expect that what worked then would work today.

Jeff J November 22, 2013 at 10:23 am

An economist named Tim Bartik has commented on Whitehead’s thoughts on the subject, relying on this net savings argument:

http://investinginkids.net/2013/06/19/brookings-article-provides-support-for-high-benefit-cost-ratios-for-state-pre-k-but-you-wouldnt-know-it-from-the-article/

john personna November 22, 2013 at 10:53 am

Wow, you wouldn’t need “a return of $7 for every dollar invested” to make me happy. A return of $1.10 on $1.00 is a win.

cliff November 22, 2013 at 11:07 am

How can he justify a 3% discount rate though? That has a huge effect on the numbers. I would use 7-8%

cliff November 22, 2013 at 11:08 am

Well I guess that’s 5-6% in real terms but still

Bill November 22, 2013 at 10:17 am

What is interesting about the study is that it ignores a complementary part of the gradeschool teaching system.

Notice I said gradeschool, no pre-school.

Kids begin to be tracked in first, second, and in some cases, third grade. Kids who have had pre-school typically get tracked higher than kids who did not have pre-school.

The question to ask is: not whether pre-school effects last, but whether tracking too early causes kids who did not have pre-school be tracked lower than they would have been tracked had they had pre-school, and whether tracking later would be better if kids have a chance in 1st and second grade to catch up with the kids who did not have pre-school prep.

Hannes November 22, 2013 at 10:28 am

These results after a 4 year follow-up do not look inconsistent with Chetty et al. http://obs.rc.fas.harvard.edu/chetty/STAR.pdf

They found that the test results effect from a good Kindergarten teacher faded quickly, but they still found significant gains in adulthood.

Ian Brown November 22, 2013 at 10:44 am

Basic causation/correlation mistake with adult outcomes? Having the type of parents who send you to pre-school (from both a nature and nurture perspective, regardless of which you think matters more) is going to make you more likely to be successful. Even if nothing productive or long lasting happens at pre-school.

Uninformed Observer November 22, 2013 at 12:24 pm

Children need no parent but the State.

JWatts November 22, 2013 at 4:17 pm

Children need no parent but the State.

It takes a Village

Marie November 22, 2013 at 4:51 pm

This was all highly recommended in the book. And it was suggested that we run parenting videos on TVs at the DMV while people waited. The irony of the DMV telling anyone how to do their jobs better apparently didn’t get caught by the editor.

ed November 22, 2013 at 2:13 pm

That’s why the focus here is on randomized controlled experiments. Did you miss that point?

Marie November 22, 2013 at 2:22 pm

What on earth makes you think having the kind of parent who sends you to pre-school is going to make you more likely to be successful?

Do you get what an incredible PITA it is to have a three year old hanging around your house all day? It takes an incredible act of will to say no when someone offers to take the snotty thing off your hands for eight hours a day, five days a week, at no cost to you. You think the poor people who say no are the bad parents?

Finch November 22, 2013 at 2:42 pm

Ian may have been distinguishing pre-school from daycare.

Pre-school is the three-morning a week thing for wealthier stay-at-home-moms, while daycare is the 40+ hour a week thing for the working poor.

Marie November 22, 2013 at 4:15 pm

Sorry if that’s so, I assumed they meant pre-school as studied here — definitely a mis-label, unless you want to consider the building of the pyramids and reign of Napoleon to be antebellum.

Finch November 22, 2013 at 4:31 pm

Yeah, I don’t actually know what he meant. But in common parlance, I hear pre-school to mean mom-doesn’t-work-outside-home and daycare to mean mom-does-work-outside-home.

There is a major class distinction between pre-school and daycare as most people use those terms.

Marie November 22, 2013 at 4:48 pm

Not where I come from, unfortunately. Preschool is free here and people are pushed, bribed, and nearly threatened to use it, particularly if their kids are showing signs of any developmental blips. Even home daycares and commercial daycares usually use “preschool” these days — why wouldn’t they? It sounds much better, and there’s no rule that says they can’t. Makes me grim-laugh when I see the signs up advertising preschool for “infant through age 5″.

I don’t think this study is talking about dropping your kid off to finger paint for an hour a couple times a week.

Finch November 22, 2013 at 4:54 pm

I think you are right in your characterization of the study. Regarding the terms daycare and preschool, I’ve seen them used the way I described in several widely separated parts of the US, but I guess it isn’t universal.

CJ November 22, 2013 at 11:43 am

Sara Mead is a must read on Whitehurst’s analysis:

“Second, I do think there’s a reason to question the logic model for comparing elementary school results for comparable children who were and weren’t randomly selected into pre-k programs. Measures of elementary learning aren’t measuring an innate quality of children; they’re evaluating what children know and can do because they’ve learned it. And what children learn in kindergarten and first grade depends on what and how well they are taught in those grades. If preschool children are being taught in the same classrooms in kindergarten and first grade, exposed to the same curriculum and lessons taught by the same teachers, how exactly do we expect the pre-k graduates to gain the additional learning that enables them to continue to stay ahead of their peers?”

More here: http://blogs.edweek.org/edweek/sarameads_policy_notebook/2013/11/pre-k_research_disappointing_devastating.html

ed November 22, 2013 at 2:15 pm

Um, because you retain something better the second time through if you’ve been exposed to it before? Seriously, this seems like a very stupid question. Am I missing something?

P November 22, 2013 at 6:53 pm

If preschool children are being taught in the same classrooms in kindergarten and first grade, exposed to the same curriculum and lessons taught by the same teachers, how exactly do we expect the pre-k graduates to gain the additional learning that enables them to continue to stay ahead of their peers?

There are plenty of individual differences in learning outcomes within any classroom. If preschool programs worked, preschool kids would outperform other kids after they enter school. They don’t.

The “achievement gap” emerges long before formal schooling begins. Preschool is supposed to remedy this by exposing lower class kids to cognitive and other stimulation that they don’t get at home. This is supposed to put them on a permanently higher trajectory of achievement. But it doesn’t work.

TMC November 22, 2013 at 11:51 am

So the main benefit to the kid is social. The less exposure to his family the better off he is.

mike November 22, 2013 at 11:59 am

Yeah that is basically the answer. Addition by subtraction. Take the ghetto trash parent (usually only one) out of the kid’s life as much as possible. But of course, we can never admit that’s what we’re doing.

anon November 22, 2013 at 1:36 pm

Take the ghetto trash parent (usually only one) out of the kid’s life

Alternatively:
Take the parents who don’t give a rat’s ass about the children out of the kid’s life.

And who do you propose to put in the kid’s life? If it is some variation of the state then frankly that worries me a lot more. It may take a village to raise a child, but if it takes a village government to raise a child then we are screwed.

mike November 22, 2013 at 1:55 pm

I’ve seen some research that suggests single-sex orphanages are the best alternative. Ideally run on the Christian model and instilling Christian values.

Finch November 22, 2013 at 1:57 pm

“Is Marriage Always Good for Children? Evidence from Families Affected by Incarceration”

http://econ.tulane.edu/kfinlay/pdf/FinlayNeumark2010.pdf

Steve Sailer November 22, 2013 at 3:26 pm

TMC says:

“The less exposure to his family the better off he is”.

Yes, that’s most of the logic behind the case for universal pre-K. That’s why you see all the references to the Hart-Risley study that supposedly proved that black people don’t talk enough. The idea behind universal pre-K is similar to the one that motivated humanitarian progressives in the 1920s in Australia and Canada to provide boarding schools for indigenous children — the much apologized for “Stolen Generations” — get the children away from their families as much of their waking hours as possible.

The Anti-Gnostic November 22, 2013 at 12:30 pm

File this one with “preventive medicine” and other failed liberal shibboleths.

At some point, surely even progressives will break down and say, “Gosh, we keep getting these results back. Maybe we need to re-examine our premises and consider taking things in a different direction.” I mean, empiricism, change, paradigm-shifting, that’s what Progress is all about right?

Nope. More money, more studies, more tinkering.

draypresct November 22, 2013 at 12:53 pm

The trial was limited to parents who volunteered their children for it when they couldn’t get into other pre-school programs. The most they can say is that pre-school may not improve outcomes among families where the parents are already intensely involved in their children’s education. According to this blog’s own link, previous studies that included more representative samples of families seemed to find a lasting improvement in outcomes among children enrolled in high-quality pre-school programs.

Bill November 22, 2013 at 8:37 pm

Yeah, but don’t tell anyone. It will upset the headline that this is a random controlled trial. It isn’t if you have selfselection by interested parents. Shhh.

This point was made, by the way, in comments to the blog so there is no excuse labeling it a random trial.

Sam November 22, 2013 at 1:38 pm

Here’s why I, as a libertarian, could be lead to favor universal preschool: it’s about the positive externalities to socialization, not education. It’s the cultural compliment to the negative income tax, viz. open immigration. The negative income tax lets you have more immigration while retaining a social safety net. Universal preschool lets you have more immigration while retaining an integrated culture and the myriad stock of intangible social capital which it supports. “You can live in our country as long as we get to program your child’s mind.”

mike November 22, 2013 at 1:57 pm

You realize that the people who (have been and) will be programming childrens’ minds are decidedly un-libertarian, right?

Marie November 22, 2013 at 2:01 pm

No study will every matter. The people who want universal (preferably compulsory) preschool want what they want and if one justification gets crossed off the list another will be added or moved up.

There’s no public policy answer to this sort of thing, we need parents to stop passing their kids through the fire just because everyone else is doing it.

Spencer November 22, 2013 at 2:17 pm

Isn’t it amazing how you can always find a study that agrees with your priors– ether liberal or conservative.

JWatts November 22, 2013 at 4:22 pm

This particular study was done by the University of Vanderbilt Peabody Research Institute. It’s not some kind of partisan think tank and it wasn’t a 10 page narrative with some cherry picked graphs.

P November 22, 2013 at 6:07 pm

So what are some contemporary, big-N RCTs that show that Pre-K is effective?

Bill November 22, 2013 at 8:53 pm

Is this an argument against tax credits or deductions for early childhood education?

Tim Bartik November 22, 2013 at 10:26 pm

I think the issue is whether one study of one state’s program trumps the weight of much other evidence for the effectiveness of many state pre-K programs. This would be true even if the Tennessee study was perfect (it isn’t) and if Tennessee’s pre-K program was regarded as the best state pre-K program in the U.S. (it isn’t).

The Tennessee study, although a random assignment study, has serious problem of sample attrition due to parental non-consent to allow data to be collected. For example, in Cohort 1 of the study, only 32% of the control group, versus 46% of the treatment group, provided parental consent to collect data. This may lead to unobserved differences between the treatment and control group that eliminate much of the advantages claimed for randomized control trials.

In addition, Tennessee’s program is not a particularly well-funded program per child. The National Institute for Early Education Research estimates that Tennessee probably would need to spend around $2,000 extra per child annually to provide a quality program.

It is unclear to me why Dr. Whitehurst thinks this one study of Tennessee should trump the many studies with good comparison groups that show good results for better-funded state and local pre-K programs, such as recent work by Weiland and Yoshikawa on Boston’s pre-K program.

I have provided a lengthier discussion of this issue, with links, at my blog: http://investinginkids.net/2013/11/22/pre-k-policy-should-be-based-on-all-the-evidence-not-one-study-of-one-states-programs/

Comments on this entry are closed.

Previous post:

Next post: