Shout it from the Rooftops! Performance Pay for Teachers in India

by on March 22, 2013 at 7:26 am in Data Source, Economics, Education | Permalink

Several years ago I reported on a very large, randomized experiment (JSTOR) on teacher performance pay in India that showed that even modest incentives could significantly raise student achievement and do so not only in the incentivized subjects but also in other non-incentivized subjects, suggesting positive spillovers. The earlier paper looked at the first two years of the program. One of the authors, Karthik Muralidharan, now has a follow-up paper, showing what happens over 5 years. The results are impressive and important:

Students who had completed their entire five years of primary
school education under the program scored 0.54 and 0.35 standard deviations (SD) higher than
those in control schools in math and language tests respectively. These are large effects
corresponding to approximately 20 and 14 percentile point improvements at the median of a
normal distribution, and are larger than the effects found in most other education interventions in
developing countries (see Dhaliwal et al. 2011).

Second, the results suggest that these test score gains represent genuine additions to human
capital as opposed to reflecting only ‘teaching to the test’. Students in individual teacher
incentive schools score significantly better on both non-repeat as well as repeat questions; on
both multiple-choice and free-response questions; and on questions designed to test conceptual
understanding as well as questions that could be answered through rote learning. Most
importantly, these students also perform significantly better on subjects for which there were no
incentives – scoring 0.52 SD and 0.30 SD higher than students in control schools on tests in
science and social studies (though the bonuses were paid only for gains in math and language). There was also no differential attrition of students across treatment and control groups and no
evidence to suggest any adverse consequences of the programs.

…Finally, our estimates suggest that the individual teacher bonus program was
15-20 times more cost effective at raising test scores than the default ‘education quality
improvement’ policy of the Government of India, which is reducing class size from 40 to 30
students per teacher (Govt. of India, 2009).

In another important paper, written for the Government of India, Muralidharan summarizes the best research on public schools in developing countries. His conclusion is that there are demonstrably effective and feasible policies that could improve the public schools thereby increasing literacy and numeracy rates and raising the incomes of millions of people.

The generation entering Indian schools today is the largest that has ever, or for the foreseeable future, will ever enter Indian schools so the opportunity to raise educational quality for essentially the entire Indian workforce over the next several generations is truly immense.

John S March 22, 2013 at 8:02 am

This is an important result. However, why must these schools be public? Why not support vouchers or tax credits as the Friedman Foundation for Education Choice does? Then we could have incentive pay determined by the market, not bureaucrats.

http://www.edchoice.org/

For American students, the priority is different. Our kids need to develop their creativity and to find their passions early in life. The best model, imo, is the Sudbury Valley School, where kids can learn whatever they want, how they want, for as long as they want. This educational model has worked successfully for decades (80% of grads go on to college).

Great intro video. This is what education looks like in a truly free society:
http://www.youtube.com/watch?v=NxPnvJE0V2E

JWatts March 22, 2013 at 8:25 am

What? Are you saying humans respond to monetary incentives? And that public school teachers are human?

Surely you are mistaken. Obviously your methodology was flawed, the research was undoubtedly secretly funded by miscreant billionaires , it won’t work in this country (fill in whichever country you’d like), it won’t work in an urban environment, etc.

And most importantly think of the Teachers and their Unions! Incentivized pay is exactly opposite of the moral approach we should all strive for. How can you trust some capitalist teacher that’s in it for the money?

Popeye March 22, 2013 at 2:56 pm

How much is Alex getting paid to write these blog posts? How much are you getting paid to write your comments? I don’t see any financial incentive for you to write things that are true instead of things that are stupid bullshit, so why should anyone trust your views? The people running the unions are at least getting a salary for their beliefs.

Jwatts March 22, 2013 at 10:08 pm

LOL, I don’t think you are helping your case with those comments.

Emily March 22, 2013 at 8:38 pm

You can only respond effectively to monetary incentives if you know how to raise the performance of your students and have the ability to do that thing. In India, there’s a major teacher absenteeism problem, so perhaps teachers can raise student performance just by showing up more. In the U.S., they’re already showing up, and the research on monetary incentives is not promising.

andrew fischer lees March 22, 2013 at 9:12 pm

sarcasm, yo

andrew fischer lees March 22, 2013 at 9:12 pm

sorry, that was for popeye

Wes Winham March 29, 2013 at 11:23 am

Emily: I would recommend you read the study.

From page 3:
“We measure changes in teacher behavior and the results suggest that the main mechanism for
the improved outcomes in incentive schools is not reduced teacher absence, but increased
teaching activity conditional on presence.”

It sounds like you’re familiar with some of the research on incentives, so it would seem beneficial to evaluate this research and adjust your priors a bit. http://econ.ucsd.edu/~kamurali/papers/Working%20Papers/Long%20Term%20Effects%20of%20Teacher%20Performance%20Pay.pdf

I don’t think the implications of this large result is that teaching is easy or that raising performance is easy. It seems to me that years of data on failed top-down initiatives is evidence that the both the problem is hard and that the techniques and strategies are not universal. Students are very different and it’s by empowering and rewarding teachers (not administrators or folks heavily removed from the act of teaching) that we have some hope for much-needed marginal improvement.

Rahul March 22, 2013 at 8:27 am

“the default ‘education quality improvement’ policy of the Government of India, which is reducing class size from 40 to 30 students per teacher”

That made me laugh. Status quo in India is 40 students per teacher?! What world are they living in! AFAIK 60-70 students-per-teacher is a more accurate number.

sunbomb March 22, 2013 at 10:59 am

In urban, private schools (from my experience a couple of decades ago) this was true. I seem to remember rural public schools having significantly smaller class sizes.

Bill March 22, 2013 at 9:14 am

Instead of monetary incentives, how about the possibility of losing your job for poor performance.

They must have a tenure system; otherwise this would be an option.

whatsthat March 22, 2013 at 9:59 am

Now compare results when the randomization is left upto the state, and local, governments.

Because that is the relevant comparison. No doubt Karthik Muralidharan designed his experiment well. But governments don’t implement experiments, they implement policy, and they will – more likely than not – tweak it to their needs.

What we need is a politics-robust policy. This is clearly impossible. But so is expecting a slow, not very clean bureaucracy to implement programs designed to avoid bias.

I find it surprising that Alex ignores the political in the political economy.

Wes Winham March 29, 2013 at 11:28 am

whatsthat: My guess is that Alex would very much agree with you. Here though, he’s not endorsing a specific policy or politician. The “Shout it from the Rooftops” theme is more about making a thing that is true more widely known. Currently, there’s still much debate about whether performance pay even works at all in the ideal situation. If we make it widely understood that it does work, then we can talk about specific policy implementations.

And just because we can agree that it does work when well-implemented doesn’t mean we have to agree on specific policy proposals to implement it. That’s where there’s room for analysis.

But if we allow folks to claim that it doesn’t work or use anecdotes to dismiss the idea, we can’t have the important discussion about how to effectively implement it (reserving the right to shout down any “pay for performance” policy that is simply political).

-Wes

JSIS March 22, 2013 at 9:59 am

Now, if they can replicate those results in private schools, I will take this study seriously. The baseline in govt schools is so low, anything will improve test scores. If they can include a control where no financial incentives were offered, but all the official govt regulations (attendance, class size, official equipmen,..t) were enforced, that would be great.

Can they point to what behaviour changes among teachers and students led to this improvement, that would be great.

Wes Winham March 29, 2013 at 11:33 am

JSIS: The baseline of govt schools is certainly low, but if it was really so easy to improve test scores, why do so many initiatives fail?

As far as identifying behavior changes, check out section 4.5:


4.5 Teacher Behavior
Our results on the impact of the programs on teacher behavior are mostly unchanged from
those reported in MS 2011. Particularly, over 5 years of measurement through unannounced
visits to schools, we find no difference in teacher attendance between control and incentive
schools (Table 9). We also find no significant difference between incentive and control schools
on any of the various indicators of classroom processes as measured by direct observation.
However, the teacher interviews, where teachers in both incentive and control schools were
asked unprompted questions about what they did differently during the school year, indicate that
teachers in incentive schools are significantly more likely to have assigned more homework and
class work, conducted extra classes beyond regular school hours, given practice tests, and paid
special attention to weaker children (Table 9).
Teachers in both GI and II schools report significantly higher levels of these activities than
teachers in control schools (Table 9 – columns 4 and 5). Teachers in II schools report higher
levels of each of these activities than those in GI schools as well, but these differences are not
always significant (column 6). While self-reported measures of teacher activity might be – 19 –
considered less credible than observations, we find a positive (and mostly significant) correlation
between the reported activities of teachers and the performance of their students (column 7)
suggesting that these self-reports were credible (especially since less than 40% of teachers in the
incentive schools report doing any one of the activities described in Table 9). In summary, it
appears that the incentive program based on end of year test scores did not change the teachers’
cost-benefit calculations on the attendance margin during the school year, but that it probably
made them exert more effort when present.

anotherpanacea March 22, 2013 at 10:06 am

I want to believe in these results, especially since Amartya Sen’s arguments India’s teaching culture and absenteeism leads me to believe that the incentives there have historically been misaligned.

Yet as a teacher I tend to think that less of the educational outcome is in my hands than in the students, and I know I’d be frustrated to have an incentive whose results were outside of my control that way. In DC’s public schools, we’ve seen teachers responding to incentives by cheating, which is certainly a more efficient response to the incentive; given what else we know about India’s teaching culture from Sen, that’s likely in India as well. Was that accounted for in this study?

Anthony March 22, 2013 at 10:34 am

It’s my understanding that in India the level started from is very, very low – such that having every teacher actually show up and say anything to the kids for a few hours would be an improvement over baseline. So if additional monetary incentives move teachers from “being on payroll” to “doing their job”, no matter how poorly, there’s going to be big improvement. I suspect that at the levels most American and European schools are at, the effects would be much subtler.

Jason K March 22, 2013 at 5:40 pm

Yep. The paper cites a study of India from 2005 that indicated on any given day about 25% of teachers were absent, and that less than half of teachers were engaged in any teaching activity. The moral is that when your norms are that bad there will likely be huge returns to policies that fix/mitigate those horrible norms. Still an important result considering how much of the world is suffering from this exact problem.

Ryan March 22, 2013 at 10:58 am

Very surprising. Every attempt at merit pay in the USA to date has shown no difference.

Rahul March 22, 2013 at 12:30 pm

Perhaps the difference is the low baseline. Indian public-school-system teachers are enormously underpaid (often as low as $5 per day). Many are doing a second or third odd job or even private tutoring. At the low level they are paid at they will work quite hard to get paid anything extra.

There are tons of low hanging fruit.

Brett March 23, 2013 at 2:35 am

That’s my guess as well. Beyond a certain level of “base-line” teaching, getting more effective teaching is difficult, and not something that can easily be motivated by more money.

Eric March 22, 2013 at 3:26 pm

another silver bullet solution, economist who think the changing of one factor/variable will fundmentally change a profoundly complex problem. With so many brilliant economists, no wonder India are so successful in educating their youth.

John March 22, 2013 at 6:25 pm

I’m not sure I’m reading this right but the implication seems to be that students only need onr oe a few good teachers (each year I would assume) to allow them to excel in their learning.

If that’s right it’s a pretty damning statement about the USA public schools and, I suspect, their administration.

Dismalist March 22, 2013 at 8:06 pm

There is value in such experiments. What bugs me is that an analogy to K-12 education institutions to any other institution, in food shopping, e.g., would never fly: You may buy at only one supermarket, purchasing a bundle defined by the supermarket. How long would anybody put up with such? And would we require a RCT to figure out that there were better ways of paying the employees of supermarkets? The public school system, always and everywhere, is fundamentally and irreparably inefficient. In addition, it hurts the worst off most.

Best Deals Hotel And Guarrantee Low March 25, 2013 at 5:56 am

This page definitely has all of the information and facts I needed concerning this subject and didn’t know who to ask.

Dianna March 26, 2013 at 5:59 am

(Try to restrain yourself with a toy that costs
a quarter, though. In that circumstance, a ordinary snow shovel will do.
While these items maybe fewer and far between, they
do exist.

Comments on this entry are closed.

Previous post:

Next post: