The new RCT results on poverty reduction

by on May 15, 2015 at 3:50 am in Data Source, Economics, Medicine | Permalink

Declan Butler reports:

Giving some of the world’s poorest people a two-year aid package — including cash, food, health-care services, skills training and advice — improves their livelihoods for at least a year after the support is cut off, according to the results of an experiment involving more than 10,000 households in six countries.

The poverty intervention had already been trialled successfully in Bangladesh, and the study’s researchers say it shows the approach works in other cultures too. “We finally have truly credible evidence that a programme for the poorest of the poor can really help them meaningfully reduce their poverty,” says Dean Karlan, an economist at Yale University in New Haven, Connecticut, and a co-author of the study, reported today in Science. “Until now, we haven’t really been able to go to a government outside Bangladesh and say, we’re confident this works.”

Ethiopia, one of the countries that was in the trial, is planning to continue and scale up the intervention to cover around 3 million people, says Karlan, and Pakistan and India are considering scaling up interventions, too.

Banerjee and Duflo are involved in the work as well, and this is sometimes called the “graduation model,” because the aim is to graduate people out of poverty.  Note this:

The intervention is not cheap. Costs per household ranged from $1,455 in India to $5,962 in Pakistan, although they were offset by positive returns on investment ranging from 133% in Ghana to 433% in India. The researchers hope to cut costs in future by scaling back the experiment’s more expensive components, such as training.

And while the model worked in many places, it failed in rural southern India and Honduras, in part due to…problems with chickens.  Nonetheless this is big, big news.  The link to the original research is here.

For pointers I thank Kevin Lewis and Michelle Dawson.

1 MikeP May 15, 2015 at 5:55 am

It would be much more surprising if recepients did not see a benefit one year out at those levels of aid. Balancing this out is now the incentive in those spots to be poor enough that you qualify for the aid.

2 chuck martel May 15, 2015 at 6:06 am

“The graduation model is not a panacea. Historically, the biggest reductions in extreme poverty have resulted from economic growth, notes Karlan. The percentage of those living on less than $1.25 a day in developing countries fell from 43% in 1990 to 17% in 2011, driven largely by economic growth in China and India.”

3 Brian Donohue May 15, 2015 at 10:05 am

It’s the story of the 21st century, except no one noticed.

4 Ziel May 15, 2015 at 6:17 am

The obvious problem in scaling this program is that not everyone can be a farmer or shopkeeper- the increased competition alone will shut a lot of them down.

5 ZZZ May 15, 2015 at 1:46 pm

It probably lowered the income of the existing poor farmers and shopkeepers too.

6 James C May 15, 2015 at 6:38 am

What message does it send communities when we give money and assets to people because specifically because they are they poorest? Interesting results but the intervention is very expensive and one could assume that returns would diminish as scaling happens because of general equilibrium effects.

Given my own experience in Africa I am very skeptical about the 5% social discount rate (or even 10%) as other studies and personal experience show that the poor change their primary employment very often. Not sure the “big push” is enough to change that but interesting results nonetheless. Karlan’s statement, though may be true, seems to set the bar far too low in my mind for such a strong statement.

7 Will May 15, 2015 at 8:53 am

This reads a bit like an Onion headline, to me. “Economists give poor $5000 and they are no longer poor.”

If this is “big, big news” it seems like Development Economics is in a sorry state, no?

8 Lewis May 15, 2015 at 9:44 am

You obviously don’t pay attention to development economics. The typical counter-argument is that if you give poor people money they will just consume it, waste it, or invest it so poorly that they don’t get a good return. What this current research agenda is showing is the opposite — if you give poor people money, they (in the aggregate) make smart decisions with that money and get positive rates of return on their investments — i.e., the initial capital infusion becomes self-sustaining and lifts them out of poverty over time. Its not just the effect of them getting $5000 more, but about how they then grow that initial infusion through subsequent investment decisions.

Compare this to arguments we hear in this country all the time about how giving poor people money would induce laziness, make them not want to work, they would just use it all on drugs, etc. These studies find the opposite — people take the money and invest it wisely in small businesses, and grow.

9 Will May 15, 2015 at 10:32 am

@Lewis. I’m aware of the claims you mention, and my comment was surely a bit tongue in cheek. But really, development economics deals with promoting economic growth in low-income nations. In all nations, we are pretty sure that certain policies (access to credit, property rights, healthcare…) promote economic growth.

When development economics finds that a holistic and not particularly low cost intervention yields consistent and non-trivial results, I don’t feel as ecstatic. Of course, I am all for ending poverty, but it just seems fairly obvious and Onion-like.

The authors write:

Following identification of the beneficiary households through a participatory process in the village, the six activities are:

1. Productive asset transfer: a one-time transfer of a productive asset

2. Consumption support: a regular transfer of food or cash for a few months to about a year (11)

3. Technical skills training on managing the particular productive assets

4. High-frequency home visits

5. Savings: access to a savings account and in some instances a deposit collection service and/or mandatory savings

6. Some health education, basic health services, and/or life-skills training

If these 6 interventions don’t help, what will?

10 Seth May 15, 2015 at 10:36 am

I think you are confusing development economics and developed economics.

I thought the typical counter argument in development economics was that poor property rights (e.g. warlords, dictators or the common local thug taking stuff of value without consequence) created an environment where people laid low and chose not to add value since it would be taken and they wouldn’t be able to benefit from their investments and efforts.

That being said, is it really all that remarkable that someone’s life who was living on a buck a day could still be benefiting a year after a two year gift valued at more than 10x their typical consumption? What would you expect the typical decay rate to be of a gift that large?

11 teenwolf May 15, 2015 at 9:35 am

“…problems with chickens”

eaten by chupacabra or regional variant

12 Tiago May 15, 2015 at 9:37 am

Will, if we take a simple average on the gains, we have 283% expected return on investment. Maybe the distribution leans to the lower side, but you see the point. Anything above 200% is indeed huge.
What I do have doubts is on scaling. Programs that involve training typically don’t reproduce well after you increase the numbers of people taking them.

13 moreno klaus May 15, 2015 at 11:05 am

Scaling + short time horizon. I want to see what happens after 5 years!

14 R Richard Schweitzer May 15, 2015 at 10:28 am

It appears that here again Duflo is well into “what” shapes motivations.
Following up with an examination of what types of motivations take form and with what relative strengths (desires, wants and “needs”).

Having previously “settled” in (or been restricted to) particular consumption and participation patterns; then “exposed” to the personal effects of different patterns (and what they provided), the motivations toward the different (or away from the prior) come into play in varying intensities and durations.

15 R Richard Schweitzer May 15, 2015 at 10:40 am

Looking at the comments, one might conclude avoidance of the basis of the Banarjee-Duflo work.

Human conduct is based on motivation.

Human conduct produces economic circumstances; rather than “economic conditions” producing human conduct.

If we wish to examine ways that human conduct might produce “different” economic circumstances, we need to examine what motivations can impact human conduct, in what ways, to what degree; and to what effects for what periods (or generations).

Look at Duflo’s works. They examine influences and impacts on motivations.

16 Rich Berger May 15, 2015 at 11:45 am

Do these returns seem absurdly high?

17 Bob McGrew May 15, 2015 at 11:47 am

I thought recent research on micro loans showed that they mostly did not graduate people from poverty. But with the ROI shown above, it should be completely possible to scale up the intervention with micro loans.

Any thought on how to resolve this puzzle?

18 TMC May 15, 2015 at 12:37 pm

” improves their livelihoods for at least a year “. If it had been longer, that would be the claim then.

An one year effect sounds like a failure.

19 Jan May 15, 2015 at 12:47 pm

To everyone unimpressed by the limited timeline, how much time would need to pass with good outcomes to convince you?

20 Cliff May 15, 2015 at 4:46 pm

Convince us of what?

21 Dan Weber May 15, 2015 at 7:54 pm

Previous comment got eaten, but summary is:

1. longer is better

2. best result is getting kids out of poverty

3. best way to be assured is an attempt to replicate with a larger group

22 Jan May 15, 2015 at 8:06 pm

How long? What proportion of kids? How would you evaluate generalizability?

23 Economist May 15, 2015 at 2:24 pm

Does anyone know if they have shared their underlying data and code ?

24 Urstoff May 15, 2015 at 3:13 pm

Summary: give people goats and they will prosper.

25 Chris Prottas May 15, 2015 at 3:52 pm

As someone who works at INGO (Trickle Up) that implements the graduation approach referenced, we are very excited about the results. Sustainably increasing income and in turn consumption has proven incredibly difficult, and many other approaches have tried and failed. No intervention is a silver bullet, but here we have found something very effective for what it appeared entirely intractable for people living in extreme poverty and social marginalization. The summaries also raise the critical question of how we scale this approach. At TU, We have begun scaling through technical assistance and government partnerships, and we and are now exploring how we can best scale with quality through a combination of capacity-building and performance incentives. I’d certainly love to hear if other organizations are exploring how we can best overcome the operational challenges of scaling in low-resource environments.

26 chrisare May 15, 2015 at 4:26 pm

It’s stretching the word to its breaking point to define results 1 year after an intervention as “sustainable”. Given the short time frame and large investment, I agree with those saying the results are not at all surprising.

27 Rich Berger May 15, 2015 at 4:28 pm

I looked at Table 4 (costs and benefits), and the lion’s share of the benefits are attributable to assuming that year 3 gains continue in perpetuity. Seems like a very dubious basis for claiming success.

28 Economist May 15, 2015 at 5:49 pm

I just could not believe your claim (given the reputation of the authors) and had to look it up myself. But, there it is, sitting proudly on pg 18 of the supplement.

“We employ a Permanent Income Hypothesis, and thus assume that the consumption gains observed at year 3 exist in perpetuity”

This paper is ridiculously sketchy. No wonder they sent it to Science. It would not stand even a little scrutiny at any economics journal. (Well they could have sent it one of the journals they edit.)
This was one (of many) critiques of Sach’s similar work. Sustainability was unclear or let’s be honest absent, with his MVP. At least Sachs argued against the importance of sustainability on ethical grounds. Much more honest, but still a bit ridiculous.

Other sketchy parts (this is from a quick glance : willing to bet that the paper is full of holes and careless/self serving assumptions) :
i) The discount rate is unbelievably low – 5 percent, 7 percent and 10 percent. International organizations use 12 to 15 percent. That itself might be conservative. The 5 percent comes from here :
This is not equivalent to the opportunity of cost of capital employed in the country. It would be surprising that such a well regarded list of economists are so dumb and don’t understand the basics of development – but that has been established in so many other parts of the paper. The fact that there are ignorant is surprising, but the fact that there have made so many careless mistakes throughout the paper reflects arrogance. Who edited and refereed this drivel ?

ii) The countries have very unequal samples but all observations are treated equally in the main specification. Why ? They claim this is standard practice in RCTs. But these people set the standard. The claim : “Regressions that instead weigh each country equally generate similar results.” Then why not report those ? Anytime you see “standard practice” without an accompanying reason: think fraud. There are many others to reach that conclusion here.

iii) In some places the intervention is the lean season, in others it is for 13-40 weeks. The interventions are not close to similar across countries. Still there is an underlying pretense that the interventions and results hold “generally”.

I asked earlier for the data and code. Now I don’t care. I don’t believe this rubbish anyway.

29 % May 16, 2015 at 8:24 am

Am I reading this right? They target people who make $250 a year, give them five grand worth of stuff and a year later they have a 14% increase in assets? The intervention was 2000% of their yearly income and produced a 14% increase in assets? Is this off the baseline of their prior income or post-intervention? Because if it is the latter, this is completely underwhelming, and if it is the former, it is a colossal waste of money. One might as well burn the cash for fuel.

Comments on this entry are closed.

Previous post:

Next post: