In 2004, Jeff Sachs and co-authors revived an old theory to explain Africa’s failure to develop, the poverty trap, and an old solution, the big push.

Our explanation is that tropical Africa, even the well-governed parts, is stuck in a poverty trap, too poor to achieve robust, high levels of economic growth and, in many places, simply too poor to grow at all. More policy or governance reform, by itself, will not be sufficient to over-come this trap. Specifically, Africa’s extreme poverty leads to low national saving rates, which in turn lead to low or negative economic growth rates. Low domestic saving is not offset by large inflows of private foreign capital, for example foreign direct investment, because Africa’s poor infrastructure and weak human capital discourage such inflows. With very low domestic saving and low rates of market-based foreign capital inflows, there is little in Africa’s current dynamics that promotes an escape from poverty. Something new is needed.

We argue that what is needed is a “big push” in public investments to produce a rapid “step” increase in Africa’s underlying productivity, both rural and urban.

As the title of the blog might suggest, I was skeptical. But even if a big push wasn’t exactly the right idea, I’m all in favor of Big Ideas and Sachs pursued his Big Idea with tremendous skill and media savvy. Pilot programs were soon up and running and then quickly expanded into full programs. In June 2010, the Millennium Villages Project released its first public evaluation and that is when things started to fall apart.

The initial MVP evaluation claimed great success but simply compared some development indicators before and after in the treated villages without comparing to trends elsewhere. In 2010 such a study was completely out of step with contemporary practices in impact evaluation. Red flag! Clemens and Demombynes showed that comparing to trends elsewhere significantly moderated the impact. A second MVP paper was published in the Lancet but then was quickly retracted when Bump, Clemens, Demombynes and Haddad demonstrated that it had  significant errors. Clemens and Demombynes wrote a summary piece on the controversy then in an astounding and under-reported scandal the MVP tried to stifle Clemens and Demombynes. The MVP, with Jeff Sachs at the head, also sicced their lawyers on Nina Munk and her book, The Idealist: Jeffrey Sachs and the Quest to End Poverty. More red flags.

Yet, despite all of this controversy and bad behavior, the MVP project continued to move ahead and in 2012, the UK Department for International Development (DFID) funded US $11 million into an MVP in Northern Ghana that ran until December 2016. Under the auspices of the DFID, we now finally have the first in-depth, independent evaluation of one MVP project and it doesn’t look great. The project did some good but the big push failed and the good that was done could have been done at lower cost.

Overall, the MVP in northern Ghana did not achieve the overall MDG target to reduce extreme poverty and hunger at the local level. Where there are attributable changes to the MDG targets, these tended to be the more limited changes than those that will fundamentally improve people’s health, educational and other outcomes. For instance, the project did increase attendance at primary school (Goal 2) but did not go beyond this MDG and improve the learning outcomes of children; the project did increase the proportion of births attended by professionals and women said to be using contraceptive methods (MDG indicators), but it is not possible to assess the effect on maternal health (Goal 5); and the project did increase the number of toilets (a target under Goal 7), but not beyond this MDG in terms of hygiene and sanitation practices. There are, however, exceptions. The project had a remarkable impact on stunting, which is a long-term health indicator and a predictor of socioeconomic outcomes in adulthood.

So the MVP had some good effects on some indicators:

But is this impact sufficient given the size of the investment? And, by doing everything together, is there a synergistic effect that offers greater value for money than would arise through implementing individual sector-based interventions? In our cost-effectiveness analysis, we demonstrate that the project has so far not yielded sufficiently positive results, and what has been achieved could have been attained at a substantially lower cost (even when we take account of investments made for future usage). As such, the project seems to have fallen short of producing a synergistic effect; and the impact is not large enough for the project to be regarded as cost-effective, even when each sector is assessed independently of the others. Of course, in the longer run, the MVP may produce welfare gains. Importantly the investments in improving the health care service may enhance health outcomes later on; or other considerable investments in infrastructure (roads, health and school facilities) may have an impact on future outcomes. 

Perhaps then, the most concerning findings are the early indications that the MVP approach will be difficult to be sustained by district institutions and at the community level; and there are signs that any gains made under the project are already being undermined.

Addendum: Andrew Gelman and co-authors, including Jeff Sachs, offer a broadly similar although less negative in tone evaluation of the entire MVP project.