The World’s Biggest Field Experiment

A new paper (another summary) in Nature reports on what is perhaps the world’s biggest field experiment which has successfully shown how to, at scale, increase crop yields and reduce fertilizer usage in China. The scope of the 10 year experiment is astounding. The researchers first conducted thousands of field experiments all over China to discover and validate best practices:

A total of 13,123 site years of field trials were conducted from 2005 to
2015 for the three crops (n=6,089 for maize, 3,300 for rice and 3,734 for wheat), with sites spread across all agro-ecological zones…Each field trial included two types of management: conventional farmers’ practice (control) and ISSM-based recommendations (treatment; developed specifically for a given area). The recommended practices were discussed with local experts and participating farmers. Adjustments were made when necessary. Finally, the agreed-upon management technologies were implemented in the fields by the farmer; the collaborators provided guidance on-site during key operations, such
as sowing, fertilization, irrigation and harvest. Campaign collaborators recorded fertilizer rate, pesticide and energy use, and calculated nutrient application rate. At maturity, grain yield and above ground biomass were sampled by the collaborators for plots with a size of 6m^2 for wheat and rice, and 10m^2 for maize. Plant samples were dried at 70 °C in a forced-draft oven to constant weight, and grain yield was standardized at 14% moisture for all crops.

With validated best practices in hand the researchers and tens of thousands of collaborators then fanned out across the country to convince farmers to adopt the best practices.

During the campaign, about 14,000 training workshops, 21,000 field days, and more than 6,000 site demonstrations were organized by campaign staff; more than 337,000 pamphlets were distributed….During the campaign, we also encountered barriers and experienced challenges.  For example, we observed that some farmers appeared indifferent during some  outreach events. We later learned that it was mainly, because they could not comprehend the scientific content that we were trying to deliver. We solved the problem by having local (county or township) agents acting as an on-site ‘interpreter’ in  ways that speaks/connects with those farmers.

This was amusing:

It is also worth noting that the interests of agribusinesses do not always align with those of our campaign staff. For example, one of our main strategies used in the campaign was to select a site (for example, a village) for a given area, establish the base with field demonstrations of ISSM-based practices, then attract and engage more farmers from the same as well as neighbouring villages, creating  a snowballing and lasting effect. But sometimes, our partners in the private sector were more interested in changing sites so as to reach more farmer-clients. Vigorous  debates and discussion ensued. Eventually, the private sector personnel conformed to our reasoned schemes while using the established sites as demonstrations for  visitors from other areas.

Outputs and inputs among the treatment and control farmers were then measured (here I would have liked more information about the randomization. A lot can go wrong or be mismeasured at this stage.).

Farmers conducted all field operations. Campaign collaborators and/or extension agents were responsible for information and data collection. Typically, 10–30 farmers were randomly selected per ISSM-adopting site; another group of randomly selected 10–30 farmers from a nearby village without ISSM intervention served as a control/comparison. From the selected pool of farmers (roughly 14,600 paired data points), information on key management practices were obtained through a questionnaire survey, including crop varieties, planting densities, planting dates, fertilizer rates and harvest dates. For some sites, grain yields were directly measured in the same way as the field trials (see ‘Field trials’) for the selected 10–30 farmers. Yield and nitrogen rate were then averaged for each site.

The results were impressive.

Aggregated 10-year data showed an overall yield improvement of 10.8–11.5% and a reduction in the use of nitrogen fertilizers of 14.7–18.1%, when comparing ISSM-based interventions and the prevailing practices of the farmers. This led to a net increase of 33 Mt grains and a decrease of 1.2 Mt nitrogen fertilizer use during the 10-year period, equivalent to US$12.2 billion.

The entire experiment cost on the order of $56 million and generating $12.2 billion dollars of increased output, not including any environmental gains.

As if this weren’t enough the researchers then surveyed over 8 million smallholder farmers in China to estimate how much output could increase if the intervention were fully scaled.

What’s especially encouraging about this project is that no new technologies, seeds or infrastructure was involved–just basic science and a tremendous outreach campaign. Moreover, since the campaign increased profits it may continue to generate gains in the future even without further intervention as the practices spread. Repeated interventions will be necessary as climate changes, however. Information technology may makes this easier. China can be intimidating.


Truly, those researchers are out standing in their field.

Woof! If this was Reddit, you would have some gold.

Clearly they spent zero time in fields. All the crops with actual value involve millions of people working in fields.

Only economists would think a diet of bread, water, rice, and corn oil and corn syrup is a virtue.

The engineered tasteless tomato on most people's plate requires lots of field labor for an inferior product. For 10 months of the year in most places, only canned tomatoes make sense because they were harvested ripe and partially processed for cooking. For four months, putting workers in fidelds to deliver tomatoes for two months is worth it for the joy of a ripe juicy tomato.

But tomatoes are the I manage to grow myself. There are perhaps a hundred crops not corn, wheat, rice that I would not give up in exchange for cheap corn, rice, wheat. And before 1500, no one had access to more than one of the three: corn, rice, wheat, and each was no more than a third of the diet.

"Only economists would think a diet of bread, water, rice, and corn oil and corn syrup is a virtue."

Actually, a Russian economist proved that sauceless salads can provide most of the nutrients people need. He devised a Soviet project for Cuba. But T.R. Preston created a animal protein alternative.

Article: "For example, we observed that some farmers appeared indifferent during some outreach events" - and if those indifferent farmers were gay liars, as opposed to straight outliers, you could stay this was the case of reaching the low-lying fruit. Da da dumb.

There is simply so much that can go wrong in a study of this size. How can they be confident of their data? They have information from so many people. How do they know it is not all made up?

The scale of the research network created is impressive: 1,200 scientists, 65,000 local officials, 140,000 industry representatives and 21 million farmers across 37.7 million hectares.

In other words the government and a lot of government institutions were massively invested in success. And yet we are to believe that a group of intellectuals who have little to no experience of farming can teach those who have been doing it for thousands of years? Sure, it is possible. Is it likely?

How do they know something else has not happened? Perhaps better crop varieties have been introduced. Perhaps government incentives have changed. Perhaps the market is making more intensive agriculture more cost-effective. Ten years is a long time in a country that is growing so fast.

The least you can say is that such an experiment is unlikely to be replicable many places. $56 million over ten years? Just north of five and a half million per year? That wouldn't get a First World professor and three of his post-grads. But it seems scientists are cheap in China.

:::: "How can they be confident of their data?"

yup. My first (and second) impression is that this project is way over-hyped and the claimed "data" is suspect.

The bottom line grain output increase (10.8–11.5%) is not impressive by merely applying modern well-known agricultural techniques. Strongly doubt they they were able to measure crop yields with a tenth-percent accuracy, or even 5% accuracy. If this was a masterpiece of social-engineering -- the proof is lacking.
The tone of summary report reads like a PR/Marketing press release, not something written by a serious researcher. Alex seems easily impressed by such splashy but vacuous media research "reports" -- he found it somehow "astounding" (astonishing/amazing/stunning/ wondrous/impressive/brilliant/sensational).

I suppose you reject every claim the economy is better today than a year or decade ago. The data is all suspect, and all created by people motivated to prove things are better.

They can give input in the shape of new ideas (brainstorming), as biologists to try certain additional species, as chemists how to best fertilize the plants and what to use or avoid.

I don't think they revolutionized the local way of farming but just encouraged to optimize and try one or two new things. Sometimes that's all it takes.

News that is a cause for great optimism, triggering my contrarianism. Many of the people making the major decisions in China now have backgrounds in engineering. The benefits of learning to think like an engineer are well planned out large scale infrastructure projects to improve efficiency and productivity, like super-rail systems and in the present case here finding and implementing best agricultural practices. In the U.S. this would correspond to the Depression generation and projects like Hoover Dam and the expansion of the US Department of Agriculture research and information dissemination programs. H.G. Wells techno-utopianism was of that same era in world history. The detriments of thinking like engineers seems to be an increased risk of a presumption of can-do optimism and the perfectability of human nature by applying the same principles to people, resulting in the sort of blank-slatism micromanaging and forcing of human behaviors to try and achieve some version of utopia. (It was with considerable pleasure when I first read The Blank Slate to realize how well Pinker covered the general topic in the final chapters.) One possible example that might fit the scenario here could be the currently developing credit score system in China that will also track citizenship virtue. Aldous Huxley's satirical and dystopic Brave New World started out as a kind of rebuttal to the techno-utopianism one of HG Wells' stories. Wells was a big influence on the Italian Futurists which were allies to Mussolini (he who 'made the trains run on time') in his rise to power, but which once he attained real power he discarded in favor of more traditionalism. Roughly -- make Rome great again. (And at the risk of extending remarks a few words too far, the science-engineering optimism-pessimism divide shows up again with Dune (1961) being a response against The Foundation (1951).)

This led to a net increase of 33 Mt grains and a decrease of 1.2 Mt nitrogen fertilizer use during the 10-year period, equivalent to US$12.2 billion.

21 million farmers. An extra 33 million tons of grain. Rice is selling for about $400 per ton on Ali Baba. So that is an extra $650 per farmer. They seem to be using a lower number because they seem to be claiming savings of $580.

However it is not per year, it is over ten years. That is less than 20 cents per day. It is a reflection of the poverty of many Asian farmers that this is not nothing. But is it worth the effort?

$12.2 billion gain from a $56 million investment? 217x return on any government program? I'll take it, especially the since the gain will continue without any new investment (hopefully grow as the use of the technique expands).

So, people have forgotten how this worked in the U.S.? 'Extension’s roots go back to agricultural clubs and societies, which sprang up after the American Revolution in the early 1800s. In 1819, a pioneer agriculture journal entitled American Farmer encouraged farmers to report on their achievements and their methods of solving problems.

The Smith Lever Act formalized extension in 1914, establishing USDA's partnership with land-grant universities to apply research and provide education in agriculture. Congress created the extension system to address exclusively rural, agricultural issues. At that time, more than 50 percent of the U.S. population lived in rural areas, and 30 percent of the workforce was engaged in farming.

Extension's engagement with rural America helped make possible the American agricultural revolution, which dramatically increased farm productivity, allowing fewer farmers to produce more food.'

Along with 4-H - ' 4-H didn't really start in one time or place. It began around the start of the 20th century in the work of several people in different parts of the United States who were concerned about young people.

During the late 1800's, researchers at public universities saw that adults in the farming communities did not readily accept the new agricultural discoveries being developed on university campuses - practices like using hybrid seed corn, milk sanitation and better home canning procedures. However, the researchers found that young people were open to new thinking and would "experiment" with new ideas and share their experiences and successes with their parents. In this way, rural youth programs became an innovative way to introduce new agriculture technology to their communities.'

The first Nature link is not public (though anyone willing to contribute the necessary Mason NetID and password is welcome to), but for those wondering what ISSM means, it is likely that this paper could help - 'Integrated soil–crop system management for food security'

Mike Pence agrees with you: "Gov. Mike Pence noted in a proclamation that Extension's work includes.....providing research-based information that has helped farmers and ranchers established the U.S. as a world leader in agriculture."

Looking the yields and scale of operation, the US is world leader in agriculture and plant science. There are places where productivity is higher, but large absolute production is an specialization by itself. Logistics are underappreciated.

'Logistics are underappreciated.'

Sort of - railroads played a major role in shaping agricultural communities west of the Mississippi. However, the way that the Mississippi is positioned to basically funnel a continent's worth of agriculture (plus the Great Lakes in addition) means that large scale logistics were possible as major farming areas were established in North America (following the previous expansion of cattle ranching, where logistics were somewhat easier).

Kudos to them. But for such a relatively inexpensive public-information campaign to produce such outsize results suggests that the baseline level of education and information-sharing infrastructure in rural China must be really low.

it would be a good idea not to think that everybody else is using as much fertilizer as China... Data from the World Bank, as printed in Bloomberg:

Here is an Upshot article on China's success in reducing air pollution: The author of the article points out that success was achieved by government fiat not markets, and predicts that further success must depend on markets: "China’s early reductions in air pollution have been achieved through an engineering-style fiat that dictates specific actions, rather than relying on markets to find the least expensive methods to reduce pollution. It’s an approach that has come with some real costs — as the many people left without heat this winter could attest. Yet further improvements will also be much costlier than necessary if they too are pursued by fiat, particularly with many of the easier fixes having already been made. In the decades after enactment of the Clean Air Act, American policymakers have used many tools to reduce pollution, with market-based regulations having proved the most cost-effective. Although China is experimenting with a cap-and-trade system for carbon dioxide, it has not yet turned to such policies to fight conventional pollution. It would be quite a twist if so-called Communist China ultimately wins the war against pollution by embracing market-based regulations, while the United States continues to use them only intermittently".

That has to be the dumbest article from the NYT ever. If only China, which does not use market mechanisms, used market mechanisms then that would show up America which does use them but not as much as the Times would like.

And yet much of this escapes Rayward, who has i) a large brain, ii) a good education, and iii) the curiosity and leisure to think about these things.


Ideological blinkers? Or merely mood affiliation?

"Ideological blinkers? Or merely mood affiliation?"

What's the difference in these two?

The emerging standard in development economics for programs like this is: (i) a series of small randomized trials to prove the concept; (ii) a series of medium-scale randomized trials to establish the potential for scale and challenges that emerge at scale; and only then (iii) large scale implementation, with continued measurement. This is the approach that gave us deworming and anti-malaria programs rated by Givewell as the most cost effective anti-poverty programs around. Programs like No Lean Season and the Bangladeshi "graduation" programs are somewhere between phases 2 and 3 but look promising.

This is emerging as a standard because so many large scale programs that seemed really promising at the outset proved to fail -- decades of research showed that ideas generated by international bureaucrats and researchers rarely had the promised transformational effects. The Millennium Villages represent the worst instance of this, but there are hundreds of other examples.

The language of the project sounds a lot like the failed history of many agricultural development projects. From the conclusion of the short Nature article: the key innovation of the project was, "persuading often intractable rural farmers to change their practices." The history of development projects has more often shown that the intractable rural farmers understand their local constraints much better than external "experts."

The ISSM program described here seems to have jumped immediately to a phase 2, and is claiming startling success. If true, then it seems that the researchers were quite lucky, since dozens and dozens of efforts around the world to teach farmers how to farm have had nothing like this level of success. It's even more surprising in China, where robust agricultural extension programs have been in place for decades, presumably also featuring skilled staff helping farmers make good decisions.

Alternately, perhaps when a 10-year $56 million state-sponsored research program is on the line, there are many incentives to find a positive result. Is there any world where this project reaches the more common conclusion that the project had a neutral or ambiguous impact?

This is the concern with a research paradigm centered around "the world's biggest field experiment." Because the experiment is unlikely to be replicated, at the end of the day, we have to take the results on our faith in the scientists and their institutional backers. At the end of the day, this project probably shifts the Bayesian needle on agricultural extension just a tiny bit, and less than it would if this were a dozen trials by a dozen different research groups.

Hopefully the researchers will have the courage to post their full data series and allow others to look into their findings. But my guess is that they will not.

Yes, this study started small and kept growing. Here for example is a journal article from 2011 by a team including this latest Nature article’s lead author, Cui, that talks about 66 experimental sites from 2006 to 2009:

Don’t have access to the Nature article itself so I am guessing a clearer explanation is provided of how the studies and experiments were aggregated over time. At any rate, the hype that produces the impression that millions of farmers were involved in an experiment for 10 years is misleading.

What would be really interesting is to see what sort of insurance was offered to participating farmers. I imagine there had to be some kind of make-whole arrangement if the farmer lost out on the experiment. Farmers are very smart people. You don’t survive in it if you are not. Risk probably explains a lot about why the Chinese farmers had not adopted the higher yielding experimental techniques earlier.

I'm wondering why so many of you are skeptical here. There's ample evidence that applying better techniques can improve agricultural yields -- cf Green Revolution. A 10% increase in yield over a number of years seems totally achievable.

How often is research on this scale faked? Anyone want to provide some examples?

It seems too small to me - the Green revolution I think achieved 100% increases in yields. 10% over 10 years is below what I would think could be expected without specific intervention, just by improvements in seeds. Crops are also highly seasonal (duh) and yields can change by 30 or 40% just by weather, so a change of 10% seems within the noise.

One item I noticed in the quotes provided by Alex (I cannot get to the full paper) is that the program sought the input of the local farmers. Anybody forgot that local participation was a key factor when Ostrom's studies found successful use of commons? I think clockwork_prior gets it on this one - as he notes the US equivalent of agricultural extensions and 4-H programs. Ostrom's work has significant application to more than commons.

Potemkin progress!

A strange set of comments here ranging from "it's a lie, they faked their data" to "so what, the Green Revolution gives 100% increases in yields".

I'm not sure why it's so hard to believe that Chinese farmers, many of whom are still using 19th century technology and methods, might be induced to increase their productivity. Nor why it's hard to believe that just because the US radically increased its productivity, then China might not have done so and thus has room for improvement. But said improvement isn't going to happen automagically, that's why the US has land grant universities and extension schools.

But according to many commenters here (on other threads), education is merely signaling so there is no way that the ag universities are doing any good.

China is so far behind they think they are in first place.

Heady and Dillon big time. They must be rejoicing.....

Since 1955, U.S. corn yields have increased at a rate of 1.9 bu/A which is easily 1% per year.

Comments for this post are closed