What the Randomistas Taught TOMS
You probably know the story of Tom’s shoes (here told by Andrew Leigh):
After a visit to Argentina businessman Blake Mycoskie decided he wanted to do something about the lack of decent footwear in developing nations. A talented entrepreneur, Mycoskie had founded and sold four companies by his thirtieth birthday. Now he was affected by the poverty he saw in villages outside Buenos Aires”… “I saw the real effects of being shoeless: the blisters, the sores, the infections.”
To provide shoes to those children, Mycoskie founded ‘Shoes for Better Tomorrows’, which was soon shortened to TOMS. The company made its customers a one-for-one promise: buy a pair of shoes and TOMS will donate a pair to a need child. Since 2006, TOMS has given away 60 million pairs of shoes.
Perhaps you see where this is going (but don’t be too sure!):
Six years in, Mycoskie and his team wanted to know what impact TOMS was having, so they made the brave decision to let economists randomize shoe distribution across eighteen communities in El Salvador…
The results from the World Bank study were not great:
Results indicate high levels of usage and approval of the shoes by children in the treatment group, and time diaries show modest evidence that the donated shoes allocated children’s time toward outdoor activities. Difference-in-difference and ANCOVA estimates find generally insignificant impacts on overall health, foot health, and self-esteem but small positive impacts on school attendance for boys. Children receiving the shoes were significantly more likely to state that outsiders should provide for the needs of their family. Thus, in a context where most children already own at least one pair of shoes, the overall impact of the shoe donation program appears to be negligible, illustrating the importance of more careful targeting of in-kind donation programs.
In other words, the shoes didn’t add much to health but did increase feelings of dependency. Another bubble punctured by economists. End of story, right? No. To their great credit TOMS took the results to heart. TOMS reevaluated how they give, they made adjustments, they changed. The lead researcher Bruce Wydick wrote:
By our agreement, [TOMS] could have chosen to remain anonymous on the study; they didn’t…For every TOMS, there are many more, both secular and faith-based, who are reticent to have the impacts of the program scrutinized carefully by outside researchers…many organizations today continue to avoid rigorous evaluation, relying on marketing cliches and feel-good giving to bring in donor cash. TOMS is different…
Will TOMS’ new methods work better? Only randomization will tell. Fortunately, TOMS is committed to doing just that.
This is from Andrew Leigh’s excellent new book Randomistas. Leigh tells the story of how randomized controlled trials are being used to improve teaching, crime fighting, charitable giving and more. It’s a good read and even in areas that I know well, such as crime research, I learned new information. Leigh is also careful to point to the studies that didn’t replicate as well as those that did.
By the way, Leigh is a person to keep to an eye on. In 2011, he was awarded the Economic Society of Australia’s Young Economist Award, given to “honour that Australian economist under the age of forty who is deemed to have made a significant contribution to economic thought and knowledge.” As you can see from Google Scholar this was a well-deserved award. Yet even as he received this award, Leigh had already left his position at Australian National University to embark on a second career as a Member of Parliament. Since starting his second career, however, he has published three well received books about politics, economics, and inequality and now the world of randomized controlled trials! Look for Andrew as a future Australian Treasurer and who knows what more.