Category: Economics

350+ coauthors study reproducibility in economics

Jon Hartley is one I know, here is the abstract:

This study pushes our understanding of research reliability by reproducing and replicating claims from 110 papers in leading economic and political science journals. The analysis involves computational reproducibility checks and robustness assessments. It reveals several patterns. First, we uncover a high rate of fully computationally reproducible results (over 85%). Second, excluding minor issues like missing packages or broken pathways, we uncover coding errors for about 25% of studies, with some studies containing multiple errors. Third, we test the robustness of the results to 5,511 re-analyses. We find a robustness reproducibility of about 70%. Robustness reproducibility rates are relatively higher for re-analyses that introduce new data and lower for re-analyses that change the sample or the definition of the dependent variable. Fourth, 52% of re-analysis effect size estimates are smaller than the original published estimates and the average statistical significance of a re-analysis is 77% of the original. Lastly, we rely on six teams of researchers working independently to answer eight additional research questions on the determinants of robustness reproducibility. Most teams find a negative relationship between replicators’ experience and reproducibility, while finding no relationship between reproducibility and the provision of intermediate or even raw data combined with the necessary cleaning codes.

Here is the full paper, here are some Twitter images.  I have added the emphasis on the last sentence.

A simple model of AI and social media

One MR reader, Luca Piron, writes to me:

 I found myself puzzled by a thought you expressed during your interview with Professor Haidt. In particular, from my understanding you suggested that in the near future AI will be able to sum up the content a user may want to see into a digest, so that they can spend less time using their devices.

I think that is a misunderstanding of how the typical user experiences social media. While there surely are some brilliant people such as the young scientists you described during the episode who use social media only to connect with peers and find valuable information, I would argue that most users, alas including myself, turn to social media when seeking mindless distraction, when bored or maybe too tired to read of watch a film. Therefore, having a digest will prove unsatisfactory. What a typical user wants is the stream of content to continue.

I think these are some of the least understood points of 2024.  Let us start with the substitution effect.  The “digest” feature of AI will soon let you turn your feeds into summaries and pointers to the important parts.  In other words, you will be able to consume those feeds more quickly.  In some cases the quality of the feed experience may go up, in other cases it may go down (presumably over time quality of the digest will improve).

We all know that if tech allows you to cook more quickly (e.g., microwave ovens), you will spend less time cooking.  That is true even if you are “addicted” to cooking, if you cook because of social pressures, if cooking puts you into a daze, or whatever.  The substitution effect still applies, noting that in some cases the new tech may make the cooked food better, in other cases worse.  In similar fashion, you will spend less time with your feed, following the advent of AI feed digests.

Somehow people do not want to acknowledge the price theory aspect of the problem, as they are content to repeat the motives of young people in spending time with their feeds.  (You will note there is the possibility of a broader portfolio effect — AI might liberate you from many tasks, and you could end up spending more time with your feed.  I’ll just say don’t bet against the substitution effect, it almost always dominates!  And yes for addictive goods too.  In fact those demand curves usually don’t look any different.)  No one has to be a young genius scientist for the substitution effect to hold.

Note that a majority of U.S. teens report they spend about the right amount of time on social media apps (8% say “too little time”) and they are going to respond to technological changes with pretty normal kinds of behavior.

I think what has in fact happened is that commentators have read dozens of MSM articles about “algorithms,” and mostly are not following very recent tech developments, including in the consumer AI field.  Perhaps that is why they have difficult processing what is a simple, straightforward argument, based on a first-order effect.

Another general way of putting the point, not as simple as a demand curve but still pretty straightforward, is that if tech creates a social problem, other forms of tech will be innovated and mobilized to help address that problem.  Again, that is not a framing you get very often from MSM.

The AI example is also a forcing one when it comes to motives for spending time with social media feeds.  Many critics wish to have it both ways.  They want to say “the feed is no fun, teenagers stick with the feed because of social pressures to be in touch with others, but they ideally would rather do something else.”  But when a new technology allows them to secede from feed obsession to some degree, (some of) those same critics say: “They can’t/won’t secede — they are addicted!”  The word “dopamine” is then likely to follow, though rarely the word “fun.”

It is better to just start by admitting that the feed is fun, and informative, for many teenagers and adults too.  Of course not everything fun is good for you, but the “social pressure” verbal gambit is a slight of hand to make social media sound like an obvious bad across all margins, and a network that needs to be taken down, rather than something we ought to help people manage better, at the margin.  If it really were mainly a social pressure problem, it would be relatively easy to solve.

For many teens, both motives operate, namely scrolling the feed is fun, and there are social pressures to stay informed.  The advent of the AI digest will allow those same individuals to cut back on the social pressure obligations, but keep the fun scrolling.  Again, a substitution effect will operate, and furthermore it will nudge individuals away from the harmful social pressures and closer to the fun.

As Katherine Boyle pointed out on Twitter, a lot of this debate is being conducted in terms of 2016 technology.  But in fact we are in 2024, not far from the summer of 2024, and soon to enter 2025.  Beware of regulatory proposals, and social welfare analyses, that do not acknowledge that fact.

In the meantime, please do heed the substitution effect.

Your Subsidies are Undercutting My Subsidies!

NYTimes: Treasury officials say that they fear that elevated Chinese production targets are causing its firms to produce far more electric vehicles, batteries and solar panels than global markets can absorb, driving prices lower and disrupting production around the world. They fear that these spillovers will hurt businesses that are planning investments in the United States with tax credits and subsidies that were created through the Inflation Reduction Act of 2022, a law that is pumping more than $2 trillion into clean energy infrastructure.

Amazing that Yellen can say this with a straight face:

as an economist, it was her view that China could benefit if it stopped giving subsidies to firms that would fail without government support.

What should I ask Joe Stiglitz?

I will be having a Conversation with him, and please note this is the conversation I want to have, not the one you want me to have.  So what should I ask?  Note that Joe has a new book coming out The Road to Freedom: Economics and the Good Society.  That said, I also would like for the dialogue to cover Joe’s career more generally, starting with 1970 or so.

So what should I ask him?

Lockean homesteading for goats, bonus added

The mayor of an Italian island is attempting to solve an animal overpopulation problem with an unusual offer: free goats for anyone who can catch them.

Riccardo Gullo, the mayor of Alicudi, in Sicily’s Aeolian archipelago, introduced an “adopt-a-goat” program when the small island’s wild goat population grew to six times the human population of about 100.

Gullo said anyone who emails a request to the local government and pays a $17 “stamp fee” can take as many goats as they wish, as long as they transport them off the island within 15 days of approval.

“Anyone can make a request for a goat, it doesn’t have to be a farmer, and there are no restrictions on numbers,” he told The Guardian.

He said the scheme is currently available until April 10, but he will extend the deadline until the goat population is back down to a more manageable number.

The mayor told CNN that officials will not investigate the intentions of prospective goat owners, but “ideally, we would like to see people try to domesticate the animals rather than eat them.”

Here is the full story, via the excellent Samir Varma.

Algorithmic Collusion by Large Language Models

The rise of algorithmic pricing raises concerns of algorithmic collusion. We conduct experiments with algorithmic pricing agents based on Large Language Models (LLMs), and specifically GPT-4. We find that (1) LLM-based agents are adept at pricing tasks, (2) LLM-based pricing agents autonomously collude in oligopoly settings to the detriment of consumers, and (3) variation in seemingly innocuous phrases in LLM instructions (“prompts”) may increase collusion. These results extend to auction settings. Our findings underscore the need for antitrust regulation regarding algorithmic pricing, and uncover regulatory challenges unique to LLM-based pricing agents.

That is a new paper by Sara Fish, Yannai A. Gonczarowski, and Ran I. Shorrer.  The authors are running too quickly into their policy conclusion there (how about removing legal barriers to free entry in many cases? not worth a mention?), but nonetheless very interesting work.  Via Ethan Mollick.

Generative AI for economists

From Anton Korinek here is a recent paper:

Generative AI, in particular large language models (LLMs) such as ChatGPT, has the potential to revolutionize research. I describe dozens of use cases along six domains in which LLMs are starting to become useful as both research assistants and tutors: ideation and feedback, writing, background research, data analysis, coding, and mathematical derivations. I provide general instructions and demonstrate specific examples of how to take advantage of each of these, classifying the LLM capabilities from experimental to highly useful. I argue that economists can reap significant productivity gains by taking advantage of generative AI to automate micro tasks. Moreover, these gains will grow as the performance of AI systems across all of these domains will continue to improve. I also speculate on the longer-term implications of AI-powered cognitive automation for economic research. The online resources associated with this paper offer instructions for how to get started and will provide regular updates on the latest capabilities of generative AI that are useful for economists.

Here is the home page for Korinek.  Here is related applied work from Benjamin Manning.  Economic research methods are changing right before our eyes, and most of the profession is asleep on this one.

Michael C. Jensen, RIP

Not only was he a major figure in both financial economics and industrial organization, but he did things too:

First, early on Mike decided that the Journal of Finance needed competition to drag it into the era of scientific research. Despite a chockful personal research agenda, Mike started the Journal of Financial Economics that he edited for 20+ years. After its 1974 debut, the JFE quickly became the top journal in finance, and it had the desired effect of upping the game of the JF.

Second, Mike’s foresight was unmatched. With the arrival of the internet, he predicted it would become the conduit for the distribution of new research. He launched SSRN (Social Science Research Network), supported it financially, and guided it for many years, never doubting it would succeed. His unfailing faith was eventually vindicated.

Here is the full tribute from Gene Fama.


The LLMs basically win:

This paper presents a novel study on harnessing Large Language Models’ (LLMs) outstanding knowledge and reasoning abilities for explainable financial time series forecasting. The application of machine learning models to financial time series comes with several challenges, including the difficulty in cross-sequence reasoning and inference, the hurdle of incorporating multi-modal signals from historical news, financial knowledge graphs, etc., and the issue of interpreting and explaining the model results. In this paper, we focus on NASDAQ-100 stocks, making use of publicly accessible historical stock price data, company metadata, and historical economic/financial news. We conduct experiments to illustrate the potential of LLMs in offering a unified solution to the aforementioned challenges. Our experiments include trying zero-shot/fewshot inference with GPT-4 and instruction-based fine-tuning with a public LLM model Open LLaMA. We demonstrate our approach outperforms a few baselines, including the widely applied classic ARMA-GARCH model and a gradient-boosting tree model. Through the performance comparison results and a few examples, we find LLMs can make a well-thought decision by reasoning over information from both textual news and price time series and extracting insights, leveraging cross-sequence information, and utilizing the inherent knowledge embedded within the LLM. Additionally, we show that a publicly available LLM such as Open-LLaMA, after fine-tuning, can comprehend the instruction to generate explainable forecasts and achieve reasonable performance, albeit relatively inferior in comparison to GPT-4.

This kind of work is in its infancy of course.  Nonetheless these are intriguing results, here is the paper.  Via an MR reader.

Why aren’t Canada and other Anglo nations turning against immigration more?

That is the topic of my latest Bloomberg column.  I cover several points, here is one of them, based on the economic idea of intertemporal substitution:

In this sense, Canada is ahead of much of the rest of the world in seeing the importance of these factors and turning it into actionable policy. It is willing to give up some of its present cultural identity to achieve a brighter cultural and political future.

This trade-off is much better than it looks at first. For one thing, birth rates for native-born citizens may fall further than they have already. If a country wants to preserve its national culture, it may be better off allowing more migration now, when there is still a critical mass of native-born citizens to ease assimilation.

To put the point more generally: Whatever costs there might be to immigration, successful nations will have to deal with them sooner or later. And the sooner they do, the better off they will be. The choice is not so much between more immigration and less immigration, but rather a lot of immigration now or a lot later. This choice will become all the more pressing as the need to fund national retirement programs requires more tax-paying citizens.

And on real estate prices:

One of the most common criticisms of immigrants is that they push up real estate prices. Yet there is a home-grown explanation: Stringent regulations on building make it difficult for the supply of housing to respond when demand increases.

In fact, there is a way immigration can help address this problem. First, immigrants may themselves induce their adopted country to free up its real estate markets. So immigration might increase real estate costs in the short run, but help reduce them in the longer run. Second, immigrants can help lower-tier cities move to the fore. The suburbs of Toronto, for example, have seen much of their growth driven by Asian in-migration, and longer term that will give Canadians more residential (and commercial) options.

These points aside, note that higher real estate prices, to the extent they result from immigrant demands, largely translate into capital gains for homeowners — most of whom are native-born. To be sure, the higher home prices may be bad for many younger Canadians, who may be locked out of housing markets, but eventually many of them will inherit high-valued homes from their parents.


The law and economics of permitting

Here is an important new paper by Zachary D. Liscow:

Given the benefits to economic growth and the need to transition to green energy, getting infrastructure built is an urgent issue. I describe what to consider in designing a system of permitting infrastructure. I then review the evidence: in the US, permitting is slow, infrastructure is expensive, and environmental outcomes are not particularly good. I propose a framework for reform with two dimensions: the power of the executive branch to decide and its capacity to plan. After considering reform possibilities, I propose that reforming both dimensions could lead to a possible “green bargain” that benefits efficiency, the environment, and democracy.

Via Heidi Williams.

Incentives matter

(To be clear, I do take the Havana Syndrome allegations seriously, but the broader equilibrium is worth pondering too.)

New Econ Journal Watch edition

Volume 21, Issue 1, March 2024

In Memoriam (.pdf)

In this issue:

Executive diversity and firm performance: Beginning in 2015, McKinsey & Company has released a series of highly impactful studies claiming a positive relationship between executive racial/ethnic diversity and firm performance. Jeremiah Green and John Hand explain why the McKinsey findings cannot be verified, and they do a replication of sorts using the S&P 500, and find, instead, no relationship between executive racial/ethnic diversity and performance. (Note: The McKinsey authors were not invited to reply for concurrent publication because this piece was finalized at too late a date. They are invited to reply in a future issue.)

Temperature~economic growth: Having tested temperature-growth claims previously in this journal (herehere, and here), David Barker now reports on his investigation into a much-cited 2015 Nature article by Marshall Burke, Solomon Hsiang, and Edward Miguel. Once again, Barker contends that the claims in the commented-on article are untenable. (Note: Professors Burke, Hsiang, and Miguel were not invited to reply for concurrent publication because this piece was finalized at too late a date. They are invited to reply in a future issue.)

The limbo bar of 5% allows too much to wiggle under itTom Engsted takes another look at the replication crises and argues that we need to lower the limbo bar—that is, raise the difficulty of claiming ‘statistical significance.’

Revisiting Hypothesis Testing with the Sharpe Ratio: Michael O’Connor contends that comparing Sharpe ratios of different investment options is not as simple as has been presented in the academic finance literature. He cautions that no method of analysis can improve the power of the test of statistical significance because the power is an innate property of the statistic; he uses simulations and other analyses to show that “when the power is low then the very best estimators perform no better than random number generators,” and advises: “Investors should be wary of claims by portfolio managers that their Sharpe ratio exceeds the ratio of other managers.”

What caused the Ukraine famine of the early 1930s? In the previous issue, Mark Tauger discussed the work of Natalya Naumenko, and Naumenko replied. Here, Tauger rejoins.

Ergodicity economics: Previously, Matthew Ford and John Kay commented on ergodicity economics, and a reply was provided by Oliver Hulme, Arne Vanhoyweghen, Colm Connaughton, Ole Peters, Simon Steinkamp, Alexander Adamou, Dominik Baumann, Vincent Ginis, Bert Verbruggen, James Price, and Benjamin Skjold. Here, Ford and Kay rejoin, contending “our criticism of ergodicity economics remains unanswered.”

Classical Liberalism in Russia: Provided here is an intellectual and political history of classical liberalism in Russia. The author is Paul Robinson, who has published the books Russian Conservatism and Russian Liberalism. In the story, notable figures include Semyon Desnitsky, Alexander Kunitsyn, Konstantin Kavelin, Boris Chicherin, and Boris Brutzkus. (An 1857 essay by Chicherin appears in the previous issue of this journal.)

Classical Liberal Think Tanks in Greece, 1974–2024: Constantinos Saravakos, Georgios Archontas, and Chris Loukas provide a guide to the course of liberal thought and movements in Greece. After some foregrounding, they pick up the story in 1974 and focus especially on the entry, exit, character, activity, and influence of liberal think tanks in Greece. The authors are affiliated with one of them, the Center for Liberal Studies (KEFiM).

Trygve Hoff’s Appeal to Ragnar Frisch: The liberal economist Trygve Hoff appealed to Ragnar Frisch, a fellow Norwegian and future Nobel Prize winner, to relent in his economic interventionism. Their four letters from 1941 are translated and presented by Hannes Gissurarson.

Christianity Changes the Conditions of Government: Three brief chapters of The Ancient City by Fustel de Coulanges (1830–1889) capture some the book’s important ideas about the composition of ancient polytheism, and how the universal benevolence of Christianity’s monotheistic gospel would in time spell a new world. “[T]o obey Cæsar is no longer the same thing as to obey God.” The book, originally in French, was first published in 1864.

EJW thanks its referees and others who contribute to its mission.

Build Back Key Bridge Better

The collapse of the Key Bridge is a national disaster but also an opportunity for societal advancement. We must rebuild but in doing so we must also address the historical discrimination faced by workers in Baltimore and beyond. Ensuring the participation of Baltimore’s workforce in the reconstruction project is essential. It’s Baltimore’s bridge and in rebuilding we must actively engage and employ a diverse pool of local talent, reflecting the city’s rich cultural tapestry. We can Build Back Better by providing meaningful, well-paying jobs to those who have been historically marginalized, fostering economic growth and equity within the community.

Furthermore, offering accessible, quality day care for workers will directly contribute to an equitable working environment, enabling parents and guardians to participate fully in the reconstruction effort without the burden of child care concerns. We must reject the idea that equity and productivity are at odds. A more inclusive workforce is a more productive workforce.

American workers are the most productive in the world thus to Build American we must Buy American. Reconstruction of the Key Bridge is not just a matter of national pride but also an essential strategy for growing our economy. By prioritizing American materials and labor, we invest in our communities, support local industries, and ensure that the economic benefits of the reconstruction project are felt widely, especially in areas hardest hit by economic challenges.

We can build back better. We must build back better. By engaging Baltimore workers in Baltimore’s bridge we can rectify long-standing discrimination. By providing accessible child care, and adhering to “Buy American” rules we can build America as we build America’s bridge. Building back better is not simply about building physical infrastructure. It’s about building a bridge to the future. A bridge of progress, equality, and unity, symbolizing our collective commitment to a future where every individual has the opportunity to thrive.

Addendum: April 1, 2024.