Causal claims in economics
From a new and very interesting web site on that topic:
-
Significant Increase in Causal Claims: The average proportion of causal claims in papers rose significantly from approximately 5% in 1990 to around 28% in 2020, reflecting the impact of the credibility revolution in economics.
-
Growth in Causal Inference Methods and decline in Theoretical and Simulation Methods…
-
Intricate Causal Narratives Enhance Publication and Citation Impact: Papers featuring intricate and interconnected causal narratives are more likely to be published in top-tier journals, particularly the top 5 journals, and receive more citations, especially within those journals.
-
Key Measures of Causal Narrative Complexity: Increases in the number of unique paths and the longest path length in causal knowledge graphs are positively associated with both publication in leading economics journals and higher citation counts. This highlights the value placed on depth and complexity in causal narratives.
-
Depth Over Quantity in Causal Claims: While the overall number of claims made is positively correlated with top journal publications, the number of causal edges alone does not show the same positive association with publication outcomes or citation counts. This suggests that depth over breadth in causal claims is valued...
-
Novel Causal Relationships Enhance Publication but Not Citation Impact: Papers introducing novel causal relationships that have not been previously documented are more likely to be published in top 5 journals, indicating a premium on originality for publication success. However, this does not necessarily translate into higher citation counts once published.
Here is an associated tweet storm,
Obviously it can be argued either way, but I see these results as more negative for the causality revolution than positive? It seems there is too much emphasis on generating a defensible results from a hitherto unused data set. I understand the signaling value here, but the social value is not always obvious. I am struck by how often I meet economics graduate students who can reason about programming more effectively than they can reason about the real world.