Category: Science
New Emergent Ventures tranche on science policy and communication
American science policy is now more important than at perhaps any previous point in history—how science is organized and funded (or not funded) in this country continues to rise in significance.
I have also spoken about the undersupply of people who understand this and are trying to act on it in Washington. Unfortunately the career paths here are neither well-defined nor well-regarded. I would like to help change that.
What we’re looking for:
- Priority 1: Metascience Policy Entrepreneurs in DC
- Funding for individuals working at the intersection of science policy and institutional reform—people who can shape how Congress and federal agencies think about science funding and governance.
- Priority 2: Science and Metascience Communicators
- Funding for communicators via any medium—bloggers, journalists, authors, podcasters, artists, filmmakers, conveners, influencers, event organizers—who can expand the reach of pro-science ideas beyond their current audience.
We are doing this with and thank Renaissance Philanthropy for the support. You can apply through the regular Emergent Ventures portal.
AI Risks
Two new papers/initiatives indicate severe risks from AI, interestingly in opposite directions. The first is that the most advanced frontier models are now capable of finding and exploiting software in ways that could be used to crash or control pretty much all the world’s major systems.
Anthropic: We formed Project Glasswing because of capabilities we’ve observed in a new frontier model trained by Anthropic that we believe could reshape cybersecurity. Claude Mythos2 Preview is a general-purpose, unreleased frontier model that reveals a stark fact: AI models have reached a level of coding capability where they can surpass all but the most skilled humans at finding and exploiting software vulnerabilities.
Mythos Preview has already found thousands of high-severity vulnerabilities, including some in every major operating system and web browser. Given the rate of AI progress, it will not be long before such capabilities proliferate, potentially beyond actors who are committed to deploying them safely. The fallout—for economies, public safety, and national security—could be severe. Project Glasswing is an urgent attempt to put these capabilities to work for defensive purposes.
That’s from Anthropic. The irony is that the company that has developed a frontier model capable of infiltrating and undermining more or less any computer system in the world is the one that has been forbidden from working with the US government. It’s as if a private firm developed nuclear weapons and the American government refused to work with them because they were too woke. Okey dokey.
The second paper on AI risks is AI Agent Traps from Google DeepMind. They point out that AI agents on the web are vulnerable to all kinds of attacks from things like text in html never read by humans, hidden commands in pdfs, commands encoded in the pixels of images using steganography and so forth.
Putting this together we have the worrying combination that very powerful AI’s are very vulnerable. Will AI solve the problems of AI? Eventually the software will be made secure but weird things happen in arms races and its going to be a bump ride.
Andy Hall advice on AI and economic research
Here is the document, excerpt:
In January, I released the results of an experiment showing how Claude Code could helpfully extend old papers “automagically.” It was pretty astonishing to me. Claude was able to come up with a plan, scrape the web, write code, run regressions, create tables and figures, and write a whole memo on what it had found—all in about 45 minutes.
Are AI tools perfect? No. Claude made some interesting mistakes in that extension, and since then, I’ve seen it make a whole bunch more. Are human researchers perfect, though? Hell no.
The evidence that AI tools should now be an essential part of your toolkit is overwhelming—look at the recent work that my Stanford colleague Yiqing Xu has put out, for example, which allows for the automated verification of empirical research. This is so clearly valuable. When it comes to empirical work, we’re never going back to the pre-AI world.
Here is a thread on the paper, heedworthy throughout. If you do not have some kind of decent plan here, other economists will leave you in the dust. Even if it is only a minority of “other economists” their total leverage and impact will be extreme.
Advice for economics graduate students (and faculty?) vis-a-vis AI
From Isiah Andrews, via Emily Oster and the excellent Samir Varma. A good piece, though I think it needs to more explicitly consider the most likely case, namely that the models are better at all intellectual tasks, including “taste,” or whatever else might be knockin’ around in your noggin…I am still seeing massive copium. But the models still are not able to “operate in the actual world as a being.” Those are the complementarities you need to be looking for, namely how you as a physical entity can enhance the superpowers of your model, or should I express that the other way around? That might include gathering data in the field, persuading a politician, or raising money. I am sure you can think of examples on your own.
NSF update
The White House seeks to slash the NSF budget by nearly 55%, to $4 billion. The proposal also cuts all funding for the NSF division that funds research on the social sciences and economics. At an internal all-hands meeting on Friday, NSF leaders announced that they would dissolve the agency’s Social, Behavioral and Economic Sciences directorate based on the budget request, according to two NSF staff members who shared information anonymously in order to speak freely.
Here is the full story.
A reminder (for academics)
Yes, there are skills AIs haven’t mastered. But if your skill still appears to be the exclusive province of humans, that might mean the major AI companies do not yet consider it very important to master right away. Eventually it will rise to the top of the list.
Here is more from my Free Press essay on AI. If not for the copied passage, it seems no one was noticing this book review? (NYT, read the emendation)
Sentences to ponder
This matters for the AI question, and the book leaves it unfinished. If the breakthroughs of the past required social conditions, not just cognitive capacity, then what does it mean when the next breakthroughs are produced by systems that have no social conditions at all? A neural net does not need a university chair or financial independence from the church. It does not need to reorganize its commitments. It does not, in any recognizable sense, have commitments. The machine that replaces the marginalist is not a better marginalist. It is a different kind of thing entirely.
That is from Jônadas Techio, presumably with LLMs, this review of The Marginal Revolution is interesting throughout. And this:
Maybe the book demonstrates only that Cowen personally remains good at something the field no longer needs.
Scott Sumner on *The Marginal Revolution*
My favorite part of Tyler’s book is where he asks a very good but non-obvious question: Why did it take so long for economics as a field to develop a coherent model or framework of analysis? Much of the book discusses how three economists simultaneously developed marginal analysis, with a focus on the work of Stanley Jevons. Here I’ll briefly provide the intuition of marginal analysis and then explain why economics is both extremely easy but also quite difficult…
Tyler does a great job explaining why Jevon’s model of marginal analysis (which underlies most of modern microeconomics) is elementary on one level, but also something that wasn’t discovered until the 1860s because it was not at all obvious. Here’s how he concludes Chapter 3:
[This is TC now] By studying the slow intellectual development of economics, and contrasting it with other fields of study, we can learn the following:
1. Some insights are very hard to grasp, even if they are apparently simple once they are understood. People need to “see around corners” in the right way to understand these insights and incorporate them into their world views.
2. Economics is one of those fields, and that is why it took intuitive economic reasoning so long to evolve, marginalism included. Those of us who are educators, or who spend time talking to policymakers, should take this point very seriously.
3. Even very, very smart people are likely unaware that these “see around the corner” insights are missing – did Euclid rue that he did not have access to proper supply and demand and tax incidence theory? Probably not.
4. Economics is not the only such field that is hard to grasp, some other examples being segments of botany, geology, and evolutionary biology.
5. Scientific revolutions come about when many complementary pieces are in place, such as financial support, intellectual independence, and networks of like-minded others to talk with.
Those conditions help people to understand that “seeing around those corners” can bring both high social and professional returns.
Are there major conceptual corners that today still no one can see around? If so, how might we discover what they are? And why are we not working harder on this? Or are we?
Here is the rest of Scott’s commentary. Here is the online book.
Henry Oliver calls it a Swiftian ending
To The Marginal Revolution: Rise and Decline, and the Pending AI Revolution, here is the very close of the book:
There is however a slightly scarier version of this story yet. Maybe our intuitions about the world, including the economic world, were never so strong in the first place. Maybe we put so much value on “intuitive” results, in 20th century microeconomics, as a kind of cope and also security blanket, to make up for this deficiency. But our intuitions, even assuming them to be largely correct, always were just a small corner of understanding, swimming in a larger froth of epistemic chaos. And now the illusion has been stripped bare, and the true complexities of economic reasoning are being revealed.
As Arnold Kling would say, “Have a nice day.”
Can I say again “Have a nice day”?
What is economics these days?
From The Marginal Revolution: Rise and Decline, and the Pending AI Revolution:
The day before drafting this paragraph, I blogged a paper on confidence gaps between men and women. It was a paper written by economists, published in the prestigious American Economic Review, the profession’s number one journal. Is this actually sociology, or personality or social psychology, or part of some gender studies field? No one in the economics profession cares to discuss that anymore. It is not that there is a dogmatic attachment to what used to be called “economic imperialism,” rather the view is that if the paper is good enough … it is good enough to publish. I also recently read a paper on using cell phone data to estimate how many people actually were attending church. Freakonomics guru Steve Levitt wrote and published well-known papers on the choice of baby names and corruption in Sumo wrestling116See Exley and Nielsen (2024), and on cell phones see Pope (2024)..
The dirty little secret is that what distinguishes economics as a field, right now, is a mix of higher standards, harder work, better math, and higher IQs. That is the real (dare I say marginal?) contribution of “empirical economics today,” not marginalism per se, though of course contemporary models typically are consistent with marginalist reasoning…
One modest sign of all these changes is how many advisors, when speaking to individuals considering economics graduate school, recommend math or even computer science as a possible background undergraduate major. While most are still undergraduate economics majors, if only because that is where their interest in economics came from, no one seems to mind if they are not. These days, a background in mathematics or computer science is at least as useful for the graduate work to come. Once you get to graduate school, you will have to learn plenty of math and programming anyway, so why not start off in those fields? The prevailing attitude is that the economics you can figure out along the way, or for some topics you may not need to know much of it at all. How complicated are all those economic principles anyway? General skills of apprenticeship and plain ol’ hard work are growing in importance too, as top graduate programs increasingly want their incoming students to have done a “predoc” with an accomplished researcher somewhere along the way.
That is from the chapter on the future of economics in a world with advanced AI.
Addendum: On The Marginal Revolution book, I would most of all like to thank Jeff Holmes for the great job he did on the project, all of the actual work (other than the writing) is from him. He is also producer of CWT, I owe much to him!
*The Marginal Revolution: Rise and Decline, and the Pending AI Revolution*
I am offering a new piece of work — I do not quite call it a book — online and free. It has four chapters, is about 40,000 words, is fully written by me (not a word from the AIs), and it is attached to an AI with a dual page display, in this case Claude. Think of it as a non-fiction novella of sorts, you can access it here. You can read it on the screen, turn it into a pdf (and upload into your own AI), send it to your Kindle, or discuss it with Claude.
Here is the Table of Contents:
1. What Is Marginalism?
2. William Stanley Jevons, Builder and Destroyer of Marginalism
3. Why Did It Take So Long for the Science of Economics to Develop?
4. Why Marginalism Will Dwindle, and What Will Replace It?
Here are the first few paragraphs of the work:
How is it that ideas, and human capabilities, become lost? And how is that new insights come to pass? If eventually the insight seems obvious, why didn’t we see it before? Or maybe we did see it before, but didn’t really know we were on to something important? Why do new insights arrive suddenly, in a kind of flood? How do new worldviews replace older ones?
And what does all of that have to do with the future of science, the future of research, and the future of economics in particular? Especially when we try to understand how the ongoing artificial intelligence revolution is going to reshape human knowledge, and the all-important question of what economists should do.
Those are the motivating questions behind this work, but I will address them in what is initially an indirect fashion. I will start by considering a case study, namely the most important revolution in economics, the Marginal Revolution (to be defined shortly). The Marginal Revolution made modern economics possible. What was the Marginal Revolution? How did it start? Why did it take so very long to come to fruition? From those investigations we will get a sense of how economic ideas, and sometimes ideas more generally, develop. And that in turn will help us see where the science, art, and practice of economics is headed today.
Recommended! I will be covering it more soon.
The rise of China as a global innovator in pharma (incentives matter)
This paper examines China’s transition from pharmaceutical “free rider” to global innovator over the last decade. In 2010, China accounted for less than 8% of global clinical trials; by 2020, it had surpassed the US in annual registered clinical trial volume. To study this transformation, we compile a comprehensive, synchronized database spanning the pharmaceutical drug development supply chain, covering scientific publications, clinical trials, drug development milestones for China, the U.S., and Europe, alongside drug sales and government policies over the same period. We provide strong evidence that China’s rise was primarily driven by the National Reimbursement Drug List (NRDL) reform, which dramatically expanded the effective market size for innovative drugs. We document a sharp rise in both the quantity (86% increase) and novelty of drug trials post reform, with growth concentrated in reform-exposed disease categories, first- or best-in-class drugs, and among domestic firms. A decomposition exercise reveals that the NRDL reform accounts for 43% of the growth in oncology trial activity, nearly doubling the combined contribution of upstream knowledge accumulation and talent flows (24%), while other government policies play a minor role. Finally, dynamic gains from induced innovation exceed the reform’s static gains in consumer access to innovative drugs by threefold, underscoring the importance of accounting for the reform’s long-run effects on innovation incentives in addition to near-term improvements in drug affordability.
That is from a new NBER working paper by
Dwarkesh chats with Terence Tao
Is AI currently helping economic research?
The third possibility, that AI helps to weed out mistakes, is trickier for the discipline. This stage could become even more important if journals do start to be hit by a wave of AI-generated slop — or, perhaps more likely, good papers with so many appendices and robustness checks that even the most dedicated referee is defeated. (The real “Dr Robust” does not have infinite energy.)
Eager to embrace the new technology, several of the top five economics journals are already experimenting with Refine, an impressive AI-powered reviewing tool that scours economics papers for errors. Ben Golub, one of its creators, shared that even with papers that had been through referees at top journals, Refine was picking up problems in at least a third of cases.
Here is more from Soumaya Keynes at the FT.
Robert Trivers, RIP
The greats have been falling…