Category: Science
Ideas Behind Their Time: Part Two
In 2010 I wrote about Ideas Behind Their Time:
We are all familiar with ideas said to be ahead of their time, Babbage’s analytical engine and da Vinci’s helicopter are classic examples. We are also familiar with ideas “of their time,” ideas that were “in the air” and thus were often simultaneously discovered such as the telephone, calculus, evolution, and color photography. What is less commented on is the third possibility, ideas that could have been discovered much earlier but which were not, ideas behind their time.
I gave experimental economics, random clinical trials and view morphing (“bullet time”) as examples. Jason Crawford has a list discussing the wheel, the steam engine and bicycles among other possibilities. In some cases, further exploration indicates that an idea required precursors and so was not as behind its times as first suspected, in rare cases, however, good ideas really could have been invented much earlier.
Using Claude, Brian Potter has significantly expanded the list by looking systematically across a wide range of inventions and asking could they have been invented earlier? Most could not. Put the other way, most useful technologies tend to be invented quite quickly once they are possible–this is reassuring. The airplane, for example, could not have been invented before a high power-to-weight engine, which happened circa 1880 making the late 1880s the earliest feasible date for powered flight. Thus, the Wright Brothers (1903) were only just behind the earliest feasible date–and that is true for many inventions.
The ideas very far behind their time include the stethoscope, general anesthesia and reinforced concrete and quite far behind are the Jacquard loom and canning. Is there a pattern here?
Addendum: Brian’s Github with the full prompt and output for each invention is here.
Using agents to build economic datasets
Constructing datasets from primary sources is one of the costliest tasks in empirical economics. We propose Deep Research on a Loop (DRIL), a methodology that uses AI agents to assemble datasets from publicly available sources. DRIL applies a fixed research instrument across a mapped unit space (e.g., countries by years), with a two-stage architecture separating design from implementation. The instrument specifies variables and coding rules, an evidence policy governs sources and citations, and data quality mechanisms track gaps and uncertainty explicitly. We exercise DRIL on a 2025 update of the Global Tax Expenditures Database for eight Latin American and Caribbean countries. The run produces 129 sources and 136 evidence records, covering 22 qualitative fields fully and 6 quantitative estimate types with documented gaps, at the cost of a standard LLM subscription comparable to a few hours of research-assistant work. We argue that even partial automation of dataset construction can shift the production function of empirical economics.
That is from a new NBER working paper by
Another use of AI in research (from my email)
“Another thing we (John [Horton] and I) have thought about is having a swarm of AIs “fight” over a literature. They could take the cumulative datasets available and continuously argue until they understand the question. One line of thinking says they reach a stalemate (as scientists currently do). But we think not. More likely, they push evidentiary understanding to the limit and coalesce around what’s most probable — if not definitive!”
That is from Benjamin Manning.
Will AI kill the research paper?
Imagine taking a macroeconomics paper and adding a little button at the end “Press this button to update this paper with the latest macro data.”
All of a sudden you have multiple papers rather than one, and no single canonical version. It is the latter versions, not created directly by the authors, that people will look at.
Imagine adding another button, to either micro or macro papers “Please rerun these results using what the AI thinks might be five other different yet still plausible specifications.”
Then you have more papers yet.
Ultimately, why not just build a “meta-paper,” using AI, to answer any possible question about the subject area under consideration. This meta-paper would allow the reader, using AI, to make many sorts of modifications and additions to the basic work. The meta-paper also would allow the reader to add new data, to run additional robustness checks, and to do whatever else you might think of. Once again, the canonical version of the paper evolves away.
A researcher might spend a significant part of his or her career building such a meta-paper. Imagine a meta-paper, or sometimes I call it a “box,” devoted to answering questions about say fiscal policy, minimum wage hikes, or maybe the Industrial Revolution. Fed researchers would spend their entire careers, not writing papers, but improving the Fed’s “box” that answers questions about monetary policy and also prudential supervision.
Who will be good at doing such things? Is it the people today who become the top economists, or not? Will it be a highly decentralized endeavor, or, given the compute and team work requirements, a highly centralized one?
Economics is going to change a lot, as will many of the other sciences.
It is funny, and tragic, how much some of you are still obsessed with writing and publishing “papers.”
Which are the most common everyday phenomena that we don’t properly understand?
Off the top of my head:
• Lightning (how does it happen?)
• Sleep; dreams (why do they exist?)
• Glass (thermodynamics of formation)
• Turbulence (when does it start?)
• Morphogenesis (how does a creature know what should go where?)
• Rain (it seems to start faster than models would predict)
• Ice (dynamics of slipperiness)
• Static electricity (which material will donate electrons?)
• General anaesthetic. (And the mechanism of a lot of drugs, e.g. paracetamol.)
That is from Patrick Collison. It is a further interesting question how many of those questions will be answered by what is sometimes called AGI. Perhaps none of them? In at least some of those cases, what is scarce is experimental data, not reasoning per se.
The UAP report so far
I will stick with my earlier Free Press predictions:
The fact remains that, if you talk with insiders, they will confirm that the federal government faces some big mysteries. It seems that we have data on what appear to be craft that move very fast, have no visible means of propulsion, and can accelerate in a surprising manner. Radar, infrared, and other forms of data are cited to varying degrees, plus there are eyewitness pilot reports, broadly consistent with what our instruments are telling us.
And this:
Assuming a reasonable chunk of the data are declassified, I think we will simply see more of the same kind of material we’ve seen in the past: more data on entities that appear to move very quickly and in mysterious ways, but with no real explanations. We will see, as I’ve argued before, that the government itself does not know what is going on, and has been afraid to admit that. That may be the real “conspiracy” and why the veil of secrecy has been relatively difficult to pierce.
As of yesterday, there are plenty of additional videos of what seem to be glowing orbs moving fast and in unpredictable ways. Or try this one. Here is another weird one. Or try this. And another one, near military craft. And what is this?
One thing we can conclude is that the debunkers, who have been suggesting this is all camera tricks, parallax issues, or people not understanding how videos work, are proven wrong in general, even though they are right about some particular cases. On that point we can move on, as I have been arguing for some while. Mick West is not your proper guide here.
Nonetheless we still do not know what it all means, and I do not see proof of anything in particular.
I also will stress my earlier point that we are not going to see alien bodies or alien technologies, or anything meaningful connected to Roswell. That is sheer fantasy, or sometimes locos.
340 million hits in the first twelve hours? More people will be believing in aliens in any case, I suspect. Or will it be demons?
It is fashionable in the comments sections of blogs to call this topic a waste of time, but the serious people in the military and national security — most of whom do not cite alien presence — do not see it that way.
And they will be releasing more materials. These materials are being released because some subsection of “the Deep State” wants to know what is going on. As do I.
William Stanley Jevons as polymath
In the 1860s Jevons built a Logical Abacus, sometimes called a logical piano, a kind of early computer that could perform (some kinds of) logical operations faster than humans could. It is held in the Museum of the History of Science at Oxford University, and you can think of its structure and operation as broadly akin to a player piano in music. It was limited in its powers, and geared mainly toward replicating Boolean logic, but extreme in its ultimate ambitions. Jevons understood the potential. In his written presentation of the project, Jevons cites the work of Charles Babbage, and noted that “material machinery is capable, in theory at least, of rivalling the labours of the most practiced mathematicians in all branches of their science. Mind thus seems able to impress some of its highest attributes upon matter, and to create its own rival in the wheels and levers of an insensible machine.” Jevons understood that science would be able to tackle some of the most difficult projects, and he wanted to be on as many of those frontiers as possible. He understood that his own work was a mere beginning, and he wanted to press forward as much as possible.25See Jevons (1870, the quotation from p.498), and also Maas (2005, chapter six). For a general background on Boolean logic, see Hailperin (1986).
Jevons also studied molecular motion in liquids and developed the concept of “pedesis,” a precursor of what we now call Brownian motion. That said, Jevons thought his pedesis was an electrical phenomenon related to osmosis, and so he turned out to be incorrect in his fundamental hypotheses. Nonetheless, this topic, like the others, showed he was an observant mind and obsessed with developing theories to explain anything and everything. He wasn’t just a pedant, rather he made real contribution to a number of scientific fields above and beyond economics.26On Jevons on pedesis and Brownian motion, see Brush (1968).
Jevons also was a “born collector” in the words of Keynes, and an extreme bibliomaniac. He accumulated thousands of books, and he lined the walls of his house and attic with them, and also stored them in piles in the attic, which became a problem for his wife upon his passing.
That is from my recent generative book The Marginal Revolution: Rise and Decline, and the Pending AI Revolution.
Growth is getting harder to find, not ideas
Relatively flat US output growth versus rising numbers of US researchers is often interpreted as evidence that ideas are getting harder to find. We build a new 45-year panel tracking the universe of US firms’ patenting to investigate the micro underpinnings of this claim, separately examining the relationships between research inputs and ideas (patents) versus ideas and growth. We find that average patents per R&D input are increasing, the elasticity of patents to R&D inputs is flat or rising, and there is no systematic evidence of a secular decline in patenting after controlling for research inputs. We then document a positive, significant, and fairly steady relationship between firms’ growth in ideas (patents) and labor productivity. Average firm growth after controlling for idea growth, however, declines. Together, these results suggest that innovative efforts play a key role in sustaining growth that has not diminished over the last four decades.
Here is the paper by Teresa C. Fort, Nathan Goldschlag, Jack Liang, Peter K. Schott, and Nikolas Zolas.
Thomas Gresham is underrated
While northern professions in 1600 did not require lengthy training in mathematics or science, there was popular interest in these topics. England’s first chair in mathematics was endowed by Thomas Gresham,61 who had founded London’s Royal Exchange and pledged the rents from that institution to fund seven professorships, who would not train student but would rather give two public lectures (in Latin and English) each week. As Gresham also gave chairs in astronomy and “physik,” this produced a cluster of scientifically minded individuals, who would later play an outsized role in the founding of the Royal Society. Robert Hooke was the Gresham Professor of Geometry, William Petty the Gresham Professor of Music, and Christopher Wren the Gresham Professor of Astronomy.
Perhaps because of Gresham’s public lectures, interest in mathematics grew. More professorships followed, including the mid-17th century Lucasian Chair in Mathematics (after William Lucas, member of parliament for Cambridge), for which Isaac Newton would be the second occupant (Clark, 1904). The popular interest in science also meant that teachers at urban universities could fill public lecture halls by teaching about chemistry, and even performing public chemistry experiments.
That is from a new NBER working paper by David M. Cutler and Edward L. Glaeser, “How Have Universities Survived for Nearly a Millennium?” Has any single individual funded three equally prestigious chairs or anything close to that?
A Comparison of Agentic AI Systems and Human Economists
This paper compares agentic AI systems and human economists performing the same causal inference tasks. AI systems and humans generally obtain similar median causal effect estimates. While there is substantial dispersion of estimates across model instances, the human distributions of estimates have wider tails. Using AI models as reviewers to compare and rank “submissions,” the following ranking emerges regardless of reviewer model: (1) Codex GPT-5.4, (2) Codex GPT-5.3-Codex, (3) Claude Code Opus 4.6, and (4) Human Researchers. These findings suggest that agentic AI systems will allow us to scale empirical research in economics.
I enjoy the name of the author, namely Serafin Grundl. Here is the paper, via Ethan Mollick. You could interpret these results as showing the AIs have fewer hallucinations. And just to reiterate a key point from the paper:
The second part of this paper is an AI review tournament in which “submissions” (codes and write-ups) from humans and the AI models are compared and ranked against each other. The reviewers are the following AI models: Gemini 3.1 Pro Preview, Opus 4.6 and GPT-5.4. For each review the reviewer is asked to write a report comparing four submissions (human, Opus 4.6, GPT-5.3-Codex, GPT-5.4). Each reviewer model writes comparison reports for the same 300 comparison groups. The average rankings are strikingly similar across reviewer models: (1) Codex GPT-5.4, (2) Codex GPT-5.3-Codex, (3) Claude Code Opus 4.6, and 2(4) Human Researchers.
Who comes in last? Hi people!
New Emergent Ventures tranche on science policy and communication
American science policy is now more important than at perhaps any previous point in history—how science is organized and funded (or not funded) in this country continues to rise in significance.
I have also spoken about the undersupply of people who understand this and are trying to act on it in Washington. Unfortunately the career paths here are neither well-defined nor well-regarded. I would like to help change that.
What we’re looking for:
- Priority 1: Metascience Policy Entrepreneurs in DC
- Funding for individuals working at the intersection of science policy and institutional reform—people who can shape how Congress and federal agencies think about science funding and governance.
- Priority 2: Science and Metascience Communicators
- Funding for communicators via any medium—bloggers, journalists, authors, podcasters, artists, filmmakers, conveners, influencers, event organizers—who can expand the reach of pro-science ideas beyond their current audience.
We are doing this with and thank Renaissance Philanthropy for the support. You can apply through the regular Emergent Ventures portal.
AI Risks
Two new papers/initiatives indicate severe risks from AI, interestingly in opposite directions. The first is that the most advanced frontier models are now capable of finding and exploiting software in ways that could be used to crash or control pretty much all the world’s major systems.
Anthropic: We formed Project Glasswing because of capabilities we’ve observed in a new frontier model trained by Anthropic that we believe could reshape cybersecurity. Claude Mythos2 Preview is a general-purpose, unreleased frontier model that reveals a stark fact: AI models have reached a level of coding capability where they can surpass all but the most skilled humans at finding and exploiting software vulnerabilities.
Mythos Preview has already found thousands of high-severity vulnerabilities, including some in every major operating system and web browser. Given the rate of AI progress, it will not be long before such capabilities proliferate, potentially beyond actors who are committed to deploying them safely. The fallout—for economies, public safety, and national security—could be severe. Project Glasswing is an urgent attempt to put these capabilities to work for defensive purposes.
That’s from Anthropic. The irony is that the company that has developed a frontier model capable of infiltrating and undermining more or less any computer system in the world is the one that has been forbidden from working with the US government. It’s as if a private firm developed nuclear weapons and the American government refused to work with them because they were too woke. Okey dokey.
The second paper on AI risks is AI Agent Traps from Google DeepMind. They point out that AI agents on the web are vulnerable to all kinds of attacks from things like text in html never read by humans, hidden commands in pdfs, commands encoded in the pixels of images using steganography and so forth.
Putting this together we have the worrying combination that very powerful AI’s are very vulnerable. Will AI solve the problems of AI? Eventually the software will be made secure but weird things happen in arms races and its going to be a bump ride.
Andy Hall advice on AI and economic research
Here is the document, excerpt:
In January, I released the results of an experiment showing how Claude Code could helpfully extend old papers “automagically.” It was pretty astonishing to me. Claude was able to come up with a plan, scrape the web, write code, run regressions, create tables and figures, and write a whole memo on what it had found—all in about 45 minutes.
Are AI tools perfect? No. Claude made some interesting mistakes in that extension, and since then, I’ve seen it make a whole bunch more. Are human researchers perfect, though? Hell no.
The evidence that AI tools should now be an essential part of your toolkit is overwhelming—look at the recent work that my Stanford colleague Yiqing Xu has put out, for example, which allows for the automated verification of empirical research. This is so clearly valuable. When it comes to empirical work, we’re never going back to the pre-AI world.
Here is a thread on the paper, heedworthy throughout. If you do not have some kind of decent plan here, other economists will leave you in the dust. Even if it is only a minority of “other economists” their total leverage and impact will be extreme.
Advice for economics graduate students (and faculty?) vis-a-vis AI
From Isiah Andrews, via Emily Oster and the excellent Samir Varma. A good piece, though I think it needs to more explicitly consider the most likely case, namely that the models are better at all intellectual tasks, including “taste,” or whatever else might be knockin’ around in your noggin…I am still seeing massive copium. But the models still are not able to “operate in the actual world as a being.” Those are the complementarities you need to be looking for, namely how you as a physical entity can enhance the superpowers of your model, or should I express that the other way around? That might include gathering data in the field, persuading a politician, or raising money. I am sure you can think of examples on your own.
NSF update
The White House seeks to slash the NSF budget by nearly 55%, to $4 billion. The proposal also cuts all funding for the NSF division that funds research on the social sciences and economics. At an internal all-hands meeting on Friday, NSF leaders announced that they would dissolve the agency’s Social, Behavioral and Economic Sciences directorate based on the budget request, according to two NSF staff members who shared information anonymously in order to speak freely.
Here is the full story.