Category: Web/Tech

How does a start-up founder hire a software engineer or a tech worker find a date in 2025? They could share a job posting on LinkedIn, log on to a dating app — or leave a handwritten note in a Waymo.

That’s what Influur CEO Alessandra Angelini was thinking when she and Jennifer Lo Chan, Influur’s head of marketing, tucked a piece of paper into a self-driving taxi’s center console on a recent trip to San Francisco.

There is an enormous and growing discussion on AI alignment, but very little on capitalizing AIs, and what effects that might have.  By capitalizing AIs, I mean simply requiring them to hold some wealth, in whichever form they might care about, so they have proverbial “skin in the game” (can we still call it that?).

Consider bank regulation.  Commonly it is recognized that regulators cannot control all bank actions, especially in the presence of off-balance sheet risk.  So in addition to some explicit regulation, most countries require their banks to hold a considerable amount of capital.  That gives the shareholders their own selfish incentive to rein in excess risk-taking.  Few if any think this approach is foolproof, but overall it is in the ascendancy and arguably higher capital requirements have been the most useful part of Dodd-Frank here in the U.S.

But can capitalization work as a means to limit AI risk?  What does that even mean?  Imagine some set of AIs that are either fully independent and unowned, or their owners are busy and de facto the AIs make financial (and other) decisions on their own.

Here is one set of possiblities:

1. Each of some subgroup of AIs has a legal identity and a level of wealth.

2. Each of those AIs has the equivalent of a utility function, thus giving it goals.  This may be “put” into the AI, or perhaps it evolves.  Note that computational complexity prevents the utility function from automatically aligning the AI in all circumstances.

3. AIs thus will behave more conservatively, not wanting to lose their wealth, as that wealth can help them achieve their goals.

4. An AI-based legal system could sue wrongdoers, and take awards from those found guilty of bad behavior, as defined by the AI legal code.  That would further discourage bad behavior.  But of course for the suing threat to be meaningful, the AIs have to hold some wealth in the first place.

The end result would be risk-averse AIs, taking care not to lose the wealth they have accumulated.  They won’t just start a bank and then take all of the deposits to Vegas.  That is not exactly full alignment, but it induces better behavior, just as capital requirements do with human-run banks.

Of course a number of things could go wrong with capitalization, just as can happen with humans, for instance:

5. The capitalization might serve as a “treasure chest” to finance wrongdoing.

6. Perhaps the utility functions somehow do not stick.

7. The legal system for judging AI behavior may not be good enough, although under some assumptions that will just make the AIs all the more risk-averse (“better not even come close to breaking that law, they might sue me!”).

8. The AIs might use this legal system to collude with each other toward unfavorable ends.

9. Undercapitalized AIs might nonetheless win out in marketplace competition.

10. Perhaps some AIs can, on their own, accumulate wealth so rapidly that any feasible capital constraint does not bind them much.  Of course this scenario could create other problems as well, if AIs hold too much of societal wealth.

I am sure you can think of further possibilities.

In any case, the capitalization of AIs is a topic deserving of further discussion.  It is easy to think of the idea’s limitations, but in fact it works tolerably well for humans.  Most of all, it is a decentralized solution that economizes on the fact that full alignment will not in general be possible.

The economics of AI remains surprisingly understudied, even as technical progress in artificial intelligence continues rapidly. The Stripe Economics of AI Fellowship aims to help fill that gap by supporting foundational academic research in the area.

We invite graduate students and early-career researchers who are interested in studying the economics of AI to apply, regardless of prior experience. Fellows receive a grant of at least $10k, participate in conferences with leading economists and technologists, and have the potential to access unique data via Stripe and its customers. Our initial cohort will include 15-20 fellows, and will form the foundation of a community at the bleeding edge of economic research.

The link has further information, here is the tweet thread from the excellent Basil Halperin.

I can think of a few reasons:

At least for the time being, even very good LLMs cannot be counted on for originality.  And at least for the time being, good economic reasoning does not require originality, quite the contrary.

Good chains of reasoning in economics are not too long and complicated.  If they run on for very long, there is probably something wrong with the argument.  The length of these effective reasoning chains is well within the abilities of the top LLMs today.

Plenty of good economics requires a synthesis of theoretical and empirical considerations.  LLMs are especially good at synthesis.

In economic arguments and explanations, there are very often multiple factors.  LLMs are very good at listing multiple factors, sometimes they are “too good” at it, “aargh! not another list, bitte…”

Economics journal articles are fairly high in quality and they are generally consistent with each other, being based on some common ideas such as demand curves, opportunity costs, gains from trade, and so on.  Odds are that a good LLM has been trained “on the right stuff.”

A lot of core economics ideas are “hard to see from scratch,” but “easy to grasp once you see them.”  This too plays to the strength of the models as strong digesters of content.

And so quality LLMs will be better at economics than many other fields of investigation.