Séb Krier, continued
Or more specifically Nenad Tomašev, Matija Franklin, Julian Jacobs, Sébastien Krier, and Simon Osindero:
AI safety and alignment research has predominantly been focused on methods for safeguarding individual AI systems, resting on the assumption of an eventual emergence of a monolithic Artificial General Intelligence (AGI). The alternative AGI emergence hypothesis, where general capability levels are first manifested through coordination in groups of sub-AGI individual agents with complementary skills and affordances, has received far less attention. Here we argue that this patchwork AGI hypothesis needs to be given serious consideration, and should inform the development of corresponding safeguards and mitigations. The rapid deployment of advanced AI agents with tool-use capabilities and the ability to communicate and coordinate makes this an urgent safety consideration. We therefore propose a framework for distributional AGI safety that moves beyond evaluating and aligning individual agents. This framework centers on the design and implementation of virtual agentic sandbox economies (impermeable or semi-permeable), where agent-to-agent transactions are governed by robust market mechanisms, coupled with appropriate auditability, reputation management, and oversight to mitigate collective risks.
Here is the link, this is some of the most important work of our time. Here is the previous MR post on Krier.
Saturday assorted links
1. Henry Oliver’s year in reading.
2. What is the greatest artwork of the century so far? One man’s view.
3. LLMs specific to a historical time period.
4. So is it actually Austro-Japanese business cycle theory?
Rent Control Creates Ghost Apartments
Adam Lehodey writing at City Journal:
In New York City, making a profit on real estate has become increasingly difficult. Rent-stabilization laws built on the mantra that “housing is a human right,” a dysfunctional housing court, and myriad other interventions have driven thousands of units off the market, giving rise to the phenomenon of New York’s “ghost apartments.”
The city now has nearly 50,000 empty units, absent from the market either because their operating costs exceed legal rents or because they require considerable renovations.
…Take a building on East 6th Street as an example. A mere five-minute walk from Tompkins Square, the building is a convenient home for students and young professionals.
One-bedroom units in the building average $3,500— except two of them, subject to the city’s rent-stabilization laws, which hold rents below $900 per month.
As a result, both units have been allowed to fall into disrepair, because the cost of restoring them to habitability is greater than what they’d generate in rent.
…Much of the predicament at the East 6th Street building and the apartments on Valentine Avenue can be traced back to one piece of legislation: the 2019 Housing Stability and Tenant Protection Act (HSTPA). Passed by a Democratic majority in the state legislature, HSTPA eliminated landlords’ abilities to raise rents after units were vacated, or when they exceeded $2,775 per month. In doing so, it also eliminated their ability to make improvements profitably and reset the stabilized rent.
Recall from the recent review by Kholodilin that “the published studies are almost unanimous with respect to the impact of rent control on the quality of housing….[namely] that rent control leads to a deterioration in the quality of those dwellings subject to regulations.”
Markets in everything
People Are Paying to Get Their Chatbots High on ‘Drugs’
An online marketplace is selling code modules that simulate the effects of cannabis, ketamine, cocaine, ayahuasca, and alcohol when they are uploaded to ChatGPT.
Here is the full Wired article. Via David.
Does the conflict between cardinal utility and ordinal preferences just keep on getting worse?
This argument is not necessarily a critique of capitalism, but it could be. At the very least, it is an observation about advanced capitalism.
As you will know from philosophy, there is a difference between what makes you happy, in the felicific sense, and what you want. Some of this difference may be due to addictions, but most of it is not. You may want to be a person of a particular kind, whether or not that makes you happier. You may wish to do things to help the world, without believing you will be personally happier as a result. You might have mixed feelings as to whether having children will make you happier (stress!), but still you might have a deep preference for raising a family. And so on. These distinctions are part of the mainsprings of human life, they are not minor exceptions standing in the corner.
The more capitalism develops, the more the gap between cardinal utility and preference satisfaction is likely to grow. Consider the polar case of a very primitive economy where the only commodity is rice. Eating rice is what makes you happy, and eating rice is also how you wish to spend your money. After all, what else is there? Given the feasible set, cardinal utility and preference satisfaction will coincide perfectly.
But as product choice grows and incomes rise, you will have more and more chances to deviate from maxing out on cardinal utility. Furthermore, your immediate “needs” likely are taken care of, so most of your income spending is discretionary rather than “I need to buy this food to avoid the miseries of starvation.”
More and more, you will be led away from cardinal utility maximization. But additional preferences will be satisfied.
Is this good or bad?
It is not quite right to say that people are becoming less happy, as they are getting what they want. That could be a central component of the good life, and of individual well-being, broadly construed. That said, some of your ordinal preferences might be harmful addictions, or you might prefer things that stress you out, either proximately or in the longer run.
Let’s say you keep on checking your phone for texts. Do you do this because you think it will make you happier? Maybe not. You simply might have a preference for wanting to know the information in those texts as soon as possible. Should we think that preference is bad? Maybe it is a mother wanting to know that her daughter got home safely, and so she checks her texts every three minutes. That might not make her happier, but I am reluctant to conclude that is a worse state of affairs. And it does not have to be an addiction, a much overused concept by intelligent people who do not define it very carefully.
I too have plenty of preferences that do not make me happier, though I consider them quite legitimate. I am keen to see as much of the world as I can, yet I am not convinced this makes me happier than say simply going back to Mexico again and again and eating the street food. I just want to know what else is out there.
If you side solely with cardinal utility, yes you condemn capitalism. Or if you think all of these ordinal preferences are addictions, again you can condemn the status quo. Your meta-preferences in that case presumably would wish to have different preferences. In any case, many books will be written about how capitalism makes us miserable. Most of them will have the incorrect framing, though most of them will have ” a point,” one way or another. Furthermore, while some of these books may be correct, in the aggregate they will push us away from viewing individual human beings as agentic. That is a negative social consequence.
I do not think those critical perspectives are, by and large, the primary correct views. Instead, I think of capitalism and markets as an unparalleled engine for making us…weirder? And for moving us into different worlds (NYT)?
YMMV.
*Central Asia*, by Adeeb Khalid
An excellent book, the best I know of on this region. Here is one bit:
The first printing press in Central Asia was established in Tashkent in 1870…
I had not understood how much Xinjiang (“East Turkestan”), prior to its absorption into newly communist China, fell under the sway of Soviet influence.
I had not known how much the central Asian republics had explicit “let’s slow down rural migration into the cities” policies during Soviet times.
The book is interesting throughout, recommended.
Friday assorted links
1. The negativity crisis of AI ethics.
2. Vitalik and governance experiments and culture.
3. Stripe runs an RCT on capital markets and lending.
4. “For the first time, an AI model (GPT-5) autonomously solved an open math problem submitted to our benchmarking project IMProofBench, with a complete, correct proof, without human hints or intervention.” Link here, it is amazing how many smart or accomplished people will deny this is possible.
5. Developments in cognitive dissonance (New Yorker).
Voices From 2099
Great little video. Winner of the Foresight Institute’s $10,000 prize for Existential Hope. Go Bryan Johnson!
Falling costs
Unbelievable progress that even I underestimated! Gemini 3 Flash has practically beaten ARC-AGI-1 [an AI evaluation] at cost/score parity! It achieved the same score at more than 500x lower cost than the o3 model from a year ago & 6x lower than the just-released GPT-5.2!
Here is the link.
Nabeel on reading Proust
Yet not a word is wasted. It sounds paradoxical, but Proust is economical with his prose. He is simply trying to describe things that are extremely fine-grained and high-dimensional, and that takes many words. He is trying to pin down things that have never been pinned down before. And it turns out you can, indeed, write 100 pages about the experience of falling asleep, and find all kinds of richness in that experience.
And this:
…, a clear-sightedness on human vanity and a total willingness to embarrass himself. There are passages in the Albertine sections which are shocking – such as the extended stretch, around 50 pages long, in which he describes watching her sleep — and, reading them, you start to understand that this was written by a dying man who did not care about anything apart from telling the whole truth in as merciless way as possible.
Third, hypotaxis in sentences. The opposite of hypotaxis is parataxis, which you often find in Hemingway, as in: “The rain stopped and the crowd went away and the square was empty.” Each item here is side by side, simple, clean. The Bible often uses such types of sentences: “And God said, Let there be light: and there was light. And God saw the light, that it was good: and God divided the light from the darkness.”.
Hypotaxis, by contrast, describes sentences with many subordinate clauses, like nesting dolls.
Nabeel says In Search of Lost Time is now his favorite novel.
Japan estimate of the day
At current second-hand market prices, says a new report, Japan’s “hidden asset” in terms of national reserves of things — defined as potentially resellable household objects that have lain unused for over a year — is worth around $580bn.
The dust-gathering contents of Japan’s cupboards, attics and garages, by that estimate, are worth roughly the same as the combined market capitalisation of the country’s most globally known corporate names: Toyota, Sony and SoftBank.
That’s an impressive stash, equivalent to roughly $4,600 for every person in Japan…
Over the past few years, Japan has become a uniquely attractive global magnet for buyers of second-hand goods — from Hermès bags, Rolexes and limited edition Nike Airs to Pokémon trading cards, vintage video games, golf clubs, fishing rods and rare Licca dolls. An increasingly powerful appeal for the tens of millions of visitors the country now draws annually is not just the traditional shopping, but the vibrant, over-the-counter trade in used items.
The aging of Japan, the fastidiousness of many Japanese, and the cheap yen are two factors behind these developments. Here is more from Leo Lewis at the FT.
Thursday assorted links
1. Andrew Batson best books he read in 2025.
2. Demis on AI.
3. Dean Ball on AGI and the programmer’s mentality.
4. New data on long-term warming trends.
5. A possible Netflix adaptation of Caro’s The Power Broker? And maybe just maybe a Villaneuve film of Rendezvous with Rama?
6. Do lower mortgage rates in fact benefit first-time home buyers?
7. Henry Oliver on Kiran Desai.
8. Australia to “crack down” on hate speech (NYT).
9. Amanda Taub at the NYT covers dogs, babies, and Taiwan.
10. That was then, this is now: “As much as a quarter of the active US navy is now in the Caribbean, according to one estimate.” (FT)
Economics job market update
From John A. List:
AEA job market update. The numbers don’t lie, as this is the toughest market for PhD economists in recent memory.
JOE listings are down 20% from last year. Worse: they are 19% below COVID levels. Let that sink in.
The academic market took the biggest hit. Full-time US positions dropped 33% year-over-year. Liberal arts colleges and PhD-granting universities? Both down about a third. International academic postings fell 13% from last year, 25% from COVID.
Nonacademic isn’t much better: down 27% from last year, 45% below COVID. And federal government hiring? That’s where it gets ugly. Down 71% year-over-year, 79% below COVID. DOGE cuts plus the shutdown created a perfect storm.
One bright spot: private sector jobs in consulting, research, banking, and finance are holding steady at recent-year levels.
Bottom line for candidates: the data confirm what you’re feeling. It’s brutal out there. Universities facing their own financial pressures should still find ways to bridge unmatched candidates for another year. The talent is there—the opportunities aren’t. H/T John Cawley
Here is the link to the tweet.
My Conversation with Alison Gopnik
Here is the audio, video, and transcript. Here is part of the episode summary:
Tyler and Alison cover how children systematically experiment on the world and what study she’d run with $100 million, why babies are more conscious than adults and what consciousness even means, episodic memory and aphantasia, whether Freud got anything right about childhood and what’s held up best from Piaget, how we should teach young children versus school-age kids, how AI should change K-12 education and Gopnik’s case that it’s a cultural technology rather than intelligence, whether the enterprise of twin studies makes sense and why she sees nature versus nurture as the wrong framework entirely, autism and ADHD as diagnostic categories, whether the success of her siblings belies her skepticism about genetic inheritance, her new project on the economics and philosophy of caregiving, and more.
Excerpt:
COWEN: If it’s something like height, where there is clearly an environmental component, especially if the child is not well-fed, but it seems perfectly fine to say above a certain dietary level, it’s mostly genetic, right? No one says that’s ambiguous, and more and more traits will become like that.
GOPNIK: Well, first of all, I’m not sure that’s true. To a striking degree, the traits that people have looked at, like educational attainment, for example — we haven’t found consistent relationships to genetics. I think the reason for that is exactly because there’s this very complicated developmental process that goes from the genetics to the outcome.
Even if you think about fruit flies, for example. I have some geneticist colleagues who work on this — fruit fly sex determination. You’d think, “Well, that has to be just the result of genes.” It turns out that there’s this long developmental — long by fruit fly standards — developmental process that goes from the genetics to the proteins to the morphology, and there’s lots of possibility of variation throughout that. I think that hasn’t turned out to be a scientifically helpful way of understanding what’s going on in development.
The other thing, of course, is, from my perspective, the common features of, say, what kids are doing are much more interesting than the variations. What I really want to know is how is it that anyone could have a brain that enables them to accomplish these amazing capacities? Thinking about, is this child smarter than the other one, given how unbelievably smart all of them are to begin with, I just think it’s not an interesting question.
COWEN: But say, what you would call the lay belief that smarter parents give birth to smarter children, at least above subsistence — surely you would accept that, right?
GOPNIK: Again, what does smarter mean?
COWEN: How you would do on an IQ test.
GOPNIK: What does genetics mean? It’s interesting, Tyler, that IQ tests, for example — they have their own scholarly and scientific universe, but they’re not something that we would teach about or think about in a developmental psychology class, and there’s a good principled reason for that. The good principled reason — this has come up a lot in AI recently. There’s this idea in AI of artificial general intelligence, and that is assuming that there’s something called general intelligence.
Again, I think, a lot like consciousness or life, it’s one of these lay ideas about how people work. When you actually look at children, for example, what you see is not just that there isn’t a single thing that’s general intelligence. You actually see different cognitive capacities that are in tension with one another. You mentioned one about the scientist who’s trying to think of some new idea versus the scientist who’s looking at a more specific idea, right? A classic example of this tension that I’ve talked about and studied is in computer sciences: exploration versus exploitation.
What do you count as IQ? In fact, most of what IQ is about is how well do you do in school? How well do you do on school tests? That’s actually, in many respects, in tension with how good are you at exploring the world around you? The kinds of things that you need to do to have particular goals, to accomplish them, the kinds of things that we emphasize a lot, say, in a school context, are actually in tension. This gets back to the point about babies being more conscious than we are — are actually in tension with the kinds of things that will let you explore.
Think about the Bayesian example. If you have a flatter prior, and you pay more attention to evidence, you are probably not going to do as well on an IQ test…
COWEN: There’s you — you’re tenured at Berkeley, you’re famous. There’s Blake, The Definitive Warhol Biography, and Adam, who’s amazing, writes for the New Yorker, and you don’t believe inheritability and IQ being very concrete things? I just don’t get it. I think you’re in denial.
GOPNIK: Actually, I think that example is maybe partly why I don’t believe in that. In fact, what I do believe is that the effect of caregiving is to increase variability, is to increase variation. Our family, our care — there were six of us in 11 years. My parents were graduate students, and even before they were graduate students, they were that great generation of immigrant kids.
We had this combination of a great deal of warmth, a great deal of love, an enormous amount of stuff that was around us — books and ideas. We got taken to the Guggenheim, when Adam was three and I was four, for the opening of the Guggenheim. We both remember this vividly. But we were also completely free. We were just in regular public schools. As was true in those days, in general, we came home after school, and we basically did whatever it was that we wanted. I was involved. The kids were taking care of each other a lot of the time.
The result is that you get a lot of variation. It’s an interesting example in our family where we have six kids who presumably all have somewhat similar genetics, all in that 11 years grow up in the same context, and they come out completely differently. They come out with really different strengths, really different weaknesses, things that they’re good at, things that they’re not good at. Even if you think about what Blake and Adam and I are like as thinkers, we’re all foxes instead of hedgehogs. We’re all people who have done lots of different things and thought about lots of different things.
So, my view is that what nurture will do is let you have variability. That’s the thing that, in a sense, is heritable. That’s contradictory, the idea that what’s heritable is the standard deviation instead of the mean, but that’s my view about that. I think my childhood did have the effect of making me suspicious of those simple nature-nurture oppositions.
Here are the books of Alison Gopnik.
An RCT on AI and mental health
Young adults today face unprecedented mental health challenges, yet many hesitate to seek support due to barriers such as accessibility, stigma, and time constraints. Bite-sized well-being interventions offer a promising solution to preventing mental distress before it escalates to clinical levels, but have not yet been delivered through personalized, interactive, and scalable technology. We conducted the first multi-institutional, longitudinal, preregistered randomized controlled trial of a generative AI-powered mobile app (“Flourish”) designed to address this gap. Over six weeks in Fall 2024, 486 undergraduate students from three U.S. institutions were randomized to receive app access or waitlist control. Participants in the treatment condition reported significantly greater positive affect, resilience, and social well-being (i.e., increased belonging, closeness to community, and reduced loneliness) and were buffered against declines in mindfulness and flourishing. These findings suggest that, with purposeful and ethical design, generative AI can deliver proactive, population-level well-being interventions that produce measurable benefits.
That is from a new paper by Julie Y.A. Cachia, et.al. A single paper or study is hardly dispositive, even when it is an RCT. But you should beware of those, such as Jon Haidt and Jean Twenge, who are conducting an evidence-less jihad against AI for younger people.
Via the excellent Kevin Lewis.
In New York City, making a profit on real estate has become increasingly difficult. Rent-stabilization laws built on the mantra that “housing is a human right,” a