Category: Web/Tech
Midnight regulations on chip access
Let us hope the Biden administration does not do too much damage on its way out the door (WSJ):
The U.S. is preparing rules that would restrict the sale of advanced artificial-intelligence chips in certain parts of the world in an attempt to limit China’s ability to access them, according to people familiar with the matter.
The rules are aimed at China, but they threaten to create conflict between the U.S. and nations that may not want their purchases of chips micromanaged from Washington.
…The purchasing caps primarily apply to regions such as Southeast Asia and the Middle East, the people said. The rules cover cutting-edge processors known as GPUs, or graphic processing units, which are used to train and run large-scale AI models.
Should we not want to bring the UAE more firmly into the American orbit? Is there not a decent chance they will have the energy supply for AI that we are unwilling to build domestically? Might not these regulations, over time, encourage foreign nations to become part of the Chinese AI network? More generally, why should an outgoing administration be making what are potentially reversible foreign policy decisions for the next regime?
Ilya’s talk
Twenty-four minutes, sixteen minutes for the core talk, self-recommending.
A new paper on the economics of AI alignment
A principal wants to deploy an artificial intelligence (AI) system to perform some task. But the AI may be misaligned and pursue a conflicting objective. The principal cannot restrict its options or deliver punishments. Instead, the principal can (i) simulate the task in a testing environment and (ii) impose imperfect recall on the AI, obscuring whether the task being performed is real or part of a test. By committing to a testing mechanism, the principal can screen the misaligned AI during testing and discipline its behaviour in deployment. Increasing the number of tests allows the principal to screen or discipline arbitrarily well. The screening effect is preserved even if the principal cannot commit or if the agent observes information partially revealing the nature of the task. Without commitment, imperfect recall is necessary for testing to be helpful.
That is by Eric Olav Chen, Alexis Ghersengorin, and Sami Petersen. And here is a tweet storm on the paper. I am very glad to see the idea of an optimal principal-agent contract brought more closely into AI alignment discussions. As you can see, it tends to make successful alignment more likely.
Austin Vernon on drones (from my email)
The offensive vs. defensive framing seems wrong, at least temporarily. It should be motivated vs. unmotivated, with drones favoring the motivated.
A competent drone capability requires building a supply chain, setting up a small manufacturing/assembly operation, and training skilled operators. They need to manage frequencies and adjust to jamming. Tight integration of these functions is a necessity. That favors highly motivated groups with broad popularity (recruiting skilled talent!) even if they are nominally weak.
Conversely, it can be challenging for overly corrupt or complacent organizations to counter. They are also more likely to fracture and lose cohesion when under attack.
We’ve seen HTS, Burmese rebels, and Azerbaijan all have a lot of success with drones. Ukraine went from hopelessly behind in drone tech to leading Russia in innovation in many niches.
It seems reasonable that the barriers to entry for a motivated drone “startup” will go up. The US military has effective, expensive interceptors like Coyote Block II to counter small attacks in locations like Syria. Fighting larger entities requires pretty absurd scaling to match enemy numbers and the low per-flight success rate – Ukraine claims they might produce millions of drones this year. Hamas had initial success attacking Israel on Oct. 7 but didn’t have the magazine depth to defend themselves.
AI targeting, the necessity of specialized components to defeat electronic warfare, and cheaper drone interceptors are all factors that could upset this balance. Entities that have the scale to deploy an AI stack, true factories, and specialized components should gain the advantage if the rate of change slows.
How to read a book using o1
You don’t have to upload any book into the system. The Great Cosmic Mind is smarter than most of the books you could jam into the context window. Just start asking questions. The core intuition is simply that you should be asking more questions. And now you have someone/something to ask!
I was reading a book on Indian history, and the author reference the Morley reforms of 1909. I did not know what those were, and so I posed a question and received a very good answer, read those here. I simply asked “What were the Morley reforms done by the British in India in 1909?”
Then I asked “did those apply to all parts of India?”
You can just keep on going. I’ll say it again: “The core intuition is simply that you should be asking more questions.”
Most people still have not yet internalized this emotionally. This is one of the biggest revolutions in reading, ever. And at some point people will write with an eye toward facilitating this very kind of dialogue.
Before AI replaces you, it will improve you, Philippines edition
Bahala says each of his calls at Concentrix is monitored by an artificial intelligence (AI) program that checks his performance. He says his volume of calls has increased under the AI’s watch. At his previous call center job, without an AI program, he answered at most 30 calls per eight-hour shift. Now, he gets through that many before lunchtime. He gets help from an AI “co-pilot,” an assistant that pulls up caller information and makes suggestions in real time.
“The co-pilot is helpful,” he says. “But I have to please the AI. The average handling time for each call is 5 to 7 minutes. I can’t go beyond that.”
Here is more from Michael Beltran, via Fred Smalkin.
Info Finance has a Future!
Info finance is Vitalik Buterin’s term for combining things like prediction markets and news. Indeed, a prediction market like Polymarket is “a betting site for the participants and a news site for everyone else.”
Here’s an incredible instantiation of the idea from Packy McCormick. As I understand it, betting odds are drawn from Polymarket, context is provided by Perplexity and Grok, a script is written by ChatGPT and read by an AI using Packy’s voice and a video is produced by combining with some simple visuals. All automated.
What’s really impressive, however, is that it works. I learned something from the final product. I can see reading this like a newspaper.
Info finance has a future!
Addendum: See also my in-depth a16z crypto podcast (Apple, Spotify) talking with Kominers and Chokshi for more.
How badly do humans misjudge AIs?
We study how humans form expectations about the performance of artificial intelligence (AI) and consequences for AI adoption. Our main hypothesis is that people project human-relevant problem features onto AI. People then over-infer from AI failures on human-easy tasks, and from AI successes on human-difficult tasks. Lab experiments provide strong evidence for projection of human difficulty onto AI, predictably distorting subjects’ expectations. Resulting adoption can be sub-optimal, as failing human-easy tasks need not imply poor overall performance in the case of AI. A field experiment with an AI giving parenting advice shows evidence for projection of human textual similarity. Users strongly infer from answers that are equally uninformative but less humanly-similar to expected answers, significantly reducing trust and engagement. Results suggest AI “anthropomorphism” can backfire by increasing projection and de-aligning human expectations and AI performance.
That is from a new paper by Raphael Raux, job market candidate from Harvard. The piece is co-authored with Bnaya Dreyfuss.
AI-generated poetry is indistinguishable from human-written poetry and is rated more favorably
That is the title of a new paper in Nature, here is part of the abstract:
We conducted two experiments with non-expert poetry readers and found that participants performed below chance levels in identifying AI-generated poems (46.6% accuracy, χ2(1, N = 16,340) = 75.13, p < 0.0001). Notably, participants were more likely to judge AI-generated poems as human-authored than actual human-authored poems (χ2(2, N = 16,340) = 247.04, p < 0.0001). We found that AI-generated poems were rated more favorably in qualities such as rhythm and beauty, and that this contributed to their mistaken identification as human-authored. Our findings suggest that participants employed shared yet flawed heuristics to differentiate AI from human poetry: the simplicity of AI-generated poems may be easier for non-experts to understand, leading them to prefer AI-generated poetry and misinterpret the complexity of human poems as incoherence generated by AI.
By Brian Porter and Edouard Machery. I do not think that pointing out the poor quality of human taste much dents the import of this result.
*Bubbles and the End of Stagnation*
By Byrne Hobart and Tobias Huber, a new Stripe Press book.
Austrian economics and AI scaling
As many of you know, I grew up reading (on knowledge) Hayek, Michael Polanyi, Ludwig Lachmann, Mario Rizzo, and others in that tradition. For these Austrian and Austrian-related thinkers, knowledge is about how different parts of a system fit together, rather than being a homegeneous metric easily expressed on a linear scale. There is no legible way to assess the smarts of any single unit in the system, taken on its own. Furthermore, there are many “walls,” meaning knowledge is bumpy and lumpy under the best of circumstances. It thus makes little sense to assert that an entity is “3x smarter” than before. In my early 20s I received a second “dose” of related ideas through complexity theory and non-linear dynamics, some of those frameworks having Hayekian roots.
When people came along and made various predictions about AI following “scaling laws,” I was never extremely impressed. I did not feel I had the technical background to contradict them, I just was never sure they were measuring the social import of that knowledge in a meaningful way.
That is one (not the only) reason I have never been much persuaded by either the AI doomsters nor the AI utopians. Both seemed to me misguided rationalists, operating with a fundamentally pre-Hayekian understanding of knowledge. Furthermore, Ludwig Lachmann frequently told me that capital was a structure based on relationships of complementarity, mirroring the Hayekian knowledge insight in capital theory.
I find Matt Clifford’s observation “There is no AI-shaped hole in most organizations” to be a very useful starting point for analysis.
I am also much influenced by the history of artistic and scientific revolutions. Consider the Florentine artistic Renaissance, or say the blossoming of Germanic classical music in the 18th century. There were thick markets of rivalrous creators, sophisticated audiences, new technologies to work with, a diversity of funding sources, and strong beliefs that something very important was at stake, among other preconditions. Do we find similar preconditions active in AI worlds today? I would argue yes, but of course that could be contested. In any case, it is another and very different way of trying to understand the likely future pace of AI progress. “Does this feel like the environment that produced the Beatles and Bob Dylan?” “The Scot tinkerers of the Industrial Revolution?” “The German Romantics?”
Now in the last two days we have seen various media accounts (Bloomberg), some citing AI experts, claiming that scaling has slowed down or is no longer working. Put aside whether these claims are correct. I remain bullish on AI progress, because the AI world is showing so many signs of significant ferment. Ask yourself for instance — is it attracting the best minds and most ambitious people? Just as the postulation of AI scaling never so much excited me, neither does talk of its possible diminution much discourage me.
It is still a question of whether and at what pace we can find or create “AI-shaped holes” in organizations, or with individuals. And that is up to us.
My Conversation with the excellent Neal Stephenson
Here is the audio, video, and transcript. Here is part of the episode summary:
In Neal’s second appearance, Tyler asks him why he sometimes shifts from envisioning the future to illustrating the past, the rise of history autodidacts, the implications of leaked secrets from the atomic age to today’s AI, the logistics of faking one’s death, why he still drafts novels in longhand, Soviet idealism among Western intellectuals, which Soviet achievements he admires, the lag in AR development, how LLMs might boost AR, whether social media is increasingly giving way to private group chats, his continuing influence on technologists, why AI-generated art might struggle to connect with readers, the primer from The Diamond Age in light of today’s LLMs, the prospect of AGI becoming an unnoticed background tool, what Neal believes the world really needs more of, what lies ahead in Polostan and the broader “Bomb Light” series, and more.
Excerpt:
COWEN: How effectively could you stage your own death? You. Say you really want to do it, and you’re willing to do it.
STEPHENSON: To fake it or to actually —
COWEN: Fake it, but everyone thinks it’s real. I read about it in the papers. “Neal is gone.” I nod my head, I weep, and then I forget about it. I don’t mean I forget about you, but you understand what I’m saying.
STEPHENSON: Wait, there’s not that many circumstances under which all physical traces of someone can be obliterated. That’s a fairly hard thing to do. It would have been easier a hundred years ago, but now we’ve got cameras everywhere, and we’ve got DNA testing and other ways to prove or to disprove that somebody’s actually dead. I guess it would have to be something like a plane crash into the ocean.
COWEN: But then how do you survive it?
STEPHENSON: Oh, yes. Okay.
COWEN: To kill yourself is one thing, but to pretend you’ve killed yourself and stay alive seems harder.
STEPHENSON: You could parachute out if it was a small plane, not a jet airline full of people, but a single-seater. I guess that might work.
COWEN: So, hire a private plane, have it crash, parachute out into somewhere where you —
STEPHENSON: You’re witnessed getting into the plane and taking off, but then there’s no way to recover the evidence for some reason. It’s pretty hard to do. If someone really wanted to, if they were just determined to go and find the . . . You see the efforts that people have gone to to go down to the Titanic. Well, if you can go find that thing and check it out with a submarine, then it’s pretty hard to really find a place that can’t be accessed in that way.
I very much enjoyed Neal’s new book Polostan. And here is my first Conversation with Neal Stephenson.
Tariff sentences to ponder
In a September 2024 report, UBS, an investment banker, predicted both tech hardware and semiconductors to be among the top four sectors that would be hardest hit by a general tariff. Their analysis is spot on. Many of the hardware components that make AI and digital tech possible rely on imported materials not found or manufactured in the United States. Neither arsenic nor gallium arsenide, used to manufacture a range of chip components, have been produced in the United States since 1985. Legally, arsenic derived compounds are a hazardous material, and their manufacture is thus restricted under the Clean Air Act. Cobalt, meanwhile, is produced by only one mine in the U.S. (80 percent of all cobalt is produced in China). While general tariffs carry the well-meaning intent of catalyzing and supporting domestic manufacturing, in many critical instances involving minerals, that isn’t possible, due to existing regulations and limited supply. Many key materials for AI manufacture must be imported, and tariffs on those imports will simply act as a sustained squeeze on the tech sector’s profit margins.
Artificial Intelligence, Scientific Discovery, and Product Innovation
This paper studies the impact of artificial intelligence on innovation, exploiting the randomized introduction of a new materials discovery technology to 1,018 scientists in the R&Dlab of a large U.S. firm. AI-assisted researchers discover 44% more materials, resulting in a 39% increase in patent filings and a 17% rise in downstream product innovation. These compounds possess more novel chemical structures and lead to more radical inventions. However, the technology has strikingly disparate effects across the productivity distribution: while the bottom third of scientists see little benefit, the output of top researchers nearly doubles. Investigating the mechanisms behind these results, I show that AI automates 57% of “idea-generation” tasks, reallocating researchers to the new task of evaluating model-produced candidate materials. Top scientists leverage their domain knowledge to prioritize promising AI suggestions, while others waste significant resources testing false positives. Together, these findings demonstrate the potential of AI-augmented research and highlight the complementarity between algorithms and expertise in the innovative process. Survey evidence reveals that these gains come at a cost, however, as 82% of scientists report reduced satisfaction with their work due to decreased creativity and skill underutilization.
That is from a new paper by Aidan Toner-Rodgers. Via Kris Gulati.
Generative AI and the Nature of Work
Here is a new paper by the following set of authors: Manuel Hoffmann Harvard Business School, Sam Boysel Harvard Business School, Frank Nagle Harvard Business School, Sida Peng Microsoft Corporation, Kevin Xu GitHub, Inc. Here is the abstract:
Recent advances in artificial intelligence (AI) technology demonstrate considerable potential to complement human capital intensive activities. While an emerging literature documents wide-ranging productivity effects of AI, relatively little attention has been paid to how AI might change the nature of work itself. How do individuals, especially those in the knowledge economy, adjust how they work when they start using AI? Using the setting of open source software, we study individual level effects that AI has on task allocation. We exploit a natural experiment arising from the deployment of GitHub Copilot, a generative AI code completion tool for software developers. Leveraging millions of work activities over a two year period, we use a program eligibility threshold to investigate the impact of AI technology on the task allocation of software developers within a quasi-experimental regression discontinuity design. We find that having access to Copilot induces such individuals to shift task allocation towards their core work of coding activities and away from non-core project management activities. We identify two underlying mechanisms driving this shift – an increase in autonomous rather than collaborative work, and an increase in exploration activities rather than exploitation. The main effects are greater for individuals with relatively lower ability. Overall, our estimates point towards a large potential for AI to transform work processes and to potentially flatten organizational hierarchies in the knowledge economy.
Via the excellent Kevin Lewis.