Generalist Intelligence: The Off Switch
The second-order effects of Anthropic's war with Washington.
Friends,
Welcome to another edition of Generalist Intelligence, the weekly intelligence briefing designed to deliver situational awareness and help you see around corners. All in 30 minutes or less.
To learn more about our process and vision for Generalist Intelligence, read our introductory piece. To unlock the full briefing, join as a member.
This is a new initiative, so we’d love to hear what you think. Reply anytime to tell us what's working and what you'd like us to cover more deeply. Our goal is for this to become the highest signal-to-noise newsletter you receive each week, and one worth setting aside the time for.
Dario has been having one of those weeks. Just days after releasing Fable — a supposedly guard-railed version of its “too powerful to be safely released” Mythos model — Anthropic got a call from the US government. Amazon had found a way to bypass some of the safety features. The fear was that bad actors (read: China) would find the same exploit and use the power for nefarious purposes. Anthropic was apparently given 90 minutes (last thing on a Friday afternoon) to comply with export controls that denied access to foreigners, even if they worked for Anthropic. The only practical option was to deny everyone, Americans included.
Whatever you think of the relevant parties (some at Generalist Towers describe themselves, unashamedly, as “Anthropic-pilled”), this sucks for the Claude-maker, is bad for America, and is revealing about the future.
What if it’s actually a good thing?
Nice try. There are people arguing that getting knocked around by Donald Trump (again) might be “good for business,” pointing to an uptick in Claude receipts after the president previously declared Anthropic a “supply-chain risk.” Others, rather gamely, suggest that being officially branded “too powerful” creates an aura that attracts customers. Both are wrong. Anthropic already declared Mythos too powerful, then spent two months building guardrails that have, one way or another, landed it in this mess. Either the guardrails aren’t good enough, which is embarrassing, or the US government (and, for some reason, Anthropic’s closely intertwined partner Amazon) has decided to try and slow them down. Both are major red flags to anyone thinking of buying from Anthropic.
I thought Anthropic wanted AI to be regulated?
Yes, there’s an irony in Dario, having made so much noise about the dangers of the AI he’s building and the need for government to constrain his almighty power, looking quite so hurt when they take him at his word. But what Anthropic actually wants is something like the Federal Aviation Administration: an arms-length regulator that keeps passengers safe without dragooning business. Not whatever this is.
What does it mean for America?
Seems bad, basically. It’s great having world-beating AI firms, but you do rather rely on the rest of the world buying their stuff to get the full benefit. The next few years look crucial in the development of AI, and for those next few years, Trump will be president. That means Anthropic, and in theory, every US model-maker, lives or dies by Caesar’s thumb. If you’re an international company buying AI tools, that’s a lot of risk.
Where else are they going to go?