Anthropic CEO says government should block dangerous AI

The government should legally be able to block or deter dangerous AI deployments, Anthropic CEO Dario Amodei wrote in an essay Wednesday.

Why it matters: Anthropic's ideas for tech regulation and economic disruption from AI go far beyond anything currently under serious consideration in Washington right now.

• They're also sure to stir up a new set of accusations that Anthropic is proposing strict rules to lock in its own dominance or using frightening future scenarios as a marketing ploy.

Driving the news: Amodei's new essay and proposed advanced AI framework argues that policy has to change in response to the rapid development of AI.

• Trump's AI executive order should go further, he writes, and require mandatory testing for risks related to cybersecurity, biological weapons, loss of control or automated R&D.
• Even more aggressive regulation might be needed in the future if AI systems become more of a threat, he writes.
• Anthropic is also proposing an economic policy framework to address AI disruption including capital accounts, wage insurance, tax incentives and an expanded social safety net.

What they're saying: "AI is advancing at a lightning pace," Amodei writes. "...By contrast, policy—and especially legislation—moves very slowly."

• Following the release of Anthropic's power model Mythos, Amodei writes that biological risks and "serious AI autonomy risks" may soon follow.
• "We now, globally and collectively, need to activate a slow and rickety policy apparatus to deal with risks and opportunities that are going to compound surprisingly quickly from here."

Amodei writes that existing transparency legislation is not enough and calls for "more serious and binding regulation of AI."

• Like cars, airplanes, or drugs, he writes, AI regulation should require frontier models to go through rigorous testing and auditing.
• "Their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety."

On the economy, Amodei writes that "it's reasonable to think that AI could produce much larger disruptions to the labor market than previous technologies, and, potentially, more enduring disruptions."

• He calls for better data on AI-related job loss, wage insurance, retention tax incentives and possibly universal basic income or universal capital accounts.
• Public opposition to data center buildouts is "largely a symbol or outlet for broader economic anxieties about AI," Amodei writes.

Between the lines: Amodei also writes that regulatory systems are not prepared for how quickly AI will bring new advancements.

• He suggests reform at agencies like the FDA to approve new drugs discovered by AI faster.
• He also suggests banning domestic use of fully autonomous weapons and advocates for continued leadership and coordination from democratic countries on AI.

The bottom line: Amodei still wants the public to be aware of AI's benefits.

• "I am optimistic about finding solutions because many of these issues—from addressing job displacement, to pre-release testing of models, to export controls on chips, to other AI related policy issues such as energy use—have a common-sense appeal across the political spectrum," he wrote.