Anthropic shops a draft federal bill to block AI models that pose an imminent catastrophic risk

AIAI #ai-policy #ai-safety #anthropic #Congress #regulation

Anthropic shops a draft federal bill to block AI models that pose an imminent catastrophic risk

TechCrunch and Tom's Hardware summarize Politico's scoop on the draft, including who would decide and the 50-deaths threshold.

Anthropic has been circulating a draft AI safety bill it hopes Congress will take up, according to Politico as summarized by TechCrunch and Tom's Hardware. A model's release could be blocked only for a "foreseeable, imminent risk" of more than 50 deaths or injuries or over $1 billion in damage, stemming from a biological weapon, a cyberattack or a model escaping its developer's control. The thresholds resemble those in California's SB 53.

Independent evaluators registered with the Commerce Department would assess labs and could be paid by them, as long as payment doesn't depend on results. Evaluators would have 72 hours to report an imminent risk to Commerce, which would ask a federal judge in Washington, D.C. to halt the release. The draft bars any other agency from ordering a release stopped. An Anthropic spokesperson told Politico the bill reflects its Advanced AI Framework, which it has pushed with lawmakers since June.

Read it at the source

TechCrunchAnthropic pitches AI safety bill that may block a model if 50+ people could be killed Independent ↗ Tom's HardwareAnthropic’s proposed AI safety bill only kicks in if 50 deaths or $1 billion in damages are on the line — proposal would effectively make the Commerce Department the only agency capable of stopping an AI model Independent ↗

Across the spectrum 2

Democrat

Scott Wiener, California state senator (D), author of SB 53: Wiener said Anthropic's draft "falls far short of protecting public health and safety," The Next Web reports: it set no minimum standards for safety plans or auditors and, he said, misused a definition from SB 53 to "gut" the draft's emergency shutdown powers. "Congress must go further," he wrote on X.

The Next Web ↗

Libertarian

Kevin T. Frazier and Dean W. Ball, in commentary published by the Cato Institute: On the related question of pre-release government review of frontier models, a May 2026 commentary (first in Lawfare) argued the administration "lacks authority to mandate frontier model vetting" but that existing CAISI and CISA tools "enable a voluntary alternative": labs could opt to share models with materially new capabilities for a "kick the tires" testing period.

Cato Institute ↗

Cascadic Analysis 2

UndertowDelayed Consequence / False Success Problem

What hidden risk could pull against this, even if the news is good?

The draft triggers only on a "foreseeable, imminent risk" found before release. Harms that build slowly, or that appear only after a model is widely used, fall outside it: a model can pass the release check and still cause damage that no evaluator flagged.

UndertowWe don't completely understand why frontier models behave as they do

What hidden risk could pull against this, even if the news is good?

Blocking a release depends on an evaluator predicting what a model will do, including whether it might escape its developer's control. Testing can show a model behaving safely in the cases tried; it can't fully establish how it will behave in situations no one tested, which is the gap a "foreseeable" standard has to live with.

Rabbit Holes 1

What is California's AI safety law?

After exploring Anthropic pitches AI safety bill that may block a model if 50+ people could be killed, for the state law the draft echoes, read What is California's AI safety law?.

Brookings' explainer on California's SB 53, the frontier AI safety law whose catastrophic-risk thresholds the draft bill resembles.

Brookings · Swim · 30 min

0