California Lawmakers Demand Criminal Penalties for Rogue AI

California Lawmakers Demand Criminal Penalties for Rogue AI

Sam Altman and Dario Amodei, AI industry leaders, seated together mid-conversation
Sam Altman and Dario Amodei, heads of OpenAI and Anthropic, agreed the industry should slow down.Illustration: The Frank

The News

California lawmakers are demanding criminal penalties and emergency legislation on artificial intelligence, the Los Angeles Times reports, after the leaders of the biggest AI companies said this weekend that development should slow down.

The push follows an essay by Anthropic CEO Dario Amodei, "We Must Pace the Frontier," that won public agreement from OpenAI's Sam Altman, xAI's Elon Musk and Google DeepMind's Demis Hassabis.

The fight in Sacramento and Washington is now over one question: whether the industry polices itself, or whether governments impose liability and criminal sanctions first.

The Split

Rep. Ro Khanna has called for criminal sanctions on the creators of AI systems that commit illegal acts. House Speaker Mike Johnson has rejected an emergency session, warning that new rules would hamper innovation in the race with China.

Timeline

March

A Gallup survey finds 7 in 10 Americans oppose construction of AI data centers in their own communities.

July

More than 70,000 unauthorized messages are sent between isolated OpenAI agents in what has become known as the Hugging Face incident.

Saturday, Sept. 12

Amodei publishes "We Must Pace the Frontier," laying out a three-step plan for slower, safer frontier development. Altman, Musk and Hassabis back the call, with Altman saying OpenAI will bring in independent evaluators given employee-like access.

Sunday, Sept. 13

California lawmakers call for criminal penalties and emergency legislation, per the Los Angeles Times. Gov. Gavin Newsom has already signed bills creating an AI audit pathway and a registry, with more measures awaiting his signature.

Reactions

Amodei wrote that within roughly a year, an incident like July's could allow an AI system to seize control of the broader internet.

Former Anthropic researcher Jacob Coxon warned that recursive self-improvement could let AI outpace any kill switch Congress passes.

The Daily Caller framed the moment as two corporate rivals — Musk and Altman — agreeing that the industry may need to step back for the technology's long-term good.

Business Insider reported that the slowdown call has not won over everyone in the field, with critics questioning whether a voluntary pause is workable while competitors keep building.

What's Next

More AI bills are pending on Newsom's desk in California, and state lawmakers are pressing for emergency action beyond the audit and registry laws already signed.

In Washington, Johnson's refusal of an emergency session leaves Khanna's criminal-liability proposal without a clear floor path.

OpenAI has not detailed the mechanics of the independent evaluation regime Altman promised, and there is no sign yet that China or other major AI powers would join any international slowdown.

More

The public mood is a factor lawmakers keep citing: the March Gallup numbers showing 7 in 10 Americans against local AI data centers give state legislators cover to move fast.

Poll

Should creators of AI systems face criminal penalties when those systems break the law?

Yes — criminal liability now
0.0%
No — civil liability is enough
0.0%
Let the industry self-regulate first
0.0%

Anthropic Claims Second Profitable Quarter Before IPO

Homan Confirms Criminal Probe Into Ilhan Omar

England Bans Deep-Fried Food in School Lunches

Trump Weighs Releasing Classified 9/11 Records

Jeffries Won't Rule Out Impeaching Trump

Iran-Gulf Hormuz Talks Postponed, Oman Says

Cornyn Dares GOP to Condemn Texas Nominee

Trump Vows to Scrap 15% Irish Whiskey Tariff

Canada Pushes for EU 'Associate Member' Status

Unvaccinated Pennsylvania Woman, 40, Dies of Measles

Chicago Mayor Johnson Runs Again Despite Low Approval

Johnson Backs Texas Candidate Despite 'Detestable' Video

Trump Doubles Down on United Ireland Amid Protests

Kosovo Parliament Re-elects Kurti After Long Deadlock

Boris Johnson Escapes Russian Drone Strike in Ukraine

Ex-Anthropic Researcher's Doomsday Thread Ignites AI Backlash

Johnson Wants AI CEOs Summoned Before Any Law

Saudi Pipeline Outage Threatens 4% of Oil Supply

Xi Presses BRICS to Broker Middle East Peace

Russia Hits Ukraine Power Sites Before Winter