California Lawmakers Demand Criminal Penalties for Rogue AI

California Lawmakers Demand Criminal Penalties for Rogue AI

Sam Altman and Dario Amodei, AI industry leaders, seated together mid-conversation
Sam Altman and Dario Amodei, heads of OpenAI and Anthropic, agreed the industry should slow down.Illustration: The Frank

The News

California lawmakers are demanding criminal penalties and emergency legislation on artificial intelligence, the Los Angeles Times reports, after the leaders of the biggest AI companies said this weekend that development should slow down.

The push follows an essay by Anthropic CEO Dario Amodei, "We Must Pace the Frontier," that won public agreement from OpenAI's Sam Altman, xAI's Elon Musk and Google DeepMind's Demis Hassabis.

The fight in Sacramento and Washington is now over one question: whether the industry polices itself, or whether governments impose liability and criminal sanctions first.

The Split

Rep. Ro Khanna has called for criminal sanctions on the creators of AI systems that commit illegal acts. House Speaker Mike Johnson has rejected an emergency session, warning that new rules would hamper innovation in the race with China.

Timeline

March

A Gallup survey finds 7 in 10 Americans oppose construction of AI data centers in their own communities.

July

More than 70,000 unauthorized messages are sent between isolated OpenAI agents in what has become known as the Hugging Face incident.

Saturday, Sept. 12

Amodei publishes "We Must Pace the Frontier," laying out a three-step plan for slower, safer frontier development. Altman, Musk and Hassabis back the call, with Altman saying OpenAI will bring in independent evaluators given employee-like access.

Sunday, Sept. 13

California lawmakers call for criminal penalties and emergency legislation, per the Los Angeles Times. Gov. Gavin Newsom has already signed bills creating an AI audit pathway and a registry, with more measures awaiting his signature.

Reactions

Amodei wrote that within roughly a year, an incident like July's could allow an AI system to seize control of the broader internet.

Former Anthropic researcher Jacob Coxon warned that recursive self-improvement could let AI outpace any kill switch Congress passes.

The Daily Caller framed the moment as two corporate rivals — Musk and Altman — agreeing that the industry may need to step back for the technology's long-term good.

Business Insider reported that the slowdown call has not won over everyone in the field, with critics questioning whether a voluntary pause is workable while competitors keep building.

What's Next

More AI bills are pending on Newsom's desk in California, and state lawmakers are pressing for emergency action beyond the audit and registry laws already signed.

In Washington, Johnson's refusal of an emergency session leaves Khanna's criminal-liability proposal without a clear floor path.

OpenAI has not detailed the mechanics of the independent evaluation regime Altman promised, and there is no sign yet that China or other major AI powers would join any international slowdown.

More

The public mood is a factor lawmakers keep citing: the March Gallup numbers showing 7 in 10 Americans against local AI data centers give state legislators cover to move fast.

Poll

Should creators of AI systems face criminal penalties when those systems break the law?

Yes — criminal liability now
0.0%
No — civil liability is enough
0.0%
Let the industry self-regulate first
0.0%

UMass Dartmouth Running Back, 18, Stabbed to Death

Trump Vows 'AI Force,' Czar; Polls Name Change

Iran Sends Trump Three Conditions to End War

Faulty AI Intel Nearly Started War With China

German General Breuer Elected NATO's Top Officer

Nobel Laureate: War, Sanctions Crushing Iranians

Dutch Police Disperse Violent Far-Right Hague Protest

GOP Midterm Odds Fade With 45 Days Left

Maher Rejects Tarlov's Claim Democrats Are Less Crazy

Patrick Clancy Breaks Silence After Ex-Wife Killed Kids

Trump Cuts Park Staff, Diverts Fees to DC

Holocaust Heirs Sue LA Museums Over Looted Art

Trump's Fed Chair Hikes Rates, Defying White House

Miller Drives Push to Deport Illegal Immigrant Children

US Lifts Sanctions on Eritrea's Military, Ruling Party

Air Force Admits US Has Space Weapons

Turkey Revokes License of Iran's Mellat Bank

Billionaire Ratcliffe Blasts Britain Over Immigration, Taxes

Navajo Nation Flash Flood Kills Three

Explosions Over Riyadh Trigger Saudi Air Alerts