Controversial California Senate Bill Aims to Curtail AI’s Greatest Dangers

A bill introduced in the California state senate aims to prevent the dangers of artificial intelligence. Critics, including Google and Meta, say the bill could stifle innovation and even kill California’s significant tech industry.

The Senate bill, SB 1047, has been introduced by Senator Scott Wiener and is currently working through California’s state assembly. It will imminently face a full vote, and if it passes, it will then head to Governor Gavin Newsom’s desk.

The “first-of-its-kind legislation,” as The New York Times puts it, would introduce several requirements for AI companies, including safety testing of new technology before they are released publicly. The bill would also allow California’s attorney general “to sue companies if their technologies cause serious harm, such as mass property damage or human casualties,” NYT writes.

The bill would create the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, which would enact additional requirements on AI companies and their models, including the ability to “promptly enact a full shutdown.” The bill, if passed and signed into law, would also require that by January 1, 2028, AI developers annually retain a third-party auditor to perform an audit to ensure the developer complies with the bill’s laws.

Wiener crafted the bill with input from the San Francisco-based Center for AI Safety, which says it believes “that artificial intelligence (AI) has the potential to profoundly benefit the world, provided that we can develop and use it safely.” The organization maintains that “many basic problems in AI safety have yet to be solved.”

One of the Center for AI Safety’s founders, Dan Hendrycks, tells The New York Times that “The bill is a call to make sure that these systems don’t have hazards or, if the hazards do exist, that the systems have the appropriate safeguards.”

Some believe AI technology is already causing harm, such as spreading disinformation across different mediums, eliminating human jobs, and causing general anxiety and concern. Experts think that much more significant harm is around the corner.

“Historically, we have waited for bad things to happen and then wrung our hands and dealt with it later, sometimes when the horse was out of the barn and it was too late,” Mr. Wiener explains in an interview with The New York Times. “So my view is, let’s try to, in a very light touch way, get ahead of the risks and anticipate the risks.”

Google and Meta have responded, sending letters to Senator Wiener. Anthropic also opposes the bill, adding that the government shouldn’t be involved in regulation until AI has caused real harm. It’s unclear what constitutes the damage in question or who would determine when or if the harm has occurred.

The criticism of the bill primarily comes down to fears that it would open up additional liability for AI companies and disrupt development.

“It could stifle innovation,” Lauren Wagner explains. Wagner is an investor and researcher who has previously worked for Google and Meta.

AI researcher and entrepreneur Jeremy Howard fears the bill would make AI technology less open-source, harming smaller developers while ensuring only the biggest fish in the pond have the most powerful AI.

“These organizations would have more power than any country — any entity of any kind. They would be in control of an artificial super intelligence. That is a recipe for disaster,” Howard explains.

Others worry the bill would encourage big companies to leave California for greener pastures with fewer rules, whether that’s other states or even different countries altogether. A tech exodus, especially if it included big companies like Google, would be devastating for California.

It’s not apparent what the best course of action is concerning AI regulation and safety, as there is widespread disagreement on the best way to handle the current risks and especially future ones. There’s little consensus on those risks, making the path forward all the more challenging.

Image credits: Featured image created using an asset licensed via Depositphotos. California state flag is in the public domain.