The Latest News on SB 1047, California’s Attempt to Govern Artificial Intelligence

California takes bold step towards AI regulation: The California legislature has passed SB 1047, a groundbreaking bill aimed at governing artificial intelligence systems, particularly focusing on the potential risks associated with foundation AI models.

Key provisions of SB 1047: The bill introduces comprehensive AI safety requirements for companies operating in California, addressing concerns about the existential risks posed by advanced AI systems.

• Companies must implement precautions before training sophisticated foundation models, including the ability to quickly shut down the model if necessary.
• The legislation mandates protection against “unsafe post-training modifications” to AI models.
• A testing procedure must be established to evaluate whether a model or its derivatives pose a risk of “causing or enabling a critical harm.”

Mixed reactions from tech industry and lawmakers: The bill has been previously amended and has sparked both support and criticism from various stakeholders in the AI and technology sectors.

• Mozilla expressed concern that the bill could harm the open-source community.
• OpenAI warned that it might impede the growth of the AI industry.
• Rep. Nancy Pelosi (D-CA) called the bill “well-intentioned but ill informed.”
• After amendments softening some provisions, Anthropic concluded that the bill’s benefits likely outweigh its costs.
• Former Google AI lead Geoffrey Hinton described it as “a sensible approach” for balancing risks and technological advancement.

Legislative journey and current status: SB 1047 has successfully navigated the California legislative process, awaiting the governor’s decision.

• The bill passed both the California State Assembly and Senate.
• Governor Gavin Newsom has until the end of September to sign or veto the bill.
• If signed, it would become one of the first significant AI regulations in the United States.

Potential impact on the AI industry: The bill’s passage could have far-reaching consequences for AI companies, particularly those based in California.

• Major foundation model companies headquartered in California would be directly affected by the new regulations.
• OpenAI’s chief strategy officer, Jason Kwon, argued that AI regulations should be left to the federal government to avoid slowing progress and causing companies to leave the state.

Broader context of AI regulation in California: SB 1047 is part of a larger trend of AI-related legislation in California.

• In January, Senator Steve Padilla introduced Senate Bills 892 and 893, aiming to establish a public AI resource and create an ethical framework for AI in the state.
• Senator Scott Wiener proposed a bill in September 2023 to regulate “frontier” model systems based on their computing power and establish transparency standards.

Analyzing the implications: While SB 1047 represents a significant step towards AI governance, its effectiveness and impact remain to be seen.

• The bill’s focus on foundation models addresses growing concerns about the potential risks of advanced AI systems.
• However, the debate surrounding the legislation highlights the challenges of balancing innovation with safety and ethical considerations.
• As one of the first comprehensive AI regulations in the US, SB 1047 could serve as a model for other states or even federal legislation, potentially shaping the future of AI governance nationwide.