California AI Startups Clash with Lawmakers Over Proposed Safety Rules

California startups and Y Combinator oppose proposed AI safety regulations, arguing they could stifle innovation and threaten the state’s tech economy.

Key stakeholders express concerns: A coalition of 140 machine-learning startups and venture capital firm Y Combinator have signed an open letter opposing California Senate Bill 1047, which would impose guardrails and transparency requirements on large AI models:

• The signatories argue that the proposed regulations are arbitrary, vague, and burdensome, potentially leading AI companies to leave California for less restrictive locales.
• They claim the bill’s language is too broad and could inadvertently apply to smaller startups, not just the intended large tech firms developing the most powerful AI models.

Provisions of the proposed legislation: SB 1047, introduced by state Senator Scott Wiener, targets AI models requiring over 1026 FLOPs of computing power or costing more than $100 million to develop:

• Companies operating these “covered” models would need to certify their safety before training and submit to annual re-certification by a new state board.
• The bill also mandates the creation of a public cloud cluster called “CalCompute” for AI safety research.

Proponents argue measured approach: Senator Wiener and other supporters contend that SB 1047 takes a balanced, light-touch approach by focusing only on the largest and most capable AI models:

• Recent amendments give state agencies flexibility to adjust the compute threshold and allow companies leeway in meeting their obligations.
• Wiener claims prominent AI pioneers like Geoffrey Hinton and Yoshua Bengio are on his side, recognizing the need for sensible safeguards against AI risks.

Fate of the bill remains uncertain: SB 1047 has passed initial votes in the California Senate and Assembly, but Governor Gavin Newsom’s signature is not guaranteed:

• Newsom has expressed reservations about over-regulating AI, fearing it could cause tech firms to flee the state.
• However, he also acknowledges the need for some form of AI oversight, noting that even the “godmothers and fathers” of the technology are calling for help in reining in their creations.

Broader implications for the AI industry: The debate over SB 1047 reflects the ongoing tension between fostering rapid AI progress and proactively managing the societal risks:

• Startups and investors worry that premature regulation could hamstring a transformative technology in its early stages, ceding U.S. leadership to countries with a more permissive approach.
• However, a growing chorus of experts argues that AI’s profound implications necessitate a precautionary stance, with appropriate oversight to ensure this powerful tool benefits humanity.
• As the world’s tech capital, California’s choices may set a influential precedent for how other jurisdictions balance AI innovation and safety moving forward.