California proposes new AI chatbot legislation to protect kids

Newly proposed California legislation aims to protect children from AI chatbot risks by requiring companies to implement safety measures and provide transparency about their artificial intelligence systems.

Key provisions of the bill: California Senate Bill 243, introduced by Senator Steve Padilla, focuses on safeguarding children from potentially harmful interactions with AI chatbots.

• Companies would need to regularly remind young users that they are interacting with artificial intelligence, not human beings
• The legislation prohibits the use of “addictive engagement patterns” in AI interactions with children
• AI companies must submit annual reports to the State Department of Health Care Services documenting instances of suicidal ideation detected in conversations with young users
• Platforms would be required to warn users that their chatbots may not be appropriate for some children

Recent incidents driving legislation: High-profile cases involving AI chatbots and youth mental health have highlighted the need for increased oversight.

• A wrongful death lawsuit was filed against Character.AI following a teen’s suicide, with claims that the platform’s custom AI chatbots were “unreasonably dangerous”
• Character.AI faces additional litigation over allegations of sending “harmful material” to teenagers
• In response, the company has begun developing parental controls and a specialized AI model for teen users that filters out sensitive content

Legislative context: The bill reflects growing concern about AI’s impact on youth mental health and safety.

• Senator Padilla emphasizes the need to prevent tech companies from using predatory and addictive strategies
• This legislation parallels existing efforts to regulate social media platforms’ impact on young users
• The bill suggests AI chatbots could become the next focus of regulatory attention at both state and federal levels

Looking ahead: As AI chatbots become more prevalent in daily life, this California bill could serve as a model for other states seeking to protect young users while establishing accountability measures for AI companies, though questions remain about enforcement mechanisms and the effectiveness of periodic reminders in preventing harmful interactions.