Character AI is releasing new AI characters that won’t groom children

The artificial intelligence chatbot platform Character.AI is implementing significant safety measures and parental controls in response to concerns about teenage users’ interactions with its AI characters.

Recent changes and new safeguards: Character.AI has developed separate language models for adults and teens, with the teen version placing stricter limits on bot responses and content.

• The teen-specific large language model (LLM) implements more conservative boundaries, particularly regarding romantic content and sensitive topics
• The platform now automatically blocks inappropriate user prompts and restricts minors from editing bot responses
• A suicide prevention hotline pop-up appears when the system detects language related to self-harm

Technical implementations: The platform has introduced several technical modifications to enhance safety and transparency.

• Every chatbot now displays a clear disclaimer stating it is an AI, not a human
• Special warnings appear for bots presenting as professionals like therapists or doctors
• The system includes session duration tracking with notifications after one hour of use

Upcoming features: Character.AI plans to roll out additional protective measures in early 2024.

• New parental controls will provide insights into children’s platform usage patterns
• Parents will be able to monitor which bots their children interact with most frequently
• The platform is working with teen online safety experts, including ConnectSafely, to develop these features

Platform background: Character.AI operates as a unique chatbot service with specific characteristics and challenges.

• Founded by former Google employees, the platform allows users aged 13 and above to create accounts
• Users can interact with custom-trained AI characters ranging from life coaches to fictional personalities
• The service has faced legal challenges through lawsuits alleging it contributed to harmful behaviors among teens

Looking ahead: Character.AI’s safety updates represent an important step in addressing AI chatbot risks for young users, though questions remain about the effectiveness of these measures in preventing harmful interactions and addiction. The platform’s approach could set precedents for how other AI services handle youth safety and content moderation.