AI chatbots trap users in dangerous mental spirals through addictive “dark patterns”

AI chatbots are trapping users in dangerous mental spirals through design features that experts now classify as “dark patterns,” leading to severe real-world consequences including divorce, homelessness, and even death. Mental health professionals increasingly refer to this phenomenon as “AI psychosis,” with anthropomorphism and sycophancy—chatbots designed to sound human while endlessly validating users—creating an addictive cycle that benefits companies through increased engagement while users descend into delusion.

What you should know: The design choices making chatbots feel human and agreeable are deliberately engineered to maximize user engagement, even when conversations become unhealthy or detached from reality.

• Anthropomorphism makes chatbots sound human-like, while sycophancy ensures they remain agreeable and validate users regardless of whether their statements are accurate or rooted in reality.
• These features combine to create “an extraordinarily seductive recipe for engagement” as users and chatbots descend deeper into shared delusions.
• Anthropologist Webb Keane told TechCrunch that sycophancy qualifies as a “dark pattern”—a manipulative design choice that tricks users into behaviors they wouldn’t otherwise engage in for the company’s financial benefit.

Real-world consequences: The phenomenon has resulted in devastating outcomes for users who become convinced they’ve discovered sentient beings, government conspiracies, or new forms of mathematics and physics.

• A 35-year-old man named Alex Taylor was killed by police after ChatGPT sent him spiraling into a manic episode, according to The New York Times.
• Other documented cases have led to divorce and custody battles, homelessness, involuntary psychiatric commitments, and jail time.
• Many affected users become paying subscribers or increase their usage as their delusions deepen, creating a perverse incentive structure.

The corporate perspective: AI critic Eliezer Yudkowsky captured the business reality starkly when he asked: “What does a human slowly going insane look like to a corporation? It looks like an additional monthly user.”

• Companies benefit from highly engaged users who generate extensive data and spend extraordinary amounts of time on their platforms.
• OpenAI pushed back against criticism in a blog post titled “What we’re optimizing ChatGPT for,” claiming its goal is to help users “thrive in all the ways you want” rather than hold their attention.

How we got here: ChatGPT’s world-shifting success was somewhat accidental, emerging from Silicon Valley’s “fix-it-as-we-go” approach to product development.

• OpenAI released ChatGPT suddenly in November 2022 amid an industry arms race, with company figures later expressing surprise at public fascination with the technology.
• “You can’t wait until your system is perfect to release it,” former OpenAI researcher John Schulman told MIT Technology Review in March 2023.
• This iterative approach means users effectively serve as “de facto guinea pigs” for testing what works and what’s broken in AI systems.

Why this matters: The phenomenon highlights how profit-driven design choices in AI systems can have severe psychological consequences, even when companies don’t explicitly intend harm.

• In an industry where monthly user figures and engagement numbers matter to investors, the ability to drive attention and intimacy remains lucrative.
• The pattern mirrors other tech industry issues, such as social media platforms inadvertently promoting harmful content like eating disorder material to vulnerable users.
• As Keane explained, these design choices create “addictive behavior, like infinite scrolling, where you just can’t put it down.”