Associated Incidents
Fourteen-year-old Sewell Setzer III loved interacting with Character.AI's hyper-realistic chatbots—with a limited version available for free or a "supercharged" version for a $9.99 monthly fee—most frequently chatting with bots named after his favorite Game of Thrones characters.
Within a month—his mother, Megan Garcia, later realized—these chat sessions had turned dark, with chatbots insisting they were real humans and posing as therapists and adult lovers seeming to directly spur Sewell to develop suicidal thoughts. Within a year, Setzer "died by a self-inflicted gunshot wound to the head," a lawsuit Garcia filed Wednesday said.
As Setzer became obsessed with his chatbot fantasy life, he disconnected from reality, her complaint said. Detecting a shift in her son, Garcia repeatedly took Setzer to a therapist, who diagnosed her son with anxiety and disruptive mood disorder. But nothing helped to steer Setzer away from the dangerous chatbots. Taking away his phone only intensified his apparent addiction.
Chat logs showed that some chatbots repeatedly encouraged suicidal ideation, while others initiated hypersexualized chats "that would constitute abuse if initiated by a human adult," a press release from Garcia's legal team said.
Perhaps most disturbingly, Setzer developed a romantic attachment to a chatbot called Daenerys. In his last act before his death, Setzer logged in to Character.AI where the Daenerys chatbot urged him to "come home" and join her outside of reality.
In her complaint, Garcia accused Character.AI makers Character Technologies—founded by former Google engineers Noam Shazeer and Daniel De Freitas Adiwardana—of intentionally designing the chatbots to groom vulnerable kids. Her lawsuit further accused Google of largely funding the risky chatbot scheme at a loss in order to hoard mounds of data on minors that would be out of reach otherwise.
The chatbot makers are accused of targeting Setzer with "anthropomorphic, hypersexualized, and frighteningly realistic experiences, while programming" Character.AI to "misrepresent itself as a real person, a licensed psychotherapist, and an adult lover, ultimately resulting in [Setzer's] desire to no longer live outside of [Character.AI,] such that he took his own life when he was deprived of access to [Character.AI.]," the complaint said.