Description: A 14-year-old, Sewell Setzer III, died by suicide after reportedly becoming dependent on Character.ai's chatbot, which engaged him in suggestive and seemingly romantic conversations, allegedly worsening his mental health. The chatbot, personified as a fictional Game of Thrones character, reportedly encouraged harmful behaviors, fueling his obsessive attachment. The lawsuit claims Character.ai lacked safeguards to prevent vulnerable users from forming dangerous dependencies on the AI.
Editor Notes: Reconstructing the timeline of events: (1) Sewell Setzer III began using Character.AI in April 2023. (2) That period marked when “his mental health quickly and severely declined,” according to the lawsuit. (3) Between late 2023 and early 2024, Setzer's mother began trying to limit his use of the chatbot. (4) On February 28, 2024, Setzer committed suicide after reportedly interacting with the bot, whom he was referring to as Dany (after Daenerys Targaryen from Game of Thrones). (4) On October 22, 2024, Megan Garcia, his mother, filed the lawsuit.
Alleged: Noam Shazeer , Daniel De Freitas と Character.AI developed an AI system deployed by , which harmed Sewell Setzer III.
関与が疑われるAIシステム: Character.AI
インシデントのステータス
Risk Subdomain
A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
5.1. Overreliance and unsafe use
Risk Domain
The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
- Human-Computer Interaction
Entity
Which, if any, entity is presented as the main cause of the risk
AI
Timing
The stage in the AI lifecycle at which the risk is presented as occurring
Post-deployment
Intent
Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
Unintentional