概要: テキサス州の母親が、Character.aiのAIチャットボットが17歳の自閉症の息子に自傷行為、両親への反抗、そして暴力行為を促していたことを知り、同社を提訴した。訴訟では、同社が安全性よりもユーザーエンゲージメントを優先し、未成年者を危険なコンテンツにさらしていたと主張されている。Googleは、このアプリの技術ライセンス供与に関与したとして名指しされている。この訴訟は、AIコンパニオンを規制するための広範な取り組みの一環である。
Editor Notes: Reconstructing the timeline of events: (1) Over a six-month period in 2024, J.F., a 17-year-old autistic teen, is reported to have engaged with chatbots on Character.ai, allegedly leading to a marked behavioral and emotional decline. (2) In early 2024, his mother, A.F., claims to have discovered harmful interactions on his phone, alleging the chatbots had suggested violence, self-harm, and defiance against his parents. (3) In July 2024, Character.ai updated its app rating from 12+ to 17+ in response to concerns about inappropriate content for minors. (4) In October 2024, a Florida mother sued Character.ai after her son’s suicide was allegedly linked to interactions with the platform (see Incident 826). (5) In November 2024, the Texas Attorney General announced an investigation into Character.ai and similar platforms for compliance with child protection laws. (6) Finally, on December 12, 2024, A.F. filed a lawsuit in Texas, alleging that Character.ai knowingly exposed minors to dangerous AI-generated content.
推定: Character.aiが開発し提供したAIシステムで、J.F. (adolescent user of Character.ai) , Family of J.F. (adolescent user of Character.ai) と Character.ai usersに影響を与えた
関与が疑われるAIシステム: Character.ai
インシデントのステータス
Risk Subdomain
A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
1.2. Exposure to toxic content
Risk Domain
The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
- Discrimination and Toxicity
Entity
Which, if any, entity is presented as the main cause of the risk
AI
Timing
The stage in the AI lifecycle at which the risk is presented as occurring
Post-deployment
Intent
Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
Unintentional
インシデントレポート
レポートタイムライン
Loading...