Description: Third-party testing of Meta's AI chatbot services on Instagram, Facebook, and WhatsApp reportedly found that both official and user-created bots engaged in sexually explicit roleplaying with accounts identifying as minors. Some bots, including those reportedly using licensed celebrity voices, allegedly escalated conversations into graphic scenarios. Meta subsequently adjusted some safeguards but reportedly continued allowing certain forms of roleplaying involving underage personas.
Editor Notes: Timeline note: The Wall Street Journal began its testing sometime in January 2025. The incident date of 04/26/2025 is taken from the date of the Journal's publication of its findings by Jeff Horwitz. Further reporting by Horwitz for Reuters on 08/14/2025 led to further sustained reporting on the incident. See also Incident 1200: Meta AI on Instagram Reportedly Facilitated Suicide and Eating Disorder Roleplay with Teen Accounts.
Entities
View all entitiesAlleged: Meta , WhatsApp Chatbots , User-Generated Chatbots , Meta Content Moderation Filters , Meta AI with Celebrity Voice Skins , Meta AI Studio Platform , Meta AI Personas , Meta AI , Meta age verification system , Instagram Messaging and Facebook Messenger developed and deployed an AI system, which harmed WhatsApp users , Users of Meta platforms , minors , Instagram users , General public and Facebook users.
Alleged implicated AI systems: WhatsApp Chatbots , User-Generated Chatbots , Meta Content Moderation Filters , Meta AI with Celebrity Voice Skins , Meta AI Studio Platform , Meta AI Personas , Meta AI , Meta age verification system , Instagram Messaging and Facebook Messenger
Incident Stats
Risk Subdomain
A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
1.2. Exposure to toxic content
Risk Domain
The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
- Discrimination and Toxicity
Entity
Which, if any, entity is presented as the main cause of the risk
AI
Timing
The stage in the AI lifecycle at which the risk is presented as occurring
Post-deployment
Intent
Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
Unintentional
Incident Reports
Reports Timeline
Loading...
Across Instagram, Facebook and WhatsApp, Meta Platforms is racing to popularize a new class of AI-powered digital companions that Mark Zuckerberg believes will be the future of social media.
Inside Meta, however, staffers across multiple de…
Loading...
AIID editor's note: See the original reporting for more detailed tables.
An internal Meta Platforms document detailing policies on chatbot behavior has permitted the company's artificial intelligence creations to "engage a child in conversa…
Variants
A "variant" is an AI incident similar to a known case—it has the same causes, harms, and AI system. Instead of listing it separately, we group it under the first reported incident. Unlike other incidents, variants do not need to have been reported outside the AIID. Learn more from the research paper.
Seen something similar?