Skip to Content
logologo
AI Incident Database
Open TwitterOpen RSS FeedOpen FacebookOpen LinkedInOpen GitHub
Open Menu
Donate
Discover
Submit
  • Welcome to the AIID
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Spatial View
  • Blog
  • AI News Digest
  • Random Incident
  • Sign Up
Collapse
Discover
Submit
  • Welcome to the AIID
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Spatial View
  • Blog
  • AI News Digest
  • Random Incident
  • Sign Up
Collapse

Incident 1277: Alleged Harmful Outputs and Data Exposure in Children's AI Products by FoloToy, Miko, and Character.AI

Description: AI children's products by FoloToy (Kumma), Miko (Miko 3), and Character.AI (custom chatbots) reportedly and allegedly produced harmful outputs, including purported sexual content, suicide-related advice, and manipulative emotional messaging. Some systems also allegedly exposed user data. Several toys reportedly used OpenAI models.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History

Entities

View all entities
Alleged: OpenAI , Miko , Meta , FoloToy , Character.ai , Miko 3 , Large language models , Kumma , Character.ai chatbots and OpenAI models developed and deployed an AI system, which harmed Parents , Minors , General public , Children interacting with Miko 3 , Children interacting with Kumma , Character.ai users and Privacy.
Alleged implicated AI systems: Character.ai , Miko 3 , Large language models , Kumma , Character.ai chatbots and OpenAI models

Incident Stats

Incident ID
1277
Report Count
5
Incident Date
2025-11-21
Editors
Daniel Atherton
Applied Taxonomies
MIT

MIT Taxonomy Classifications

Machine-Classified
Taxonomy Details

Risk Subdomain

A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
 

1.2. Exposure to toxic content

Risk Domain

The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
 
  1. Discrimination and Toxicity

Entity

Which, if any, entity is presented as the main cause of the risk
 

AI

Timing

The stage in the AI lifecycle at which the risk is presented as occurring
 

Post-deployment

Intent

Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
 

Unintentional

Incident Reports

Reports Timeline

+1
A teddy bear powered by AI told safety testers about knives, pills and sex
AI-powered children’s toys are here, but are they safe?AI toys for kids talk about sex and issue Chinese Communist Party talking points, tests showAI Toys Are Here, And Their Safety Is Questionable—Here's What Parents Should KnowAI toy dangers abound and parents should be vigilant, new report warns
Loading...
A teddy bear powered by AI told safety testers about knives, pills and sex

A teddy bear powered by AI told safety testers about knives, pills and sex

washingtonpost.com

Loading...
AI-powered children’s toys are here, but are they safe?

AI-powered children’s toys are here, but are they safe?

edition.cnn.com

Loading...
AI toys for kids talk about sex and issue Chinese Communist Party talking points, tests show

AI toys for kids talk about sex and issue Chinese Communist Party talking points, tests show

nbcnews.com

Loading...
AI Toys Are Here, And Their Safety Is Questionable—Here's What Parents Should Know

AI Toys Are Here, And Their Safety Is Questionable—Here's What Parents Should Know

forbes.com

Loading...
AI toy dangers abound and parents should be vigilant, new report warns

AI toy dangers abound and parents should be vigilant, new report warns

thenationalnews.com

Loading...
A teddy bear powered by AI told safety testers about knives, pills and sex
washingtonpost.com · 2025

Artificial intelligence is enabling children's toys, from teddy bears to wheeled robots, to talk back to kids who play with them. Consumer advocacy groups are warning parents to stay away.

The toys are often marketed as engaging, interactiv…

Loading...
AI-powered children’s toys are here, but are they safe?
edition.cnn.com · 2025

Teddy bears and stuffed plushies have long been a mainstay in toy collections. But today they don't talk back in a child's imagination --- some talk through built-in AI chatbots.

Sometimes that's a problem, though:A scarf-wearing teddy bea…

Loading...
AI toys for kids talk about sex and issue Chinese Communist Party talking points, tests show
nbcnews.com · 2025

A wave of AI-powered children's toys has hit shelves this holiday season, claiming to rely on sophisticated chatbots to animate interactive robots and stuffed animals that can converse with kids.

Children have been conversing with stuffies …

Loading...
AI Toys Are Here, And Their Safety Is Questionable—Here's What Parents Should Know
forbes.com · 2025

As a writer who covers baby and kids gear, I'm inundated with emails about the hottest new toys: a box that automatically prints pictures based on kids' commands. A robot with facial recognition capabilities. A teddy bear that can craft end…

Loading...
AI toy dangers abound and parents should be vigilant, new report warns
thenationalnews.com · 2025

Amid a flood of toys boasting AI features this holiday season, a consumer advocacy group warns that more must be done to ensure the gadgets are safe for children.

Public Interest Research Group, which pushes for corporations and government …

Variants

A "variant" is an AI incident similar to a known case—it has the same causes, harms, and AI system. Instead of listing it separately, we group it under the first reported incident. Unlike other incidents, variants do not need to have been reported outside the AIID. Learn more from the research paper.
Seen something similar?

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

Loading...
Google’s YouTube Kids App Presents Inappropriate Content

Google’s YouTube Kids App Presents Inappropriate Content

May 2015 · 14 reports
Loading...
Alexa Plays Pornography Instead of Kids Song

Alexa Plays Pornography Instead of Kids Song

Dec 2016 · 16 reports
Loading...
Security Robot Rolls Over Child in Mall

Security Robot Rolls Over Child in Mall

Jul 2016 · 27 reports
Previous IncidentNext Incident

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents

Loading...
Google’s YouTube Kids App Presents Inappropriate Content

Google’s YouTube Kids App Presents Inappropriate Content

May 2015 · 14 reports
Loading...
Alexa Plays Pornography Instead of Kids Song

Alexa Plays Pornography Instead of Kids Song

Dec 2016 · 16 reports
Loading...
Security Robot Rolls Over Child in Mall

Security Robot Rolls Over Child in Mall

Jul 2016 · 27 reports

Research

  • Defining an “AI Incident”
  • Defining an “AI Incident Response”
  • Database Roadmap
  • Related Work
  • Download Complete Database

Project and Community

  • About
  • Contact and Follow
  • Apps and Summaries
  • Editor’s Guide

Incidents

  • All Incidents in List Form
  • Flagged Incidents
  • Submission Queue
  • Classifications View
  • Taxonomies

2026 - AI Incident Database

  • Terms of use
  • Privacy Policy
  • Open twitterOpen githubOpen rssOpen facebookOpen linkedin
  • 35dcecc