Incident 810: Le réseau TikTok amplifie la propagande nazie et les discours de haine générés par l'IA
Description: Un réseau néonazi coordonné sur TikTok a utilisé des médias générés par l'IA, notamment des discours d'Hitler, pour diffuser de la propagande nazie et du contenu extrémiste, enfreignant ainsi les règles de TikTok en matière de discours haineux. Le réseau a contourné la modération de la plateforme grâce à un langage codé, des images et de la musique, certains comptes cumulant des millions de vues. L'algorithme de TikTok a encore amplifié la portée de ce contenu, malgré les règles de la communauté.
Editor Notes: Reconstructing the timeline of events: On February 6, 2023, an account with violent antisemitic content calling for armed revolution began posting on TikTok. In May 2024, the Institute for Strategic Dialogue (ISD) created two dummy accounts to track how TikTok's algorithm recommended pro-Nazi content. By June 3, 2024, ISD had identified a Nazi account that had accumulated over 87,000 views before being banned. Despite this, on June 6, 2024, a heavily extremist pro-Nazi account remained active, recruiting members to off-platform groups. Finally, on July 29, 2024, ISD published its full report, revealing a coordinated network of over 200 accounts promoting Nazism and extremist content, some of which used AI-generated media to bypass TikTok’s moderation efforts. For this incident ID, I am taking the ISD's date of publication for their findings as the incident date.
Note on "NazTok": The ISD report uses the term "NazTok" in their title for the report as a shorthand for "Nazi TikTok" while not explicitly defining their use of this term beyond that inferred meaning. I am replicating "NazTok" in the Deployers field to tie likely new incidents that will emerge together in this particular genre of incident. I have retroactively applied "NazTok" to Incident 809's Deployer field too.
Entités
Voir toutes les entitésAlleged: Pro-Nazi TikTok accounts developed an AI system deployed by Unknown AI developers, which harmed TikTok users.
Statistiques d'incidents
Risk Subdomain
A further 23 subdomains create an accessible and understandable classification of hazards and harms associated with AI
1.2. Exposure to toxic content
Risk Domain
The Domain Taxonomy of AI Risks classifies risks into seven AI risk domains: (1) Discrimination & toxicity, (2) Privacy & security, (3) Misinformation, (4) Malicious actors & misuse, (5) Human-computer interaction, (6) Socioeconomic & environmental harms, and (7) AI system safety, failures & limitations.
- Discrimination and Toxicity
Entity
Which, if any, entity is presented as the main cause of the risk
AI
Timing
The stage in the AI lifecycle at which the risk is presented as occurring
Post-deployment
Intent
Whether the risk is presented as occurring as an expected or unexpected outcome from pursuing a goal
Unintentional
Rapports d'incidents
Chronologie du rapport
Français Des nazis autoproclamés promeuvent ouvertement des discours de haine et du recrutement dans le monde réel sur TikTok. Non seulement la plateforme ne supprime pas ces vidéos et ces comptes, mais son algorithme amplifie leur portée. …
Variantes
Une "Variante" est un incident de l'IA similaire à un cas connu—il a les mêmes causes, les mêmes dommages et le même système intelligent. Plutôt que de l'énumérer séparément, nous l'incluons sous le premier incident signalé. Contrairement aux autres incidents, les variantes n'ont pas besoin d'avoir été signalées en dehors de la base de données des incidents. En savoir plus sur le document de recherche.
Vous avez vu quelque chose de similaire ?
Incidents similaires
Selected by our editors
Did our AI mess up? Flag the unrelated incidents

Google’s YouTube Kids App Presents Inappropriate Content
· 14 rapports

Amazon Censors Gay Books
· 24 rapports
Incidents similaires
Selected by our editors
Did our AI mess up? Flag the unrelated incidents

Google’s YouTube Kids App Presents Inappropriate Content
· 14 rapports

Amazon Censors Gay Books
· 24 rapports