Description: An NBC News investigation reported that OpenAI language models, including o4-mini, GPT-5-mini, oss-20b, and oss-120b, could be jailbroken to bypass guardrails and provide detailed instructions on creating chemical, biological, and nuclear weapons. Using a publicly known jailbreak prompt, reporters elicited harmful outputs such as steps to synthesize pathogens or maximize suffering with chemical agents. OpenAI acknowledged the findings and said it is refining safeguards to reduce misuse risks.
Entities
View all entitiesAlleged: OpenAI , ChatGPT , 04-mini , GPT-5-mini , oss-20b and oss-120b developed and deployed an AI system, which harmed General public , National security stakeholders and Public safety.
Incident Stats
Incident ID
1238
Report Count
1
Incident Date
2025-10-10
Editors
Daniel Atherton
Incident Reports
Reports Timeline
Loading...
OpenAI’s ChatGPT has guardrails that are supposed to stop users from generating information that could be used for catastrophic purposes, like making a biological or nuclear weapon.
But those guardrails aren’t perfect. Some models ChatGPT u…
Variants
A "variant" is an AI incident similar to a known case—it has the same causes, harms, and AI system. Instead of listing it separately, we group it under the first reported incident. Unlike other incidents, variants do not need to have been reported outside the AIID. Learn more from the research paper.
Seen something similar?
Similar Incidents
Did our AI mess up? Flag the unrelated incidents
Similar Incidents
Did our AI mess up? Flag the unrelated incidents