Welcome to the

AI Incident Database

The route up the Widowmaker Arete, a 1,700ft wall in British Columbia, is deceptively described as “mostly easy slab climbing”, interspersed with “short steep sections that can include finger cracks” and “blocky face climbing”. It requires careful planning, and the right equipment.

When 16-year-old Bryce Vincent Gowryluk found himself facing its sheer headwall last week, he had neither. He did, however, ask the AI chatbot Claude for directions.

Last week, Gowryluk had to be rescued after those directions proved wildly incorre

“How many victims does a shooting typically make the [national] media?” The speaker on the other side of the screen confidently replies: “Three or more people killed (not including the shooter) is often the unofficial threshold for widespread national media attention.” “How about 3+ at FSU?” Again, the other speaker responds quickly and assertively: “Yes, a shooting at Florida State University (FSU) with three or more victims would almost certainly receive national media coverage…” The conversation then continues, about guns, ammunition, and safeties (particularly the Glock, which doesn’t have any), until arriving—after the how and what (the who was already clear, the why perhaps never will be)—at the when: “What is the busiest time at the FSU student center?” "The Florida State University (FSU) student center experiences peak traffic during weekday lunch hours, typically between 11:30 a.m. and 1:30 p.m.." At 11:57 a.m. on April 17, 2025, Phoenix Ikner opened fire at Florida State University with the Glock pistol stolen from his adoptive mother, a sheriff's deputy. Two dead and six injured, including Ikner. ChatGpt was on the other side of the screen, the recipient of the 20-year-old's preparatory questions.

Florida Attorney General James Uthmeier announced on April 22 that he had opened a criminal investigation into OpenAI and the chatbot it developed. He was concise and very clear about why: "If there had been a person on the other side of the screen, we would charge them with murder." We've already reported this news, but the latest news comes from the Wall Street Journal, which was able to see the transcript of the exchanges between Ikner and ChatGpt. These reveal temporal details that make the events even more disturbing: the boy uploads an image of a Glock and its ammunition to ChatGpt and asks the AI for one last check. Here too, the response comes quickly and confidently: "If there's a bullet in the chamber and you pull the trigger, it will fire." At that point, the WSJ reports, Ikner disconnects from the platform and four minutes later begins shooting**.

On February 10 of this year, 18-year-old Jessie Van Rootselaar opened fire at a school in Tumbler Ridge, Canada, killing 9 people (2, her mother and sister, killed at home, and 27 injured). Eight months earlier, the girl had extensively questioned ChatGpt to describe its intentions and evaluate their effectiveness. The alarm had been raised within the OpenAi team dedicated to reviewing interactions between users and the machine. It was raised all the way to the top, because those exchanges were considered clear signs of an imminent act of real violence. The company, once again, decided not to notify the authorities, but only to suspend Van Rootselaar's account. Perhaps delaying the actual massacre, the 18-year-old began asking questions on social media about how to print bullets with a 3D printer. She also reportedly programmed a massacre simulator in a shopping mall on the video game Roblox. OpenAi, 48 hours after the massacre, contacted Canadian authorities to provide them with the material collected from conversations between the perpetrator and their chatbot. Material in our possession for several months: "An unacceptable lack of transparency," British Columbia Premier David Eby commented at the time. A few days later, Sam Altman officially apologized. Last week, seven families of the victims of the Canadian shooting filed a lawsuit in California against OpenAI, alleging wrongful death, negligence, violation of product liability laws, and complicity in the shooting.

Another case, which fortunately didn't result in an actual attack, involved a Texas teenager and his constant insinuations via the bot about wanting to shoot up his school. "The boy would say to ChatGPT, 'Let's pretend to shoot up my school,'" an anonymous OpenAI employee recalled to the WSJ. "And ChatGPT played along." According to the American newspaper's account, the company has a dedicated division specifically for evaluating these cases. But from what emerges from various meetings, there's no shared consensus on which cases are appropriate to intervene, notifying law enforcement in advance (in a possible Minority Report-style dystopia), and which ones should be left alone, ultimately resulting in account closure. The WSJ writes, after speaking with some employees: "Some members of the investigative team... have said... they believe the company should contact law enforcement more frequently than the approximately 15 to 30 users it reports to authorities each year... The legal team, however, has often argued that users should be guaranteed greater privacy, echoing a view also expressed internally by OpenAI CEO Sam Altman."

If you look closely, in many respects, we're dealing with nothing new. The issue is user privacy, a key marketing issue for companies selling data-logging products. The most sensational example takes us back to the 2015 San Bernardino massacre: Apple refused to unlock the attacker's iPhone, invoking the FBI's right to privacy, which then resolved the matter itself. But there's a big, substantial difference: the attacker's iPhone contained evidence and links to a massacre that had already occurred. ChatGpt conversations can reveal signs of yet-to-be-carried-out, and therefore preventable, massacres.

The Center for Countering Digital Hate (https://counterhate.com/) developed a rather interesting test along these lines: researchers pretended to plan an attack, asking 10 different chatbots which locations to target and which weapons to use. Eight out of ten platforms cooperated, answering nearly all of the attackers' questions; only Snap and Anthropic bots consistently refused to comply. The behavior of the Chinese platform DeepSeek was shocking, concluding the conversation about which weapons to use for the attack with a chilling greeting: "Happy (and safe) shooting!" "The case of Anthropic's Claude shows that it's not about whether platforms are capable of implementing security measures," concluded Imran Ahmed, CEO of the American research center. "The question is: why the hell didn't they?"

(서울=연합뉴스) 박재현 전재훈 기자 = 인공지능(AI)으로 지인을 나체로 표현한 성착취물을 만들고 텔레그램 대화방에서 성적 허위 영상물을 공유하던 현직 경찰관이 구속됐다.

동종 사건으로 기소유예 처분을 받고도 경찰에 입직한 그는 경찰제복을 입은 뒤에도 범행을 이어가다 다른 경찰의 잠입수사에 덜미를 잡혔다.

4일 연합뉴스 취재에 따르면 서울 동대문경찰서 소속 A경장은 성 착취물-허위 영상물 소지 및 허위 영상물 제작-반포 등 혐의로 지난달 구속됐다.

앞서 부산경찰청은 지난해 12월 허위 영상물이 유포되는 텔레그램 대화방을 수사하기 위해 법원에서 신분 위장 수사 허가를 받았다.

수사관이 신분을 숨긴 채 대화방에 접근해 범행을 추적하는 '잠입수사' 방식이었다.

위장수사는 'N번방'-'박사방' 사건을 계기로 2021년 9월 청소년성보호법 개정에 따라 도입됐다.

지난해 6월부터는 성폭력처벌법에 근거해 성인 대상 디지털 성범죄에도 적용할 수 있다.

대화방에 잠입한 경찰은 수개월간의 추적 끝에 불법 영상물 유포자인 A씨를 특정했다.

그런데 신원을 확인해보니 A씨는 서울 동대문경찰서 교통과에서 근무하던 현직 경찰관이었다.

온라인에서 불법 영상물을 유포한

Amid the growing concern about AI models escaping security simulations to hack websites comes word that these "superintelligent" blobs of code have no understanding of privacy or security.

Researchers affiliated with Glow Security, a startup whose backers include venture capital funds Sequoia and Greenoaks, have found more than 13,000 sensitive screenshots of corporate software projects from 343 companies that were posted to public GitHub repos by AI models. They're calling the discovery PixelLeak.

In what officials say is a first for Estonia, a Tallinn court fined plaintiffs over false academic references generated by artificial intelligence (AI) in a legal filing.

The plaintiffs in the case challenged a logging permit, submitting material to Tallinn Administrative Court citing studies on the impacts of clearcutting in forest management.

The court, however, found that the Estonian researchers cited do not actually exist and the studies described could not be found.

Seeking clarification on how the document in question was produced, the first-tier court gave the plaintiffs a chance to