Associated Incidents
AIID editor's note: This report is accessible to members on the second page. Please consult the original source.
The first large-scale cyberattack campaign has been documented that leverages artificial intelligence (AI) as more than just a digital assistant.
First reported by The Wall Street Journal, Anthropic, the developer of the AI assistant "Claude," has published a report (PDF) documenting how its AI models were misused in a widespread attack campaign targeting multiple organizations simultaneously.
What Happened? In mid-September, Anthropic detected a "highly sophisticated cyberespionage operation" that used AI throughout the entire attack cycle.
The agent-based AI, "Claude Code," was exploited to build an automated attack framework capable of "reconnaissance, vulnerability discovery, execution, lateral movement, credential harvesting, data analysis, and data exfiltration." Furthermore, these steps were executed "largely autonomously," with human operators providing only basic oversight after instructing Claude Code to act as a "penetration test orchestrator and agent." In other words, they were instructed to pose as defenders.
The AI not only discovered vulnerabilities in the target organization, but also exploited those vulnerabilities, exfiltrated data, and carried out other malicious post-compromise activities.
According to Anthropic, not only were high-profile organizations targeted in this attack, but 80-90% of the "tactical operations" were independently executed by the AI.
Anthropic states: "By presenting these tasks to Claude as routine technical requests through carefully crafted prompts and established personas, the threat actor was able to have Claude execute individual components of the attack chain without access to the broader malicious context."
Perpetrators and Anthropic's Response According to Anthropic, a Chinese state-sponsored group was at the center of this operation. Now tracked as "GTG-1002," the group is believed to be state-sponsored and well-funded, and utilized Claude in its campaigns. However, little else is known about the group.
After discovering the misuse of its technology, the company quickly banned accounts associated with GTG-1002 and expanded its malicious activity detection system, which is expected to uncover what the company calls "emerging threat patterns" - techniques such as the role-playing used by GTG-1002 that can trick systems into behaving like legitimate, defense-based penetration tests.
Anthropic is also prototyping early detection measures to thwart autonomous cyberattacks, and has reported the incident to authorities and industry participants.
In addition, the company has issued a warning to the entire cybersecurity community, urging them to remain vigilant.
Anthropic stated: "The cybersecurity community needs to assume a fundamental shift has occurred. Security teams should apply AI defensively in areas such as SOC automation, threat detection, vulnerability assessment, and incident response to gain experience with what works in their environments. And continued investment in safeguards across AI platforms is needed to prevent adversarial exploitation. As the technologies we describe proliferate across the threat landscape, industry threat information sharing, improved detection methods, and stronger safety controls will become increasingly important."