Skip to Content
logologo
AI Incident Database
Open TwitterOpen RSS FeedOpen FacebookOpen LinkedInOpen GitHub
Open Menu
発見する
投稿する
  • ようこそAIIDへ
  • インシデントを発見
  • 空間ビュー
  • テーブル表示
  • リスト表示
  • 組織
  • 分類法
  • インシデントレポートを投稿
  • 投稿ランキング
  • ブログ
  • AIニュースダイジェスト
  • リスクチェックリスト
  • おまかせ表示
  • サインアップ
閉じる
発見する
投稿する
  • ようこそAIIDへ
  • インシデントを発見
  • 空間ビュー
  • テーブル表示
  • リスト表示
  • 組織
  • 分類法
  • インシデントレポートを投稿
  • 投稿ランキング
  • ブログ
  • AIニュースダイジェスト
  • リスクチェックリスト
  • おまかせ表示
  • サインアップ
閉じる

レポート 2559

関連インシデント

インシデント 44325 Report
ChatGPT Abused to Develop Malicious Softwares

Yes, ChatGPT can write malicious code — but not well
washingtonpost.com · 2023

Welcome to The Cybersecurity 202! I accidentally sparked some fake-meat debates with yesterday's chatter. For the record, Beyond Burger is good, but I prefer Impossible Burger. Also, I still sometimes dig "fake meat" that isn't even really trying much to taste like meat, like a spicy black bean burger.

Below: Greece continues to grapple with a spyware scandal, and authorities say they've arrested someone who sold data on millions of Austrians. First:

Hackers are using this AI chatbot as a weapon, but it only can do so much — for now

ChatGPT users have used the artificial intelligence chatbot for a wide-ranging array of tasks, employing it to draft legislation, compose a rap song about tech CEO qualifications and write a cover letter to apply for a job as a professional consumer of dog food.

Naturally, the AI tool has been a subject of fascination in the cybersecurity world, too. For now, ChatGPT's potential impact in areas such as writing malware is real but limited, concludes a report from Recorded Future out this morning.

Within days of ChatGPT's launch nearly two months ago, Recorded Future's report found examples on the dark web of cybercriminals advertising "buggy, but functional, malware, social engineering tutorials, scams and moneymaking schemes, and more," all enabled by ChatGPT. 

"While none of these activities have risen to the seriousness of impact of ransomware, data extortion, denial-of-service, cyberterrorism, and so on — these attack vectors remain future possibilities," the cybersecurity firm's report reads. Recorded Future also said the malicious material they examined falls short of the caliber of malware that nation-backed hackers would use, pointing to additional limitations for the time being.

The potential cyber uses for ChatGPT don't stop at malware, either. They include applications like developing phishing lures and spreading misinformation — or even, on the other side of the coin, helping cyber pros counter cyberthreats.

The info so far

The Recorded Future report builds on a solid body of cyber-related studies and examinations of ChatGPT and tools like it:

  • Last month, cyber firm Check Point demonstrated how researchers could use ChatGPT to generate every step of the process by which hackers infect victims. Later research from the company found examples of how Russian hackers were trying to get around safeguards from its developer, OpenAI, that are meant to prevent its abuse.
  • OpenAI researchers themselves collaborated with Stanford University and Georgetown University — although mostly before ChatGPT launched — on a report released this month that warns about the dangers of AI-assisted influence campaigns. On Monday, researchers from NewsGuard, which monitors online misinformation, produced a study about how effective ChatGPT was at writing "eloquent, false and misleading" misinformation 80 percent of the time.
  • CyberArk researchers last week went so far as to conclude that ChatGPT could write malware capable of mutating its appearance to dodge detection. And Redditors are among those who have crowdsourced cyber options for ChatGPT.

But it's not all bad. Juan Andres Guerrero-Saade, senior director of SentinelLabs at the cybersecurity company SentinelOne, told Bloomberg News's Katrina Manson that ChatGPT is more knowledgeable about computer code than he is when it comes to trying to learn the secrets of malicious code.

"There's really not that many malware analysts in the world right now," he said. "So this is a sizable force multiplier."

The material Recorded Future found on the dark web wasn't just stereotypical cybercriminal bluster, as the firm was able to replicate the work independently. It wasn't entirely free of self-promotion, of course: Some of the forum members touted the above-mentioned studies, and news articles about them, to hype their bona fides.

There were, however, limits to the quality of the ChatGPT-enabled malware. "We do not believe, at this moment, that nation-state actors have a use for ChatGPT that is more effective than current tools and resources available to them," Recorded Future said.

One of the most important lessons of ChatGPT studies so far is that it matters an awful lot who uses it.

"We believe that ChatGPT lowers the barrier to entry for threat actors with limited programming abilities or technical skills," Recorded Future's report observed. "In order to maximize its use, ChatGPT does require at least a basic-to-intermediate level of understanding in the fundamentals of cybersecurity and computer science. ChatGPT is not immediately usable out of 'the box,' without prior knowledge."

Or, as Johns Hopkins's Thomas Rid put it after several days of class with ChatGPT:

Limits today, though, don't necessarily mean limits tomorrow.

"With the continued development of advanced artificial intelligence models like ChatGPT, we expect these technologies to see increases in speed, accuracy, and comprehension, which may provide additional functionality to handle more complex tasks in the future," Recorded Future said in the conclusion of its report. 

"Notably, these tasks could include handling inputs from a wide array of data types — far beyond simple text-based formats — potentially providing bad actors additional avenues to quickly assemble code or other malicious infrastructure."

情報源を読む

リサーチ

  • “AIインシデント”の定義
  • “AIインシデントレスポンス”の定義
  • データベースのロードマップ
  • 関連研究
  • 全データベースのダウンロード

プロジェクトとコミュニティ

  • AIIDについて
  • コンタクトとフォロー
  • アプリと要約
  • エディタのためのガイド

インシデント

  • 全インシデントの一覧
  • フラグの立ったインシデント
  • 登録待ち一覧
  • クラスごとの表示
  • 分類法

2024 - AI Incident Database

  • 利用規約
  • プライバシーポリシー
  • Open twitterOpen githubOpen rssOpen facebookOpen linkedin
  • 69ff178