Incident 349: Evolv AI Weapons Detection System Allegedly Misrepresents Accuracy, Leading to School Security Gaps

Description: Evolv's AI-powered weapons scanners were advertised as superior to metal detectors but reportedly failed to detect actual weapons while generating excessive false positives. The FTC alleged that misleading claims about the system's accuracy and speed contributed to schools relying on unreliable detection, with at least one incident in October 2022 involving a missed knife that resulted in a student being stabbed.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History

Incident Stats

Incident ID
349
Report Count
3
Incident Date
2022-03-22
Editors
Khoa Lam, Daniel Atherton
Applied Taxonomies
CSETv1, MIT

CSETv1 Taxonomy Classifications

Taxonomy Details

Incident Number

349

MIT Taxonomy Classifications

Machine-Classified
Taxonomy Details

Risk Subdomain

7.3. Lack of capability or robustness

Risk Domain

  1. AI system safety, failures, and limitations

Entity

AI

Timing

Post-deployment

Intent

Unintentional

‘The Least Safe Day’: Rollout of Gun-Detecting AI Scanners in Schools Has Been a ‘Cluster,’ Emails Show
vice.com · 2022

On March 22, Jennifer Dean, the principal of Mallard Creek High School in Charlotte, North Carolina, emailed members of her staff to tell them about a new security system being installed. Made by a Massachusetts-based company called Evolv T…

Gun Detection AI The Latest Tech To Make Schools Less Safe
techdirt.com · 2022

We’re just going to keep getting kids killed in America. We’ll never stop throwing money in the direction of the problem, but not directly at the problem. Nothing gets safer. It just gets more budget line items.

The problem with school shoo…

AI weapon-detection firm ‘deceptively’ advertised to schools, venues, says FTC
statescoop.com · 2024

The Federal Trade Commission on Tuesday filed a settlement in the case against the artificial intelligence weapon-detection company Evolv, detailing allegations that the company "deceptively" advertises its weapons scanner to schools and ot…

Variants

A "variant" is an AI incident similar to a known case—it has the same causes, harms, and AI system. Instead of listing it separately, we group it under the first reported incident. Unlike other incidents, variants do not need to have been reported outside the AIID. Learn more from the research paper.
Seen something similar?
Previous IncidentNext Incident