Incident 51: Security Robot Rolls Over Child in Mall

Responded
Description: On July 7, 2016, a Knightscope K5 autonomous security robot collided with a 16-month old boy while patrolling the Stanford Shopping Center in Palo Alto, CA.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscover
Alleged: Knightscope developed an AI system deployed by Stanford Shopping Center, which harmed Child.

Incident Stats

Incident ID
51
Report Count
27
Incident Date
2016-07-12
Editors
Sean McGregor

CSET Taxonomy Classifications

Taxonomy Details

Full Description

On July 7, 2016, a Knightscope K5 autonomous security robot patrolling the Stanford Shopping Center in Palo Alto, CA collided with a 16-month old boy, leaving the boy with a scrape and minor swelling. The Knightscope K5 carries nearly 30 environment sensors including LIDAR, sonar, vibration detectors, and 360-degree HD video cameras. The company called this a “freakish accident” and apologized to the family.

Short Description

On July 7, 2016, a Knightscope K5 autonomous security robot collided with a 16-month old boy while patrolling the Stanford Shopping Center in Palo Alto, CA.

Severity

Minor

Harm Type

Harm to physical health/safety

AI System Description

Knightscope K5 autonomous security robot uses several environmental sensors and voice commands to conduct security operations.

System Developer

Knightscope

Sector of Deployment

Administrative and support service activities

Relevant AI functions

Perception, Cognition, Action

AI Techniques

machine learning

AI Applications

Image classification, image recognition, facial recognition, self-driving, environment sensing

Location

Palo Alto, CA

Named Entities

Knightscope, Knightscope K5, Stanford Shopping Center, Tiffany Teng, Harwin Cheng, William Santana Li

Technology Purveyor

Knightscope

Beginning Date

2016-07-07T07:00:00.000Z

Ending Date

2016-07-07T07:00:00.000Z

Near Miss

Harm caused

Intent

Accident

Lives Lost

No

Data Inputs

LIDAR, sonar, video camera, vibration detection, thermal anomaly detection, automatic signal detection, audio

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents