Incident 54: Predictive Policing Biases of PredPol

Description: Predictive policing algorithms meant to aid law enforcement by predicting future crime show signs of biased output.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscover
Alleged: PredPol developed an AI system deployed by PredPol and Oakland Police Department, which harmed Oakland Residents.

Incident Stats

Incident ID
54
Report Count
17
Incident Date
2015-11-18
Editors
Sean McGregor

CSET Taxonomy Classifications

Taxonomy Details

Full Description

Predictive policing algorithms meant to aid law enforcement by predicting future crime show signs of biased output. PredPol, used by the Oakland (California) Police Department, and the Strategic Subject List, used by Chicago PD, were subjects of studies in 2015 and 2016 showing their bias against "low-income, minority neighborhoods." These neighborhoods would receive added attention from police departments expecting crimes to be more prevalent in the area. Notably, Oakland Police Department used 2010's record of drug crime as their baseline to train the system.

Short Description

Predictive policing algorithms meant to aid law enforcement by predicting future crime show signs of biased output.

Severity

Minor

Harm Distribution Basis

Race, National origin or immigrant status, Financial means

Harm Type

Harm to civil liberties

AI System Description

Predictive policing algorithms meant to aid police in predicting future crime.

System Developer

PredPol, Chicago Police Department

Sector of Deployment

Public administration and defence

Relevant AI functions

Cognition

AI Techniques

machine learning

AI Applications

Predictive policing

Named Entities

Oakland Police Department, Chicago Police Department, PredPol, Human Rights Data Analysis Group, Strategic Subject List

Technology Purveyor

PredPol, Chicago Police Department, Oakland Police Department

Beginning Date

2015-01-01T00:00:00.000Z

Ending Date

2017-01-01T00:00:00.000Z

Near Miss

Unclear/unknown

Intent

Unclear

Lives Lost

No

Laws Implicated

Fourth Amendment of the US Constitution

Data Inputs

Crime statistics

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents