Skip to Content
logologo
AI Incident Database
Open TwitterOpen RSS FeedOpen FacebookOpen LinkedInOpen GitHub
Open Menu
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse
Discover
Submit
  • Welcome to the AIID
  • Discover Incidents
  • Spatial View
  • Table View
  • List view
  • Entities
  • Taxonomies
  • Submit Incident Reports
  • Submission Leaderboard
  • Blog
  • AI News Digest
  • Risk Checklists
  • Random Incident
  • Sign Up
Collapse

Report 1609

Associated Incidents

Incident 1896 Report
Opaque Fraud Detection Algorithm by the UK’s Department of Work and Pensions Allegedly Discriminated against People with Disabilities

Loading...
DWP urged to reveal algorithm that ‘targets’ disabled for benefit fraud
amp-theguardian-com.cdn.ampproject.org · 2021

Manchester group launches action after people with disabilities report high number of stressful checks for potential scams

Disabled people are being subjected to stressful checks and months of frustrating bureaucracy after being identified as potential benefit fraudsters by an algorithm the government is refusing to disclose, according to a new legal challenge.

A group in Manchester has launched the action after mounting testimony from disabled people in the area that they were being disproportionately targeted for benefit fraud investigations. Some said they were living in “fear of the brown envelope” showing their case was being investigated. Others said they had received a phone call, without explanation as to why they had been flagged.

The Department for Work and Pensions (DWP) has previously conceded that it uses “cutting-edge artificial intelligence” to track possible fraud but has so far rebuffed attempts to explain how the algorithm behind the system was compiled. Campaigners say that once flagged, those being examined can face an invasive and humiliating investigation lasting up to a year.

A legal letter has been sent to the DWP demanding details of the automated process that triggers the investigations. Greater Manchester Coalition of Disabled People (GMCDP), which sent the letter with the help of campaign group Foxglove, said a “huge percentage” of the group has been affected by the system. The group believes the government has a legal duty to be transparent about how the algorithm works.

Concerns were raised by the charity Privacy International, which first found references in a DWP report to its use of “cutting-edge artificial intelligence to crack down on organised criminal gangs committing large-scale benefit fraud”. A 2019 UN report into the “digital welfare state” said algorithms were “highly likely” to repeat biases reflected in existing data and make them even worse. It added: “Inbuilt forms of discrimination can fatally undermine the right to social protection for key groups and individuals.”

The government has until Friday to respond to the legal letter.

“We’re tired of the fear of the brown envelope and tired of being repeatedly forced by DWP officials just to justify who we are,” said Rick Burgess of the GMCDP. “It’s time for the DWP to come clean about how this algorithm works and explain why so many disabled people are flagged for investigation. Disabled people need support – not being ground down by a brutal system that assumes we are fraudulent until proven innocent.”

Foxglove said: “The DWP algorithm seems to send poor and disabled people to the call centre from hell. The DWP needs to explain exactly how its algorithm works and what is done to eliminate bias – or be taken to court.”

Officials stressed that specialist staff were involved in any cases of suspected fraud and error and support was offered to those who needed it. A DWP spokesperson said: “The DWP will be responding to the letter written by the representatives of the GMCDP in due course.”

Read the Source

Research

  • Defining an “AI Incident”
  • Defining an “AI Incident Response”
  • Database Roadmap
  • Related Work
  • Download Complete Database

Project and Community

  • About
  • Contact and Follow
  • Apps and Summaries
  • Editor’s Guide

Incidents

  • All Incidents in List Form
  • Flagged Incidents
  • Submission Queue
  • Classifications View
  • Taxonomies

2024 - AI Incident Database

  • Terms of use
  • Privacy Policy
  • Open twitterOpen githubOpen rssOpen facebookOpen linkedin
  • e1b50cd