Incident 611: UK Government AI Allegedly Targets Disproportionate Numbers of Certain Nationals for Fraud Review

Description: The UK's Department for Work and Pensions (DWP) faced scrutiny after many Bulgarian nationals reported unexplained suspensions of their Universal Credit benefits. The MP for Edmonton raised concerns about potential nationality-based targeting for benefit fraud investigations, leading to poverty and homelessness among affected individuals. The Home Office's own equality impact assessment found it was flagging a disproportionate number of marriages from Greece, Albania, Bulgaria and Romania.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History

Incident Stats

Incident ID
611
Report Count
13
Incident Date
2021-12-06
Editors
Daniel Atherton
MP raises concern over Bulgarian nationals’ UK benefit suspensions
theguardian.com · 2021

Dozens of Bulgarian nationals have been tipped into poverty after unexplained benefit suspensions following the creation of a “risk review” unit in the Department for Work and Pensions, an MP has said.

Kate Osamor, the Labour MP for Edmonto…

DWP ramps up investment in AI models to detect fraud despite algorithmic bias concerns
techmonitor.ai · 2023

The Department for Work and Pensions (DWP) is ploughing £70m into its digital transformation projects over the next three years. However, the National Audit Office (NAO) has raised concerns about investments the department is making in arti…

AI use widened to assess universal credit applications and tackle fraud
theguardian.com · 2023

The government has widened its deployment of artificial intelligence to uncover welfare fraud, despite warnings of algorithmic bias against groups of vulnerable claimants.

In a £70m investment applying “advanced analytics” to requests for u…

Universal Credit: Warnings over AI use to risk-score benefit claims
bbc.com · 2023

The Department of Work and Pensions (DWP) has revealed plans to widen its use of the technology to tackle fraud.

Campaigners say more information is needed to ensure the system does not make biased referrals for benefit investigations.

The …

DWP Widening Use of Anti-fraud Tech Amid Concerns From Campaign Groups
digit.fyi · 2023

In a recent National Audit Office (NAO) report concerning the Department for Work and Pensions (DWP) financial accounts, it has come to light that the DWP is expanding its use of machine learning for identifying potential benefits fraud.

Si…

UK warned over lack of transparency on use of AI to vet welfare claims
theguardian.com · 2023

The UK government risks contempt of court unless it improves its response to requests for transparency over the use of artificial intelligence (AI) to vet welfare claims, the information commissioner has said.

Over the past two years, the D…

DWP’s fraud and error checking AI still displaying signs of bias
techmonitor.ai · 2023

Senior civil servants told a committee of MPs that an AI being developed by the Department for Work and Pensions (DWP) to detect fraud and error is still in the learning stages of its development and is exhibiting bias against some claimant…

UK risks scandal over ‘bias’ in AI tools in use across public sector
theguardian.com · 2023

Kate Osamor, the Labour MP for Edmonton, recently received an email from a charity about a constituent of hers who had had her benefits suspended apparently without reason.

“For well over a year now she has been trying to contact DWP [the D…

AI employed for crucial decisions in over 8 Whitehall departments
interestingengineering.com · 2023

In the past, artificial intelligence (AI) use in public services has caused uproars. For instance, in the Netherlands, tax authorities used technology to find fraud but made many mistakes. This resulted in a hefty fine of €3.7m, pushing tho…

UK govt using AI to decide benefits for people, marriage licenses: Report
hindustantimes.com · 2023

The report made findings of some potentially discriminatory results made by the artificial intelligence.

The UK government officials are using artificial intelligence (AI) and complex algorithms to help make various decisions including thos…

Revealed: Government using AI to decide on benefits and driving licences
independent.co.uk · 2023
UK government using AI to decide benefit payments and spot sham marriages, investigation reveals
uk.news.yahoo.com · 2023

The government's use of artificial intelligence (AI) risks producing discriminatory results against benefit claimants and ethnic minorities, an investigation has found.

A total of eight Whitehall departments and some police forces are using…

Decisions on benefits and migrants ‘being made by AI’
thetimes.co.uk · 2023

Civil servants in at least eight government departments and the police are using artificial intelligence to make decisions on welfare, immigration and criminal justice, it has been claimed.

The Guardian reported that the systems were being …

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.