Incident 57: Australian Automated Debt Assessment System Issued False Notices to Thousands

Description: Australian Department of Human Services (DHS)’s automated debt assessment system issued false or incorrect debt notices to hundreds of thousands of people, resulting in years-long lawsuits and damages to welfare recipients.

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscoverView HistoryView History
Alleged: Centrelink developed an AI system deployed by Australian Department of Human Services, which harmed Australian welfare recipients.

Incident Stats

Incident ID
57
Report Count
39
Incident Date
2015-07-01
Editors
Sean McGregor, Khoa Lam

CSETv0 Taxonomy Classifications

Taxonomy Details

Full Description

In July 2016, Australia’s Department of Human Services increased the autonomy of their Centrelink social services distribution program resulting in thousands of citizens receiving unexpected debt notices. The Centrelink Master Program manages many of Australia's social welfare programs by receiving requests from citizens and employers, verifying entries against government documents, and distributing payments. The changes to the system allowed Centrelink’s internal programs to autonomously send information requests to citizens if their requests did not match with government records. This caused a dramatic increase in these requests, which brought widespread complaint, especially because a significant portion of the alleged discrepancies could be validly explained by the recipient. An inspection by Australia's ombudsman found that the accuracy of the new system was comparable to the previous system, which required a human auditor, however the process by which the system was announced and deployed had some flaws.

Short Description

In July 2016, Australia’s Department of Human Services reprogrammed the software behind their Centrelink social services distribution program to autonomously issue discrepancy notices, resulting in an exponential increase in those requests.

Severity

Minor

Harm Distribution Basis

Financial means

Harm Type

Financial harm

AI System Description

The internal auditing software at Centrelink receives inputs from individuals and employers filing for benefits and compares that data against government records to identify descripancies in order to reduce welfare fraud.

System Developer

Centrelink Master Program

Sector of Deployment

Public administration and defence

Relevant AI functions

Cognition, Action

AI Techniques

Unclear

AI Applications

decision support

Location

Australia

Named Entities

Centrelink, Centrelink Master Program, Department of Human Services, Richard Glenn, Alan Tudge

Technology Purveyor

Department of Human Services, Centrelink Master Program

Beginning Date

7/2016

Ending Date

2017-01-01

Near Miss

Harm caused

Intent

Accident

Lives Lost

No

Data Inputs

welfare requests, tax records

CSETv1 Taxonomy Classifications

Taxonomy Details

Harm Distribution Basis

none

Sector of Deployment

public administration

Students accused of welfare fraud say Centrelink's sums are wrong
abc.net.au · 2016

Former and current students say Centrelink is falsely accusing them of being welfare cheats and saddling them with thousands of dollars of debt.

Sianaye studies full-time, works retail part-time. One month ago, the 23-year-old from Melbourn…

‘Hit with a scary $25,000 demand’: Centrelink debt scandal hits Aboriginal communities
sbs.com.au · 2017

Labor has called for an immediate halt to the debt recovery system until the current problems are fixed. (AAP)

The Centrelink welfare debt crisis engulfing the Department of Human Services has been affecting Aboriginal Australians, with NIT…

Centrelink debt letter scandal worsens
finance.nine.com.au · 2017

Centrelink is threatening dozens of Australians with bad credit ratings and legal action if they refuse to sign up to payment plans to repay thousands of dollars in questionable debt.

‘If I don't set up a payment scheme it will go to debt c…

High Farce: The Turnbull Government’s Centrelink ‘Robo-Debt’ Debacle Continues To Grow
newmatilda.com · 2017

In the lead-up to Christmas, tens of thousands of Australians started receiving threatening debt collection notices from Centrelink and its agents. Coming from a government that couldn’t run an online census, you can probably guess what hap…

A union claims almost every Centrelink debt letter is wrong and will bring a 'perfect storm' of complaints
businessinsider.com.au · 2017

A Medicare and Centrelink office sign is seen at Bondi Junction on March 21, 2016 in Sydney, Australia. (Source: Getty)

Centrelink staff will not be able to cope in an expected “perfect storm” of customer enquiries arising out of the debt d…

Centrelink robo-debt system wrongly targets Australian of the Year finalist
theguardian.com · 2017

Medical researcher Janet Hammill, who works voluntarily, struggling to contact anyone at Centrelink after she was incorrectly told she owes $7,600

This article is more than 2 years old

This article is more than 2 years old

A leading fetal a…

Centrelink debt recovery program to undergo changes following public criticism
abc.net.au · 2017

Centrelink debt recovery program to undergo changes following public criticism

Updated

The Federal Government will introduce changes to Centrelink's controversial debt recovery program, despite insisting it is working and dismissing calls f…

Centrelink's controversial data matching program to target pensioners and disabled, Labor calls for suspension
abc.net.au · 2017

The Federal Government will expand Centrelink's automatic debt recovery program later this year to focus on aged pensioners and disability support payments.

Parliamentary Budget Office charts reveal the Government plans to use a similar dat…

Government is using false figures to defend its robo-debt system
yourlifechoices.com.au · 2017

Government is using false figures to defend its robo-debt system

Although the Government asserts that it has reclaimed $300 million in debts owed by welfare recipients, it refuses to disclose exactly how much money it has recovered.

Launche…

Centrelink staff to strike over debt scandal, budget cuts and pay
theguardian.com · 2017

This article is more than 2 years old

This article is more than 2 years old

Department of Human Services staff will strike over pay, budget cuts and the Centrelink “robo debt” scandal, which together form an “ideological attack” by the Turn…

We're all talking about the Centrelink debt controversy, but what is 'robodebt' anyway?
abc.net.au · 2017

We're all talking about the Centrelink debt controversy, but what is 'robodebt' anyway?

Posted

It's been making headlines since summer, when thousands of Australians suddenly found out they owed money to Centrelink.

Critics call it "robodeb…

Centrelink: Tax office says it cannot be blamed for automated debt recovery system's failings
abc.net.au · 2017

Centrelink: Tax office says it cannot be blamed for automated debt recovery system's failings

Updated

The Australian Tax Office (ATO) has sought to distance itself from Centrelink's controversial debt recovery program, telling a Senate inqu…

Welfare recipients to blame for Centrelink debt system failures, Senate inquiry told
theguardian.com · 2017

The head of the Department of Human Services has blamed the “robo debt” scandal largely on welfare recipients’ failure to engage with Centrelink.

The Senate inquiry into Centrelink’s “robo-debt” system began on Wednesday, hearing from the c…

Centrelink's debt recovery system lacks transparency and treated some customers unfairly, ombudsman finds
abc.net.au · 2017

Centrelink's debt recovery system lacks transparency and treated some customers unfairly, ombudsman finds

Updated

Centrelink's controversial debt recovery system lacks transparency and has treated some customers unfairly, the Commonwealth O…

Centrelink debt scandal: report reveals multiple failures in welfare system
theguardian.com · 2017

This article is more than 2 years old

This article is more than 2 years old

An ombudsman’s report on the roll out of Centrelink’s automated debt recovery service has identified multiple failures that placed unreasonable burdens on welfare r…

Centrelink 'Robo-Debt' Program Unclear, Unreasonable And Unfair: Ombudsman
huffingtonpost.com.au · 2017

Centrelink failed to communicate properly with welfare recipients, their staff lacked training and its so-called 'robo-debt' program placed "unreasonable" expectations on claimants and should have been more rigorously tested.

Those are the …

Centrelink robo-debt 'abject failure' and arguably unlawful, Victoria Legal Aid says
theguardian.com · 2017

Scathing assessment comes as system again put under the microscope, this time by external auditors PwC Australia

This article is more than 2 years old

This article is more than 2 years old

Victoria Legal Aid has described Centrelink’s robo-…

Centrelink's debt data-matching failed government's privacy guidelines, campaigners say
theguardian.com · 2017

Australian Privacy Foundation tells Senate inquiry into ‘robo debt’ scandal there is no evidence guidelines were followed

This article is more than 1 year old

This article is more than 1 year old

Privacy campaigners say Centrelink’s use of …

Centrelink loses more than one third of debt cases appealed to tribunal
theguardian.com · 2017

More than one third of Centrelink welfare debt recovery cases that are appealed to the independent tribunal are overturned.

The Administrative Appeals Tribunal has set aside 960 Centrelink debt decisions out of 2,699 appeals lodged between …

Senate inquiry calls for Centrelink robo-debt system to be suspended until fixed
theguardian.com · 2017

Senate inquiry report recommends all debts calculated under flawed model be re-assessed and system be redesigned

This article is more than 1 year old

This article is more than 1 year old

A Senate inquiry has called for Centrelink’s controve…

Centrelink says it has no plans to use robo-debt system to target pensioners
theguardian.com · 2017

Expansion of data-matching program aimed at scrutinising whether person’s earnings from trusts or family daycare make them ineligible for welfare, government says

This article is more than 1 year old

This article is more than 1 year old

Cen…

Citizens ‘monstered by own government’
news.com.au · 2017

CENTRELINK’S “threatening” debt recovery letters have reached the next level and Australians are terrified, according to a radio host who angrily confronted the government.

The notes sent to people in selected locations now feature the Aust…

20,000 people sent Centrelink 'robo-debt' notices found to owe less or nothing
smh.com.au · 2017

The Turnbull government has admitted it issued robo-debt recovery notices to 20,000 welfare recipients who were later found to owe less or even nothing.

Documents presented to Parliament by Human Services Minister Alan Tudge showed the use …

Centrelink scandal: tens of thousands of welfare debts wiped or reduced
theguardian.com · 2017

Data shows 7,456 debts were reduced to zero and another 12,524 partially reduced between July last year and March

This article is more than 1 year old

This article is more than 1 year old

At least 20,000 Centrelink debts were either wiped o…

Public sector union​ condemns Centrelink move to privatise call centre
theguardian.com · 2017

This article is more than 1 year old

This article is more than 1 year old

The public sector union has condemned moves to privatise Centrelink’s much-criticised call centre, saying it would give Serco access to vast amounts of personal infor…

Government Rejects Findings From Centrelink Robo-Debt Inquiry
probonoaustralia.com.au · 2017

A Community Affairs References Committee released a report in June into Centrelink’s controversial robo-debt program, making 21 recommendations to fix the program that was rolled out in mid-2016.

The report said the system was “so flawed it…

Centrelink to use 1,000 labour-hire staff to help recover welfare debts
theguardian.com · 2017

Department says surge in use of private contractors is temporary but union says government is ‘privatising our safety net’

Centrelink to use 1,000 labour-hire staff to help recover welfare debts

Centrelink will enlist up to 1,000 staff from…

'This is a joke': fighting Centrelink’s robo-debt, one year on
abc.net.au · 2017

Retail worker Laura has had a $24,000 debt hanging over her head since October last year.

Laura, who’s asked Hack not to use her surname, works casually and had been getting income support payments through Centrelink from 2011 to 2014.

Late…

Centrelink robo-debt program accused of enforcing 'illegal' debts
theguardian.com · 2018

The Turnbull government’s robo-debt program involves enforcement of “illegal” debts that in some cases are inflated or nonexistent, a former member of the Administrative Appeals Tribunal has said.

The scathing indictment of the program is c…

Planning an overseas trip? Make sure you are not in debt to Centrelink!
neoskosmos.com · 2018

In a bid to recoup $800 million of debt, former welfare recipients who owe money to the federal government will face a travel ban until the funds are repaid.

So far 20 orders have been issued according to human services minister, Michael Ke…

Government admits robodebt was unlawful as it settles legal challenge
theguardian.com · 2019

The federal government has settled a landmark challenge against its robodebt program – conceding a $2,500 debt raised against Deanna Amato was not lawful because it relied on income averaging.

In orders made by consent on Wednesday, the fed…

Robodebt removed humans from Human Services, and the Government is facing the consequences
abc.net.au · 2020

The problem with artificial intelligence is that it's artificial.

The problem with the intelligence of humans is that it's limited, variable and compromised by judgement and values.

Put both together and you get a good understanding of how …

Government to pay back $721m as it scraps Robodebt for Centrelink welfare recipients
abc.net.au · 2020

The Federal Government will refund $721 million worth of debts it clawed back through its controversial Robodebt scheme.

The scheme saw hundreds of thousands of people issued with computer-generated debt notices, some of which made demands …

Robodebt: government to refund 470,000 unlawful Centrelink debts worth $721m
theguardian.com · 2020

The federal government has finally agreed to pay back hundreds of thousands of people who were hit with unlawful and incorrect Centrelink debts over a four-year period through the Coalition’s botched robodebt scheme.

Years after vowing to r…

Robodebt scandal: leak reveals unlawful debts predate 2015 but government has no plans to pay back money
theguardian.com · 2020

The Australian government has privately acknowledged the robodebt scandal may go beyond the unlawful 470,000 debts already identified for refunds, but it has no plans to pay back the money because it believes it would be too difficult to id…

'Robodebt-related trauma': the victims still paying for Australia's unlawful welfare crackdown
theguardian.com · 2020

Nathan Kearney says he lost two and half years of his life to robodebt. He's still seeing a counsellor about it.

Four years ago he was living in Brisbane, seeing and playing gigs, working several different casual jobs and generally enjoying…

Robodebt: court approves $1.8bn settlement for victims of government’s ‘shameful’ failure
theguardian.com · 2021

A “shameful chapter” in public administration has led to the federal court approving a settlement worth $1.8bn between the commonwealth and victims of the Coalition’s robodebt scheme.

Justice Bernard Murphy in Friday’s judgment criticised t…

Robodebt: government fights to keep secret documents that may show 'what went wrong'
theguardian.com · 2021

Robodebt: government fights to keep secret documents that may show ‘what went wrong’

The Morrison government is fighting to keep under wraps documents that a former public servant says could show “what went wrong” with Centrelink’s botched …

‘Platitudes and false words’: mother of robodebt victim who took own life tells inquiry of government stonewalling
theguardian.com · 2023

The mother of a robodebt victim who took his own life has told a royal commission that she was stonewalled by the former Coalition government while she sought answers about her son’s welfare debts for more than five years.

The commission is…

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.