Incident 57: Australian Automated Debt Assessment System Issued False Notices to Thousands

Description: Australian Department of Human Services (DHS)’s automated debt assessment system issued false or incorrect debt notices to hundreds of thousands of people, resulting in years-long lawsuits and damages to welfare recipients.
Alleged: Centrelink developed an AI system deployed by Australian Department of Human Services, which harmed Australian welfare recipients.

Suggested citation format

Yampolskiy, Roman. (2015-07-01) Incident Number 57. in McGregor, S. (ed.) Artificial Intelligence Incident Database. Responsible AI Collaborative.

Incident Stats

Incident ID
57
Report Count
38
Incident Date
2015-07-01
Editors
Sean McGregor, Khoa Lam

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscover

CSET Taxonomy Classifications

Taxonomy Details

Full Description

In July 2016, Australia’s Department of Human Services increased the autonomy of their Centrelink social services distribution program resulting in thousands of citizens receiving unexpected debt notices. The Centrelink Master Program manages many of Australia's social welfare programs by receiving requests from citizens and employers, verifying entries against government documents, and distributing payments. The changes to the system allowed Centrelink’s internal programs to autonomously send information requests to citizens if their requests did not match with government records. This caused a dramatic increase in these requests, which brought widespread complaint, especially because a significant portion of the alleged discrepancies could be validly explained by the recipient. An inspection by Australia's ombudsman found that the accuracy of the new system was comparable to the previous system, which required a human auditor, however the process by which the system was announced and deployed had some flaws.

Short Description

In July 2016, Australia’s Department of Human Services reprogrammed the software behind their Centrelink social services distribution program to autonomously issue discrepancy notices, resulting in an exponential increase in those requests.

Severity

Minor

Harm Distribution Basis

Financial means

Harm Type

Financial harm

AI System Description

The internal auditing software at Centrelink receives inputs from individuals and employers filing for benefits and compares that data against government records to identify descripancies in order to reduce welfare fraud.

System Developer

Centrelink Master Program

Sector of Deployment

Public administration and defence

Relevant AI functions

Cognition, Action

AI Techniques

Unclear

AI Applications

decision support

Location

Australia

Named Entities

Centrelink, Centrelink Master Program, Department of Human Services, Richard Glenn, Alan Tudge

Technology Purveyor

Department of Human Services, Centrelink Master Program

Beginning Date

7/2016

Ending Date

2017-01-01

Near Miss

Harm caused

Intent

Accident

Lives Lost

No

Data Inputs

welfare requests, tax records

Incident Reports

Students accused of welfare fraud say Centrelink's sums are wrong

Former and current students say Centrelink is falsely accusing them of being welfare cheats and saddling them with thousands of dollars of debt.

Sianaye studies full-time, works retail part-time. One month ago, the 23-year-old from Melbourn…

‘Hit with a scary $25,000 demand’: Centrelink debt scandal hits Aboriginal communities

Labor has called for an immediate halt to the debt recovery system until the current problems are fixed. (AAP)

The Centrelink welfare debt crisis engulfing the Department of Human Services has been affecting Aboriginal Australians, with NIT…

Centrelink debt letter scandal worsens

Centrelink is threatening dozens of Australians with bad credit ratings and legal action if they refuse to sign up to payment plans to repay thousands of dollars in questionable debt.

‘If I don't set up a payment scheme it will go to debt c…

High Farce: The Turnbull Government’s Centrelink ‘Robo-Debt’ Debacle Continues To Grow

In the lead-up to Christmas, tens of thousands of Australians started receiving threatening debt collection notices from Centrelink and its agents. Coming from a government that couldn’t run an online census, you can probably guess what hap…

A union claims almost every Centrelink debt letter is wrong and will bring a 'perfect storm' of complaints

A Medicare and Centrelink office sign is seen at Bondi Junction on March 21, 2016 in Sydney, Australia. (Source: Getty)

Centrelink staff will not be able to cope in an expected “perfect storm” of customer enquiries arising out of the debt d…

Centrelink robo-debt system wrongly targets Australian of the Year finalist

Medical researcher Janet Hammill, who works voluntarily, struggling to contact anyone at Centrelink after she was incorrectly told she owes $7,600

This article is more than 2 years old

This article is more than 2 years old

A leading fetal a…

Centrelink debt recovery program to undergo changes following public criticism

Centrelink debt recovery program to undergo changes following public criticism

Updated

The Federal Government will introduce changes to Centrelink's controversial debt recovery program, despite insisting it is working and dismissing calls f…

Centrelink's controversial data matching program to target pensioners and disabled, Labor calls for suspension

The Federal Government will expand Centrelink's automatic debt recovery program later this year to focus on aged pensioners and disability support payments.

Parliamentary Budget Office charts reveal the Government plans to use a similar dat…

Government is using false figures to defend its robo-debt system

Government is using false figures to defend its robo-debt system

Although the Government asserts that it has reclaimed $300 million in debts owed by welfare recipients, it refuses to disclose exactly how much money it has recovered.

Launche…

Centrelink staff to strike over debt scandal, budget cuts and pay

This article is more than 2 years old

This article is more than 2 years old

Department of Human Services staff will strike over pay, budget cuts and the Centrelink “robo debt” scandal, which together form an “ideological attack” by the Turn…

We're all talking about the Centrelink debt controversy, but what is 'robodebt' anyway?

We're all talking about the Centrelink debt controversy, but what is 'robodebt' anyway?

Posted

It's been making headlines since summer, when thousands of Australians suddenly found out they owed money to Centrelink.

Critics call it "robodeb…

Welfare recipients to blame for Centrelink debt system failures, Senate inquiry told

The head of the Department of Human Services has blamed the “robo debt” scandal largely on welfare recipients’ failure to engage with Centrelink.

The Senate inquiry into Centrelink’s “robo-debt” system began on Wednesday, hearing from the c…

Centrelink: Tax office says it cannot be blamed for automated debt recovery system's failings

Centrelink: Tax office says it cannot be blamed for automated debt recovery system's failings

Updated

The Australian Tax Office (ATO) has sought to distance itself from Centrelink's controversial debt recovery program, telling a Senate inqu…

Centrelink 'Robo-Debt' Program Unclear, Unreasonable And Unfair: Ombudsman

Centrelink failed to communicate properly with welfare recipients, their staff lacked training and its so-called 'robo-debt' program placed "unreasonable" expectations on claimants and should have been more rigorously tested.

Those are the …

Centrelink debt scandal: report reveals multiple failures in welfare system

This article is more than 2 years old

This article is more than 2 years old

An ombudsman’s report on the roll out of Centrelink’s automated debt recovery service has identified multiple failures that placed unreasonable burdens on welfare r…

Centrelink's debt recovery system lacks transparency and treated some customers unfairly, ombudsman finds

Centrelink's debt recovery system lacks transparency and treated some customers unfairly, ombudsman finds

Updated

Centrelink's controversial debt recovery system lacks transparency and has treated some customers unfairly, the Commonwealth O…

Centrelink robo-debt 'abject failure' and arguably unlawful, Victoria Legal Aid says

Scathing assessment comes as system again put under the microscope, this time by external auditors PwC Australia

This article is more than 2 years old

This article is more than 2 years old

Victoria Legal Aid has described Centrelink’s robo-…

Centrelink's debt data-matching failed government's privacy guidelines, campaigners say

Australian Privacy Foundation tells Senate inquiry into ‘robo debt’ scandal there is no evidence guidelines were followed

This article is more than 1 year old

This article is more than 1 year old

Privacy campaigners say Centrelink’s use of …

Centrelink loses more than one third of debt cases appealed to tribunal

More than one third of Centrelink welfare debt recovery cases that are appealed to the independent tribunal are overturned.

The Administrative Appeals Tribunal has set aside 960 Centrelink debt decisions out of 2,699 appeals lodged between …

Senate inquiry calls for Centrelink robo-debt system to be suspended until fixed

Senate inquiry report recommends all debts calculated under flawed model be re-assessed and system be redesigned

This article is more than 1 year old

This article is more than 1 year old

A Senate inquiry has called for Centrelink’s controve…

Centrelink says it has no plans to use robo-debt system to target pensioners

Expansion of data-matching program aimed at scrutinising whether person’s earnings from trusts or family daycare make them ineligible for welfare, government says

This article is more than 1 year old

This article is more than 1 year old

Cen…

Citizens ‘monstered by own government’

CENTRELINK’S “threatening” debt recovery letters have reached the next level and Australians are terrified, according to a radio host who angrily confronted the government.

The notes sent to people in selected locations now feature the Aust…

20,000 people sent Centrelink 'robo-debt' notices found to owe less or nothing

The Turnbull government has admitted it issued robo-debt recovery notices to 20,000 welfare recipients who were later found to owe less or even nothing.

Documents presented to Parliament by Human Services Minister Alan Tudge showed the use …

Centrelink scandal: tens of thousands of welfare debts wiped or reduced

Data shows 7,456 debts were reduced to zero and another 12,524 partially reduced between July last year and March

This article is more than 1 year old

This article is more than 1 year old

At least 20,000 Centrelink debts were either wiped o…

Public sector union​ condemns Centrelink move to privatise call centre

This article is more than 1 year old

This article is more than 1 year old

The public sector union has condemned moves to privatise Centrelink’s much-criticised call centre, saying it would give Serco access to vast amounts of personal infor…

Government Rejects Findings From Centrelink Robo-Debt Inquiry

A Community Affairs References Committee released a report in June into Centrelink’s controversial robo-debt program, making 21 recommendations to fix the program that was rolled out in mid-2016.

The report said the system was “so flawed it…

Centrelink to use 1,000 labour-hire staff to help recover welfare debts

Department says surge in use of private contractors is temporary but union says government is ‘privatising our safety net’

Centrelink to use 1,000 labour-hire staff to help recover welfare debts

Centrelink will enlist up to 1,000 staff from…

'This is a joke': fighting Centrelink’s robo-debt, one year on

Retail worker Laura has had a $24,000 debt hanging over her head since October last year.

Laura, who’s asked Hack not to use her surname, works casually and had been getting income support payments through Centrelink from 2011 to 2014.

Late…

Centrelink robo-debt program accused of enforcing 'illegal' debts

The Turnbull government’s robo-debt program involves enforcement of “illegal” debts that in some cases are inflated or nonexistent, a former member of the Administrative Appeals Tribunal has said.

The scathing indictment of the program is c…

Planning an overseas trip? Make sure you are not in debt to Centrelink!

In a bid to recoup $800 million of debt, former welfare recipients who owe money to the federal government will face a travel ban until the funds are repaid.

So far 20 orders have been issued according to human services minister, Michael Ke…

Government admits robodebt was unlawful as it settles legal challenge

The federal government has settled a landmark challenge against its robodebt program – conceding a $2,500 debt raised against Deanna Amato was not lawful because it relied on income averaging.

In orders made by consent on Wednesday, the fed…

Robodebt removed humans from Human Services, and the Government is facing the consequences

The problem with artificial intelligence is that it's artificial.

The problem with the intelligence of humans is that it's limited, variable and compromised by judgement and values.

Put both together and you get a good understanding of how …

Government to pay back $721m as it scraps Robodebt for Centrelink welfare recipients

The Federal Government will refund $721 million worth of debts it clawed back through its controversial Robodebt scheme.

The scheme saw hundreds of thousands of people issued with computer-generated debt notices, some of which made demands …

Robodebt: government to refund 470,000 unlawful Centrelink debts worth $721m

The federal government has finally agreed to pay back hundreds of thousands of people who were hit with unlawful and incorrect Centrelink debts over a four-year period through the Coalition’s botched robodebt scheme.

Years after vowing to r…

Robodebt scandal: leak reveals unlawful debts predate 2015 but government has no plans to pay back money

The Australian government has privately acknowledged the robodebt scandal may go beyond the unlawful 470,000 debts already identified for refunds, but it has no plans to pay back the money because it believes it would be too difficult to id…

'Robodebt-related trauma': the victims still paying for Australia's unlawful welfare crackdown

Nathan Kearney says he lost two and half years of his life to robodebt. He's still seeing a counsellor about it.

Four years ago he was living in Brisbane, seeing and playing gigs, working several different casual jobs and generally enjoying…

Robodebt: court approves $1.8bn settlement for victims of government’s ‘shameful’ failure

A “shameful chapter” in public administration has led to the federal court approving a settlement worth $1.8bn between the commonwealth and victims of the Coalition’s robodebt scheme.

Justice Bernard Murphy in Friday’s judgment criticised t…

Robodebt: government fights to keep secret documents that may show 'what went wrong'

Robodebt: government fights to keep secret documents that may show ‘what went wrong’

The Morrison government is fighting to keep under wraps documents that a former public servant says could show “what went wrong” with Centrelink’s botched …

Variants

A "variant" is an incident that shares the same causative factors, produces similar harms, and involves the same intelligent systems as a known AI incident. Rather than index variants as entirely separate incidents, we list variations of incidents under the first similar incident submitted to the database. Unlike other submission types to the incident database, variants are not required to have reporting in evidence external to the Incident Database. Learn more from the research paper.