レポート 2805

From the outside, Rotterdam’s welfare algorithm appears complex. The system, which was originally developed by consulting firm Accenture before the city took over development in 2018, is trained on data collected by Rotterdam’s welfare department. It assigns people risk scores based on 315 factors. Some are objective facts, such as age or gender identity. Others, such as a person’s appearance or how outgoing they are, are subjective and based on the judgment of social workers.

In Hoek van Holland, a town to the west of Rotterdam that is administratively part of the city, Pepita Ceelie is trying to understand how the algorithm ranked her as high risk. Ceelie is 61 years old, heavily tattooed, and has a bright pink buzz cut. She likes to speak English and gets to the point quickly. For the past 10 years, she has lived with chronic illness and exhaustion, and she uses a mobility scooter whenever she leaves the house. 

Ceelie has been investigated twice by Rotterdam’s welfare fraud team, first in 2015 and again in 2021. Both times investigators found no wrongdoing. In the most recent case, she was selected for investigation by the city’s risk-scoring algorithm. Ceelie says she had to explain to investigators why her brother sent her €150 ($180) for her sixtieth birthday, and that it took more than five months for them to close the case.

Sitting in her blocky, 1950s house, which is decorated with photographs of her garden, Ceelie taps away at a laptop. She’s entering her details into a reconstruction of Rotterdam’s welfare risk-scoring system created as part of this investigation. The user interface, built on top of the city’s algorithm and data, demonstrates how Ceelie’s risk score was calculated—and suggests which factors could have led to her being investigated for fraud.

All 315 factors of the risk-scoring system are initially set to describe an imaginary person with “average” values in the data set. When Ceelie personalizes the system with her own details, her score begins to change. She starts at a default score of 0.3483—the closer to 1 a person’s score is, the more they are considered a high fraud risk. When she tells the system that she doesn’t have a plan in place to find work, the score rises (0.4174). It drops when she enters that she has lived in her home for 20 years (0.3891). Living outside of central Rotterdam pushes it back above 0.4. 

Switching her gender from male to female pushes her score to 0.5123. “This is crazy,” Ceelie says. Even though her adult son does not live with her, his existence, to the algorithm, makes her more likely to commit welfare fraud. “What does he have to do with this?” she says. Ceelie’s divorce raises her risk score again, and she ends with a score of 0.643: high risk, according to Rotterdam’s system.

“They don’t know me, I’m not a number,” Ceelie says. “I’m a human being.” After two welfare fraud investigations, Ceelie has become angry with the system. “They’ve only opposed me, pulled me down to suicidal thoughts,” she says. Throughout her investigations, she has heard other people’s stories, turning to a Facebook support group set up for people having problems with the Netherlands’ welfare system. Ceelie says people have lost benefits for minor infractions, like not reporting grocery payments or money received from their parents.

“There are a lot of things that are not very clear for people when they get welfare,” says Jacqueline Nieuwstraten, a lawyer who has handled dozens of appeals against Rotterdam’s welfare penalties. She says the system has been quick to punish people and that investigators fail to properly consider individual circumstances.

The Netherlands takes a tough stance on welfare fraud, encouraged by populist right-wing politicians. And of all the country’s regions, Rotterdam cracks down on welfare fraud the hardest. Of the approximately 30,000 people who receive benefits from the city each year, around a thousand are investigated after being flagged by the city's algorithm. In total, Rotterdam investigates up to 6,000 people annually to check if their payments are correct. In 2019, Rotterdam issued 2,400 benefits penalties, which can include fines and cutting people’s benefits completely. In 2022 almost a quarter of the appeals that reached the country’s highest court came from Rotterdam.

From the algorithm’s deployment in 2017 until its use was halted in 2021, it flagged up to a third of the people the city investigated each year, while others were selected by humans based on a theme—such as single men living in a certain neighborhood.

Rotterdam has moved to make its overall welfare system easier for people to navigate since 2020. (For example, the number of benefits penalties dropped to 749 in 2021.) De Rotte, the director of the city’s income department, says these changes include adding a “human dimension” to its welfare processes. The city has also relaxed rules around how much money claimants can receive from friends and family, and it now allows adults to live together without any impact on their benefits. As a result, Nieuwstraten says, the number of complaints she has received about welfare investigations has decreased in recent years.

The city’s decision to pause its use of the welfare algorithm in 2021 came after an investigation by the Rotterdam Court of Audit on the development and use of algorithms in the city. The government auditor found there was “insufficient coordination” between the developers of the algorithms and city workers who use them, which could lead to ethical considerations being neglected. The report also criticized the city for not evaluating whether the algorithms were better than the human systems they replaced. Singling out the welfare fraud algorithm, the report found there was a likelihood of biased outcomes based on the types of data used to determine people’s risk scores.

Since then, the city has been working to develop a new version—though minutes from council meetings show there are doubts that it can successfully build a system that is transparent and legal. De Rotte says that since the Court of Audit report, the city has worked to add “more safeguards” to the development of algorithms in general, including introducing an algorithm register to show what algorithms it uses. “A new model must not have any appearance of bias, must be as transparent as possible, and must be easy to explain to the outside world,” de Rotte says. Welfare recipients are currently being selected for investigation at random, de Rotte adds.

WHILE THE CITY works to rebuild its algorithm, those caught up in the welfare system have been battling to discover how it works—and whether they were selected for investigation by a flawed system.

Among them is Oran, a 35-year old who’s lived in Rotterdam all his life. In February 2018 he received a letter saying he was being investigated for welfare fraud. Oran, who asked that his real name not be used for privacy reasons, has a number of health issues that make it difficult to find work. In 2018, he was receiving a monthly loan from a family member. Rotterdam’s local government asked him to document the loan and agree that it be paid back. Although Oran did this, investigators pursued fraud charges against him, and the city said he should have €6,000 withheld from future benefits payments, a sum combining the amount he had been loaned plus additional fines.

From 2018 to 2021, Oran fought against the local authority in court. He says being accused of committing fraud took a huge toll. During the investigation, he says, he couldn't focus on anything else and didn’t think he had a future. “It got really difficult. I thought a lot about suicide,” he says. During the investigation, he was not well enough to find paid or volunteer work, and his relationship with his family became strained.

Two court appeals later, in June 2021, Oran cleared his name, and the city refunded the €6,000 it had deducted from his benefits payments. “It feels like justice,” he says. Despite the lengthy process, he did not find out why he was selected for scrutiny, what his risk scores were, or what data contributed to the creation of his scores. So he requested it all. Five months later, in April 2021, he received his risk scores for 2018 and 2019.

While his files revealed he was not selected for investigation by the algorithm but rather part of a selection of single men, his risk score was among the top 15 percent of benefits recipients. His zip code, history of depression, and assessments by social workers contributed to his high score. “That’s not reality, that’s not me, that’s not my life, it’s just a bunch of numbers,” Oran says.

As the use of algorithmic systems grows, it could become harder for people to understand why decisions have been made and to appeal against them. Tamilla Abdul-Aliyeva, a senior policy advisor at Amnesty International in the Netherlands, says people should be told if they are being investigated based on algorithmic analysis, what data was used to train the algorithm, and what selection criteria were used. “Transparency is key for protecting human rights and also very important in the democratic society,” says Abdul-Aliyeva. De Rotte says Rotterdam plans to give people more information about “why and how they were selected” and that more details of the new model will be announced “before the summer.”

For those already caught in Rotterdam’s welfare dragnet, there is little solace. Many of them, including Oran and Ceelie, say they don’t want the city to use an algorithm to judge vulnerable people. Ceelie says it feels like she has been “stamped” with a number and that she is considering taking Rotterdam’s government to court over its use of the algorithm. Developing and using the algorithm won’t make people feel like they are being treated with care, she says. “Algorithms aren’t human. Call me up, with a human being, not a number, and talk to me. Don’t do this.”

If you or someone you know needs help, call 1-800-273-8255 for free, 24-hour support from the National Suicide Prevention Lifeline. You can also text HOME to 741-741 for the Crisis Text Line. Outside the US, visit the International Association for Suicide Prevention for crisis centers around the world.

Additional reporting by Eva Constantaras, Justin-Casimir Braun, and Soizic Penicaud. Reporting was supported by the Pulitzer Center’s AI Accountability Network and the Eyebeam Center for the Future of Journalism.