Associated Incidents
Summary
This submission examines the human rights implications of Artificial Intelligence (AI) and other data-driven technologies in welfare benefits programs, such as cash and food assistance programs. Through a series of case studies, this submission explains how States delegate key welfare functions, such as determinations of eligibility and benefits levels, to automated decision-making models, some of which rely on data mining, machine learning and other processes or technologies typically associated with the field of AI. It also assesses how automated decision-making interferes with the rights to privacy and social security, and the obligations of States to guarantee the exercise of these rights without discrimination and undue private interference.
Detection, Investigation and Punishment of Welfare Fraud
Michigan Integrated Data Automated System (Michigan, United States)
In October 2013, Michigan’s Unemployment Insurance Agency (UIA) launched the Michigan Integrated Data Automated System (MiDAS) to adjudicate and impose penalties for unemployment benefits fraud. Between October 2013 and August 2015, MiDAS was programmed to automatically treat differences between income figures reported by beneficiaries and their employers as evidence of fraud. The system was not capable of investigating whether there are legitimate reasons for these discrepancies, such as employer error or pay disputes. Like OCI, MiDAS was also unable to determine whether these discrepancies are attributable to fluctuations in a beneficiary’s income.
Based on its initial assessments, MiDAS sent beneficiaries suspected of fraud online multiple-choice questionnaires asking whether they are “intentionally provid[ing] false information to obtain benefits you were not entitle[d] to receive,” and “[w]hy ... you believe you were entitled to benefits.” Failure to respond in ten days, or a response that MiDAS deemed unsatisfactory, would automatically trigger conclusive determinations of fraud. Based on these determinations, MiDAS would terminate the benefits of affected beneficiaries and initiate proceedings to seize their tax refunds or garnish their wages.
UIA subsequently found that, between October 2013 and August 2015, about 44,000 of the 62,784 determinations of fraud that MiDAS generated were in error. In a lawsuit that a group of beneficiaries filed against UIA, the U.S. Federal Court of Appeals for the region concluded that MiDAS “did not allow for a fact-based adjudication or give the claimant the opportunity to present evidence to prove that he or she did not engage in disqualifying conduct.”
Despite these failures, UIA continues to operate MiDAS. It has committed to additional data analysis to detect benefits payments needing “further review” and to enhance the appeals process. However, it is unclear whether UIA has made any changes to the underlying data matching algorithm or incorporated meaningful human review into the system’s fraud detection functions.