Computer programs that perform risk assessments of crime suspects are increasingly common in American courtrooms, and are used at every stage of the criminal justice systems to determine who may be set free or granted parole, and the size of the bond they must pay. By 2016, the results of these assessments were given to judges during criminal sentencing and a sentencing reform bill was proposed in Congress to mandate the use of such assessments in federal prisons. In a study of the risk scores assigned to more than 7,000 people in Florida's Broward County in 2013 and 2014, ProPublica found that only 20% of the people the system predicted would commit violent crimes had actually done so. For the full range of crimes including misdemeanours, 61% of those predicted to re-offend were arrested for later crimes over the following two years.
ProPublic also found significant racial disparities. Although the algorithm made errors at roughly the same rate for black and white defendants, it incorrectly labelled black defendants as likely to commit further crimes at twice the reats as white defendants. Conversely, white defendants were mislabelled as low risk more often than black defendants. Northpointe, the company that produced the system, known as COMPAS, disputed ProPublic's analysis but declined to share its calculations, which the company said were proprietary. However, it did disclose that the basics of its formula included factors such as education levels and employment status among the 137 questions that are either answered by defendants or extracted from criminal records. These tools have been rolled out in many areas before they have been rigorously evaluated, and defendants are rarely able to find out the basis for the scores they're assigned.
https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Writer: Julia Angwin, Jeff Larson, Surya Mattu, Lauren Kirchner
Publication: ProPublica