BANGKOK, April 12 (Thomson Reuters Foundation) - Few cases ruffle Hamid Ismail after nearly two decades as a lawyer, but he was taken aback when a man he defended was sentenced with the help of an artificial intelligence tool in the Malaysian state of Sabah.
Ismail knew courts in Sabah and neighbouring Sarawak were testing the AI tool for sentencing recommendations as part of a nationwide pilot, but was uneasy that the technology was being used before lawyers, judges and the public fully understood it.
There was no proper consultation on the technology's use, and it is not contemplated in the country's penal code, he said.
"Our Criminal Procedure Code does not provide for use of AI in the courts ... I think it's unconstitutional," said Ismail, adding that the AI-recommended sentence for his client for a minor drug possession charge was too harsh.
The courts of Sabah and Sarawak piloted software developed by Sarawak Information Systems, a state government firm, which said at the time that it had held consultations during the process, and taken steps to address some of the concerns raised.
World over, the use of AI in the criminal justice system is growing quickly, from the popular DoNotPay chatbot lawyer mobile app to robot judges in Estonia adjudicating small claims, to robot mediators in Canada and AI judges in Chinese courts.
Authorities say AI-based systems make sentencing more consistent and can clear case backlogs quickly and cheaply, helping all parties in legal proceedings to avoid lengthy, expensive and stressful litigation.
More than a third of government respondents in a global survey last year by research firm Gartner indicated that they planned to increase investments in AI-powered systems including chatbots, facial recognition and data mining across sectors.
This month, Malaysian federal authorities aim to conclude their nationwide trial of the AI sentencing tools, which they have said "can improve the quality of judgment", though it is not entirely clear how they will be used in courts.
A spokesperson for Malaysia's Chief Justice said the use of AI in courts was "still in the trial stage", declining further comment.
Bias, Mitigating Factors
Critics warn AI risks entrenching and amplifying bias against minorities and marginalised groups, saying the technology lacks a judge's ability to weigh up individual circumstances, or adapt to changing social mores.
"In sentencing, judges don't just look at the facts of the case - they also consider mitigating factors, and use their discretion. But AI cannot use discretion," Ismail told the Thomson Reuters Foundation.
Considering aggravating and mitigating factors "requires a human mind", said Charles Hector Fernandez, a Malaysian human rights lawyer.
"Sentences also vary with changing times and changing public opinion. We need more judges and prosecutors to handle increasing caseloads; AI cannot replace human judges," he added.
Seeking to address concerns that its AI software might lead to bias in sentencing, Sarawak Information Systems said it had removed the "race" variable from the algorithm.
But while "such mitigating measures are valuable, they do not make the system perfect", said a 2020 report on the tool from the Khazanah Research Institute (KRI), a policy think-tank.
It also noted that the company had only used a dataset of five years from 2014-19 to train the algorithm, "which seems somewhat limited in comparison with the extensive databases used in global efforts".
Sarawak Information Systems could not be reached for comment on whether it had since expanded its database.
An analysis by KRI of cases in Sabah and Sarawak showed that judges followed the AI sentencing recommendation in a third of the cases, all of which involved rape or drug possession under the terms of the two states' pilot.
Some of the judges reduced the suggested sentences in light of mitigating factors. Others were toughened on the basis that they would not serve as a "strong enough deterrent".
'Opaque Algorithm'
Technology does have the potential to improve efficiency in the criminal justice system, said Simon Chesterman, a professor of law at the National University of Singapore.
But its legitimacy depends not only on the accuracy of the decisions made, but also the manner in which they are made, he added.
"Many decisions might properly be handed over to the machines. (But) a judge should not outsource discretion to an opaque algorithm," said Chesterman, a senior director at AI Singapore, a government programme.
Malayasia's Bar Council, which represents lawyers, has also voiced concern about the AI pilot.
When courts in Kuala Lumpur, the capital, started using it in mid-2021 for sentencing in 20 types of crimes, the council said it was "not given guidelines at all, and we had no opportunity to get feedback from criminal law practitioners".
In Sabah, Ismail appealed his client's sentence recommendation by the AI tool, which the judge followed.
But he said many lawyers would not mount a challenge - potentially condemning their clients to overly harsh sentences.
"The AI acts like a senior judge," Ismail said.
"Young magistrates may think it's the best decision, and accept it without question."