Incident 186: Algorithm Assessing Risk Faced by Victims of Gender Violence Misclassified Low-Risk Cases, Allegedly Leading to Homicide of Women and Children in Spain

Description: In Spain, the algorithm that assesses recidivism risk in gender violence, VioGén, have critically underestimated the level of risk in a series of cases that ended in homicide of women and children since its first deployment.

Suggested citation format

AIAAIC. (2007-07-26) Incident Number 186. in McGregor, S. (ed.) Artificial Intelligence Incident Database. Responsible AI Collaborative.

Incident Stats

Incident ID
Report Count
Incident Date
Sean McGregor, Khoa Lam


New ReportNew ReportDiscoverDiscover

Incidents Reports

Fourteen of the 15 murdered in 2014 who had reported their aggressor for sexist violence had a police risk assessment "not appreciated » or «low», according to the latest report of the Observatory of Domestic and Gender Violence of the General Council of the Judiciary. The document studies 15 cases because it is dated December 1, so it does not include the two murders of yesterday in Valladolid (the victim had filed a complaint) and in Valencia (he did not). The fifteenth case analyzed -the woman from Madrid who, along with her daughter, died at the hands of her boyfriend and whose bodies appeared in November- is under summary secrecy, a circumstance that means that the report does not have been able to delve into it. The Report on deaths due to gender-based violence with previous legal proceedings in 2014 indicates that even in two of the three cases in which the protection order was in force -Villarejo de Salvanés, Cubillos de Sil and Berja-, the risk was assessed as "low" or "not appreciated". And that danger, whether evaluated high, low or non-existent, always lived in the 47 women in total (plus eight cases under investigation) murdered so far this year. Most did not file a complaint, but the 15 analyzed in the report did. They dared to come out of the intimacy of violence and told the State to protect them: social workers, doctors, policemen, judges... But they died. What went wrong? The 23-page dossier prepared by the Group of Experts of the Gender Violence Observatory of the CGPJ responds to this with its own information and that of the Government Delegation of Gender Violence of the Ministry of Health, Social Services and Equality. Part of the text was made public last week, but EL MUNDO has had access to the complete report, which reviews, one by one, 14 cases and points out some data on the fifteenth, the fatal story of A. P. and her little **A. ** On January 16, the mother and grandmother of the victims filed a complaint with the Dean of the Courts of Madrid for mistreatment of the three. The CGPJ indicates that, in the absence of receiving the full copy of the case, "it seems that the preliminary proceedings were dismissed", because the complainant was not located in the house indicated by the Court Number 16. The bodies were found two weeks ago, the car is under summary secrecy and the risk assessment is unknown. “More than a majority of victims had a medium to unappreciated risk assessment. Such assessment should not restrict protection options or should avoid a perception of impunity or disbelief”, writes the CGPJ. All the protection orders that the victims requested, eight, were granted. But the judges did not impose on any convict the telematic devices that alert of their approaches to the victims. "We are encouraging training in the matter, taking into account that there has been no homicide/murder with a bracelet." The dispensation from testifying against a family member, “in the wording that is maintained in art. 416 of the Criminal Procedure Law since the 19th century”, is behind many of the resignations, which happened in half of the cases. The will or the coercion - «I hurt myself because I fell», alleged M. J. in the court of Lugo; «It was a fight between the two. The mistake was mine," M said. F. in Arenys two months before she was murdered- provoked files and the resumption of coexistence. In other words, an increase in “vulnerability”. “Victims of abuse don't want their aggressors to go to jail, they want them to stop coming closer. That is why there are regrets when the lawyers transfer the criminal consequences of the complaint to the victims. «Thus -continues the report- the victims come to not want to testify or change their statements to protect their aggressors (...) Specialized training is essential (...) The Court must have elements to assess the objective situation of risk ». And the CGPJ launches a criticism of the waiver: «The wording of 416 generates a good part of the acquittals. And it creates distortions in gender violence. (...) In no other crime, the victim not only does not limit himself to forgiving his aggressor, but he blames himself for his own aggression, and is immersed in the so-called cycle of violence: aggression-denunciation-repentance-aggression (...) This procedural resource is a new instrument of domination at the service of the violent». Three of the killers had a history of assaulting other couples. The report argues that the aggressor must be monitored after serving the sentence, since "a majority of convicts continue to be obsessed with the feeling of ownership that the ex-partner belongs to them 'and to no one else', a fact that provokes permanent persecution. ». And he says that in one of the cases of this fortnight of reported blood, the quick trial took eight months to settle. But it was late: one day before it was celebrated, he killed her. Broken Shields, Dead Lives ---------------------------- ### 18 month hiatus, death in three Three months after it expired the restraining order lasting a year and a half, N. R. P.'s partner ended his life. She had denounced him on September 12, 2011 for degrading treatment and on the 14th, the Investigating Court Number 6 of Vilanova i la Geltrú (Barcelona) sentenced him to six months in prison and a ban on speaking to and approaching the woman unless 1,000 meters for 18 months. The sentence was carried out between September 14, 2011 and September 12, 2013. The abuser and the abused lived together again, but on the night of Kings this year, he killed her. "The victim dies not having any precautionary measure or procedure in process," remarks the CGPJ, so there was no risk assessment. "The resumption of coexistence with the aggressor after a complaint and having been sentenced greatly increases the vulnerability of the victim." ### Violence, psychiatry and risk On October 15, 2007, the man who seven years later would end up killing L. H. S. was sentenced for libel to six months in prison and four months of restraint. Later, the Mixed Court Number 1 of Amposta (Tarragona) sentenced him for mistreatment to six months in prison and not to approach him for two years, a stay that ended in February 2010. The Civil Guard and the Mossos followed up during that period. On November 11, 2012, the court ordered the internment of the man in a psychiatric center until the doctors saw fit. But on January 28, 2014, the woman was murdered "without having any precautionary measures in force." «The relationship continued after the conviction and removal. The victim was in a clear position of risk, with the aggravating factor of the aggressor's psychiatric illness. ### 'No protection measure' MDP also died without any protection measure or police risk assessment, at least in the report handled by the Gender Violence Observatory of the General Council of the Judiciary. “This is a situation identical to the previous ones. After the convictions of the aggressor, there is no protection measure in force and the relationship continued afterwards. It all started on March 3, 2009, when the Court of Violence against Women Number 1 of Elche sentenced the accused for mistreatment at home to six months in prison, deprivation of the right to own weapons for 16 months and the prohibition of communication and approach to the victim or his home for 14 months. The measure ceased to be in force in 2010. But four years and no protection later, on January 30, 2014, the woman was murdered. ### Background, pardon and file The Mixed Court Number 6 of Arenys did not consider proven the mistreatment of M. F. and on April 6, 2013 it dismissed. On September 29, the woman filed a complaint again and Court Number 1 ordered a removal. Two weeks later, the Mixed Court 3 decreed freedom without bail because the woman violated the protection, accepted the waiver to testify and denied the facts. "We want to withdraw the restraining order, because it was a fight and my mistake," he said. On October 11, Court Number 3 shelved the case and on October 22, Court Number 6 annulled the protection. That Court opened a trial for threats on January 10. But 20 days later, it was all over: "The victim dies without a protection order in force." The aggressor had a history of abuse with another partner. “After several complaints and removals, the victim is interested in lifting the measure, placing her in an obvious position of risk.” ### Kidnapping, pregnancy and no protection "I got the wounds when I fell," said M. J. to the magistrate of Court Number 1 of Lugo. She gave that judicial statement on October 18, 2013, four months before the man she had dared to report to the Police murdered her. It didn't even go to trial. Her husband, "after kidnapping her from her family home in Vilaboa", took her to Lugo. Meanwhile, on November 11, the inhibition of the case was agreed in the Court of Instruction Number 3 and on December 3 preliminary proceedings were initiated. However, just two and a half months later, on February 23, 2014, the victim who had denied the facts, probably due to coercion and fear, and also suffered a kidnapping, was murdered by her husband. I was pregnant. "No protection order had been requested," says the CGPJ, which does not reflect any type of police risk assessment. ### She does not testify against him, she files and kills her On April 5, 2012, M. D. F. saw how the Court of Violence against Women Number 2 of Barcelona issued a provisional dismissal order «because the perpetration of acts was not duly justified penal", according to the document of the Observatory of the CGPJ. The legal argument was that the woman had not ratified the complaint that she had previously filed with the Mossos d'Esquadra. It didn't matter that the victim had been living intermittently and apparently without high risk assessments for years with a husband who had been arrested for death threats. The report ensures that the man had "multiple police records and had entered prison for other crimes." On March 28 of this year, the aggressor killed the woman who two years earlier had not wanted to ratify her complaint or testify before the judge against him. ### Three complaints and a violent history M. B. E. denounced three times the one who ended up being her executioner, who had already been prosecuted for the alleged exercise of sexist violence against two other women in the past. The man was denounced by M. B. E. on February 7, March 2 and April 17, 2014. The Court of Violence against Women Number 1 of La Coruña dismissed one of them. On March 5, he initiated proceedings for injuries and family abuse and on April 22, for threats and violence. But he decreed dismissal and filed the case because "there was insufficient evidence to formulate an accusation based on law." Eight days later, before a final sentence was issued, he murdered the woman. Both had resumed living together and there were no precautionary measures, nor did the CGPJ indicate any risk assessment, although the victim had received legal and psychological assistance. ### Three complaints, 'low' risk and murder This story of reported violence begins in December 2006, when the Mixed Court Number 5 of Guadalajara did not grant a protection order and dismissed the case of H. B. Four years later, the woman She denounced her aggressor again and on November 5, 2010, the Court of Violence against Women Number 1 of Arganda del Rey (Madrid) repeated the decision with the dismissal and filing of the case. However, on April 1 of this year, that Arganda court did consider abuse and sentenced a protection order for her and a restraining order for him. The CGPJ report states that the Civil Guard carried out "three risk assessments ranging from low to unappreciated." A month after the judge's decision, on May 5, the man killed the woman in Villarejo de Salvanés (Madrid). It is one of three murders with a protection order in force. ### Protection order in force E.N. twice reported her attacker, who had an active restraining order when he killed her. On April 24, 2013, the Mixed Court Number 5 of Ponferrada sentenced the abuser to four months in prison and a distance from his victim "within a radius of 15 meters" for one year and four months for injuries. Seven months after that car, on November 7, the man was arrested for breaking the distance and assured that he was going to report the woman. In fact, there were complaints between the parties and a restraining order was being processed for the victim with respect to the aggressor. But on June 24 of this year, the man broke the restraining order he had set until August 15 and killed his victim in Cubillos del Sil (León), who was a beneficiary of the ATENPRO telephone, an immediate protection service. It was the second case of death with protection in force. ### The medical report that arrived late Long judicial history of V. F. that began in October 2009, when the Investigating Court Number 2 of Malaga granted him a protection order. Instead, a month later, Court 1 shelved the case "because the commission of the crime did not seem duly justified." And the same was done in December by the Court of Violence against Women. Years later, on July 22, 2014, the Investigating Court 14 of Malaga initiated proceedings and the Violence against Women Number 1 was inhibited, which, in turn, transferred the matter to Number 2. And in that bustle, seven days later, V.F. was murdered "without having any protection measure in force", nor proof of police risk assessment, according to the CGPJ. The report says that the victim went to a medical center, which issued an injury report. He sent it by regular mail to the court but it arrived late: a day after the murder. ### Order in force, unappreciated risk On June 26, 2014, the Mixed Court 22 of Berja (Almería) ordered protection for M. C. and a 200-meter distance for his aggressor. On July 8, the defendant was sentenced for harassment to six days of permanent location and a three-month stay. The report highlights that the complaint was filed in the Court instead of in the police station or the barracks, "so that it did not appear in the Viogen system (comprehensive Interior monitoring system), not being able to make a new risk assessment, which at the time of death it was unappreciated.” He killed her on August 2, with a restraining order in effect, just the day after he served his location-monitoring sentence. The CGPJ affirms that the fact that this location was controlled by the Local Police and not by the National or the Civil Guard, "also put the victim at risk as this information was unknown in Viogen." ### Acquittal and crime without a shield The man who murdered G. D. on October 3 of this year in Hospitalet de Llobregat (Barcelona) had several records of gender-based violence with a previous partner and for which he had been convicted. On June 6, 2008, the Court of Violence against Women Number 1 of Barcelona agreed to a protection order on his previous partner and imposed a restraining order on him that was canceled four months later. On May 5 of the following year he acquitted him. The aggressor changed his wife and on April 14, 2011, G. D. denounced him for "habitual abuse". But on September 1 of that year, the Court of Violence against Women Number 1 of Hospitalet decreed the provisional dismissal of the case. Life went on for a few years until on October 3, 2014, the victim was murdered "without any protection measure" and without risk assessed by the Security Forces. ### No trial was necessary: he killed her a day before The Observatory points to this case as an example of “excessive time” between the “urgent proceedings” and the trial date: eight months. I mean, one day after the murder. In March 2014, the victim reported her aggressor for domestic abuse, but later did not ask for removal or protection. The Criminal Court 16 of Barcelona did not issue any precautionary measure of protection. And he pointed out the "quick trial" for some events that had occurred in March for November 11. It was not necessary to summon the parties because, without a protection measure of any kind in force, or a police risk rating, according to the Observatory report, the aggressor killed the woman on November 10, one day before the trial. The CGPJ document underlines that "in Barcelona there is no specialization of Criminal Courts in matters of violence against women." ### The Prosecutor does not protect, the woman dies Less than a month ago H. J. was murdered in Gerona by the man she had denounced a few months before without obtaining anything from the Justice. “In the absence of receiving all the background information”, the report of the Gender Violence Observatory of the General Council of the Judiciary states that in February 2014, the woman denounced her partner for a crime of threats in the field of sexist violence. The Investigating Court Number 7 of Figueras initiated proceedings against the aggressor, but denied a protection order, which not even the Prosecutor's Office had requested. The victim returned to live with his executioner without protection measures of any kind or police risk assessment that appears reflected in the report. Until November 14 of this year. On that day, the man who had escaped a restraining order nine months earlier ended the life of H. J.

The murdered women who reported were assessed as 'low or no risk'

The Police will begin to apply this Wednesday the new protocol for assessing the level of risk of victims of gender violence, after the Secretary of State for Security has officially notified the instruction. As a novelty, the judicial authority will be alerted through an express diligence of those cases that are likely to evolve into more serious violence and even murder.

In this internal communication, to which Europa Press has had access, the reasons for the "improvement" of the forms and tools for the preparation of the police risk assessment (VPR) version 5.0, framed in the VioGén System, are explained.

The implementation of the new protocol occurs after the holding of several training days for the agents, according to sources from the Ministry of the Interior to Europa Press.

Detect especially serious cases or with minors

This is the fifth time that these mandatory forms have been updated since 2007. The aim is to "improve the prediction of recurrence of new episodes of violence", as well as "identify and alert" the judicial authority and the Ministry Prosecutor of the cases that have a risk "of special relevance" that are "susceptible of evolving into more serious violence", such as the murder of the woman.

The new police protocol also has the objective of detecting cases with minors in charge of the victim "in a possible situation of vulnerability", as indicated in the instruction of the Secretary of State for Security.

In the event that there are minors, it will be transferred to the judicial authority through a diligence and the practice of additional assessments will be advised. "Now these cases are going to be identified because they are minors in a possible situation of risk and, therefore, in another proceeding the judicial authority will be told that it is convenient to assess these minors," Rodríguez pointed out.

Another objective is to "clarify and simplify" some issues, such as the treatment of certain cases of gender violence, the application of mandatory police protection measures according to each level of risk, as well as the development of a personalized security plan. (PSP).

Risk Rating

Women who report gender-based violence by their partners or ex-partners in a Police or Civil Guard station cannot leave it without that assessment having been completed by an agent. The evaluation process takes about two hours to complete and, depending on the assessed risk, some police protection measures or others are imposed.

The agents, using different sources of information - victim, family records, witnesses, etc. - classify the risk as unappreciated, low, medium, high and extreme and, after that, it is incorporated into the VioGén System.

According to data as of February 28, 10 women remain at extreme risk, 179 at high risk, 5,183 at medium risk, 26,691 at low risk, and 25,071 at unappreciated risk, out of more than 57,134 active cases, although the system varies continuously. because its update is constant. The VPR associated with each case is always included in the police report.

"The expert, trained, proactive and rigorous intervention of these agents is essential in the assessment process," the instruction highlights. In it, the agents are summoned "not to ask direct questions of the victims, except in very specific cases and whenever there is a lack of very specific information that can only be collected by this means."

Specifically, it calls for "avoiding double victimization when collecting very sensitive and personal information from the victim or their aggressor" and "avoiding suggestions that lead to deviations or biases in the responses."

Likewise, the notification sent by the Interior indicates that the assumptions "of absence of complaint" must be made "expressly stated to the judicial and fiscal authority." It also makes special mention of the "many" crimes of gender violence committed through new information and communication technologies -digital gender violence-.

"Especially Relevant" Cases

A novelty of the new forms for risk assessment is that the judicial authority will be informed of the cases called "especially relevant" due to the risk associated with them through express diligence. The practice of additional evaluations in the judicial sphere will be recommended.

As the person in charge of the Gender Violence and Studies area, María Rodríguez, explained to Europa Press, in general, the cases that are classified as "of special relevance" will present, at least, a medium level of risk, and will concentrate above all at the top or end. "This identification of especially sensitive cases will necessarily lead to an increase in the level of risk," said the commissioner.

The new version of the police risk assessment system has fewer indicators than the previous one, which dates from 2016 -- around 30 items -- and "improves its formulation and the weight assigned to each of them," according to Rodríguez. "The new procedure includes some new forms that have been studied and learned from the previous ones, and with which they are more evolved," he told Europa Press.

Protection measures

Each level of risk is associated with specific police protection measures that will be mandatory, although any other that the police unit considers may be adopted.

For example, when it classifies a case with extreme risk, an intensive control of the aggressor's movements will be carried out "until he ceases to be an imminent threat" for the victim. The victim will be offered permanent protection and, if appropriate, there will be surveillance in the schools of the victim's children.

If the risk associated with the case is high, random control of the aggressor's movements and sporadic contacts with people he frequents or those around him will be carried out.

It also contemplates the transfer of the victim to a shelter, a relative's house or a different address, at least until the aggressor is located by the Police. There will also be frequent and random checks of the victim's home and workplace, as well as their children's schools.

With the new police assessment, those cases that include minors in charge of the victim who could be in a vulnerable situation will also be transferred to the judicial authority through a diligence. In these cases, additional assessments will be recommended.

"Now these cases are going to be identified because they are minors in a possible situation of risk and, therefore, in another proceeding the judicial authority will be told that it is convenient to assess these minors," Rodríguez pointed out.

New protocol to assess the risk of gender victims

This study describes the rationale, development, and validation of the intimate partner violence (IPV) police risk assessment forms of the VioGén System of the Spanish Ministry of Interior (VPR4.0 and VPER4.0), which promote greater predictive effectiveness and an improvement in the IPV law enforcement prevention. A validation study of the mentioned protocols is presented, including inter-observer reliability, estimated by the equivalence or inter-judge reliability method, while the convergent validity of these protocols was calculated with the RVD-BCN protocol. The sample consisted of 6613 new cases of IPV included in the VioGén System over a period of 2 months and which were longitudinally followed up for 6 months. The discrimination indexes are not only the summarized odds ratio (OR), area under the ROC curve (AUC), sensitivity, and specificity, but also the calibration indexes positive predictive value (PPV) and negative predictive value (NPV). The results show the suitability of using procedures which, in a coordinated manner, incorporate two risk assessment instruments, one for a first screening assessment and a second one to re-assess IPV danger situations on a regular basis. The values obtained are within the margins reported by different meta-analyses regarding this type of instruments, which supports their use for professional practice.

Validation and Calibration of the Spanish Police Intimate Partner Violence Risk Assessment System (VioGén)

As part of a program to curb feminicides, Spain built VioGén, an algorithm that assesses the risk faced by victims of gender violence. It remains a work in progress.

In the early morning of 24 February 2018, Itziar P., a psychologist living in the Spanish city of Castellón, went to a police station to report threats from her husband, Ricardo C..

In audio recordings she made with her mobile, the husband could be heard saying: “We will all end up dead and me in jail”, or “I will take away from you what you love most”.

According to Itziar P., Ricardo C. had also broken in pieces the buggy of their smaller child (Martina, 2 years old) and slapped the older one (Nerea, 6), when the two were under his custody.

The police officer asked Itziar P. a set of questions and fed the answers to VioGén, a software that helps the Spanish police estimate the risk of recidivism in gender violence. The officer issued a report in which that risk was deemed as low.

Critical failure

In the following days, both Itziar P. and Ricardo C. were called to declare in court. She asked that he be forbidden to visit their children, but the judge denied the request, based on the low risk estimation made by the police, among other reasons.

Seven months later, on 25 September 2018, Nerea and Martina where sleeping at Ricardo C.’s place. In the early morning, he killed them “with cruelty” and threw himself out of a window.

Itziar P.’s story was shocking. Why was the case deemed as low risk? VioGén had missed its goal of supporting police personnel in assessing the risk of new assaults, and so assigning the right level of protection. Since the software was first deployed in 2007, there has been a series of “low risk” cases that have ended in homicide of women or children.

Better than nothing

The program is by far the most complex of its sort in the world. It has reasonable performance indexes. Nobody believes that things would be better without it – except the far-right, which makes spurious claims that it helps women report innocent men.

But critics point out some flaws. Few police personnel are educated in gender-based violence. Some of the others blindly rely on the software’s outcome. Moreover, the program may have systematically underestimated risk. Some victims’ organizations believe that the possibility of a low risk score is nonsense. Reporting to the police is a high risk situation in itself, they say, because abusers perceive it as a challenge.

As of January 2020, 600,000 cases had gone through VioGén. About 61,000 of them were considered active, meaning they were being followed-up by the police (the system is designed to check periodically on women until they are considered safe).

Reporting an assault

When a woman goes to report an assault from an intimate partner, she triggers a process that takes at least a couple of hours. First, the police agent goes through an online form with her. The agent ticks each of the items of the VPR form (from the Spanish initials of “Police Risk Assessment”) as “present” or “non present”. There are 39 items in the latest published version of the form (the VPR4.0). Agents can also rely on police databases, witnesses and materials proofs.

Questions explore the severity of previous assaults (for example, whether weapons were ever used); the features of the aggressor (jealous, bully, sexual abuser, unemployed, drug addict, etc.); the vulnerability of the victim (pregnant, foreign, economically dependent, etc.); and aggravating factors (like assaults by other men).

Answers are thrown automatically into a mathematical formula that computes a score, measuring the risk that the aggressor repeats violent actions. This quantitative approach is different to the one used in DAS-H, the British equivalent of VioGén. The latter is basically a paper check-list that helps agents to get an idea of the situation.

Keeping the score

In theory, Spanish agents can increase the score by hand, if they appreciate a higher risk. But a 2014 study found that they stuck to the automatic outcome in 95% of the cases.

The formula used in VioGén is a “simple algorithm,” according to Juan José López Ossorio, a psychologist who has been in charge of VioGén from its early stages, in a written statement to AlgorithmWatch. The algorithm gives more weight to items that empirical studies have shown to be more related with recidivism, Mr López Ossorio wrote. He declined to disclose the exact formula.

Once a case’s score is established, the agent decides on a packet of protection measures associated to that level of risk . For the lowest scores, agents will discreetly check on the woman from time to time. For the highest, the police will give the victim an alarm button, track the aggressor’s movements, or guard her house. The agent also sends the form and risk score to the prosecutors and judges that will see the woman’s case.

After the first report, the police meet again with the woman to fill in a second form, in order to assess whether the situation has worsened or improved. This happens periodically, more or less frequently depending on the risk level. Police stops following up only if judicial measures are not pursued and the risk level falls below medium.

VioGén is one of the outcomes of a pioneering law on gender-based violence that Spain approved in 2004, ten years before the Council of Europe adopted a common framework on the subject, the Istanbul Convention. Nowadays, the software is used by the main Spanish police forces (Policía Nacional and Guardia Civil) and by hundreds of local police forces (except in Catalonia and the Basque Country that have independent police bodies).

The best available system

VioGén is the best device available to protect women’s lives, according to Ángeles Carmona, president of the Domestic and Gender-Based Violence Observatory of the Spanish General Council of the Judiciary (CGPJ).

She recalls a case she saw in a court in Seville, of an aggressor that had a high-risk of recidivism, according to VioGén. A control wristband was applied to the man. One day, the police saw that the signal of the wristband was moving fast towards the victim’s home. They broke into it just in time to prevent him from suffocating her with a pillow.

It’s impossible to know how many lives have been saved thanks to VioGén, according to Antonio Pueyo, a professor of psychology at the Univeristy of Barcelona who has advised VioGén from the beginning.

However, a 2017 study by Mr López Ossorio and his team tried to measure how good the protocol was. They found that VioGén’s Area Under the Curve (AUC), a widely-used measure of performance for predictive models, stood between 0.658 and 0.8. An AUC of 0.5 is as good as a coin’s toss and an AUC of 1 means the model never fails. Cancer screening tests are considered good when their AUC is between 0.7 and 0.9. In other words, VioGén works.

“Comparing with what is around and within the existing limitations, VioGén is among the best things available”, says Juanjo Medina, a professor of quantitative criminology at the University of Manchester that has compared instruments of assessing the risk of intimate partner violence.

Spain is the only place where victims can be followed-up across different regions. Close to 30,000 police officers and other agents across the country had access to VioGén in 2018.

However, the cases that have slipped through the cracks of VioGén have raised concern. The latest one happened in February 2020, when a 36-year-old woman and mother of two had her throat cut by her former partner, who then threw her body in a container in the town of Moraira. The two had been registered in the VioGén system after the police reported him for attacking her, but the case had become inactive after a judge cleared him.

False negatives

In 2014, the newspaper El Mundo published a leaked document from the General Council of the Judiciary that showed that 14 out of 15 women who were killed that year, having reported their aggressor before, had low or non-specific risk (the classification used for any person reporting a threat to the police).

Some critics say that low risk should not even be an option. Reporting is a maximum risk moment for a woman, according to Carme Vidal Estruel, spokesperson of Tamaia, an association that helps victims in Barcelona. She says that the situation is akin to divorcing or becoming pregnant, both moments in which the aggressor realizes that he is loosing grip on the victim.

Another widespread criticism is that few agents among those who should validate the computer's outcome receive enough training in gender issues. Some VioGén items are embarrassing, like those related to sexual violence, humiliation, or intimate messages on mobile phones.

Agents should ask circular questions (instead of blunt, direct questions) and avoid transmitting the feeling that the object of investigation is the woman, according to Chelo Álvarez, president of Alanna, an association of former victims in Valencia. Ms Carmona of the General Council of the Judiciary recalls a woman that reported her husband for robbing her car’s keys. She was so scared that she could not say anything else. The day after, the man killed her.

Few agents are aware of these nuances. In 2017, there was a total of 654 agents in all Spain belonging to the Women-Children Teams (EMUME) of the Guardia Civil. That is much less than one for every police station.

Ignored requirements

This is very different from what the 2004 law that created VioGén required. According to it, cases should be dealt with by an interdisciplinary team including psychologists, social workers, and forensic doctors.

This team should go into psychological aspects that the VioGén form does not cover. Moreover, it should carry out a forensic assessment directly on the aggressor. The current system is tantamount to evaluating how dangerous is a person without ever talking with him, critics points out. Several teams were created after the law was passed in 2004, but the process was cut sharply by the austerity following the 2008 financial crisis.

Mr Pueyo, the psychology professor, acknowledges some of the criticism, but believes that VioGén should be judged for its ability to predict new assaults, not homicides, because these events are very rare. The probability that a women be killed after reporting is about one in ten thousands, according to Mr López Ossorio.

However, the Istanbul Convention requires precisely to reduce the risk of death. And not only of the women, but also of their children. Overlooking the risk for children is another criticism VioGén faces.

The convention entered into force in Spain in 2014, but VioGén forms were not changed accordingly until Itziar P.’s case occurred in 2018, according to her lawyer.

VioGén 5.0

A new protocol was put in place in March 2019, the fifth big change VioGén has gone through since its first deployment in 2007. Now, the program identifies cases “of special relevance”, in which the danger is high, and cases “with minors at risk”.

This is done through the “dual evaluation procedure” of the new VPR form (VPR5.0-H), Mr López-Ossorio explains. Two calculations are carried out in parallel: one related to recidivism and a new one related to lethal assault.

Depending on the outcome of the latter (called “H-scale”), the risk score can be increased automatically. Moreover, the case can be signaled to the prosecutors and judges as being “of special relevance”.

Mr López-Ossorio declined to disclose how the H-scale was built, but wrote that it was based on a study his group carried out throughout four years, to find which factors were specifically related to cases that end up in homicides.

The new protocol seems to have triggered a major shift in the risk scores of VioGén. Passing from VPR4.0 to VPR5.0-H, the number of extreme risk cases rose and those of high risk almost doubled, according to Mr López Ossorio.

As the president of Valencia’s former victims association Ms Álvarez puts it: “Things are improving, but they should go faster, because we are being killed”.

In Spain, the VioGén algorithm attempts to forecast gender violence

We present the results of 7 months of work of Eticas and Ana Bella Foundation with the available data and affected women and other stakeholders. As mentioned above, it is a part of a broader external auditing project where Eticas, in collaboration with other civil society organizations, reverse engineers and assesses the impact of algorithms in different fields. With this External Audit project we aim to develop methodological tools to externally audit automated risk assessment systems in the absence of access to the code, input, output, and administrative data to provide methodological tools to community organizations for externally auditing algorithms with social impact and advocating for policy change. In this way, we seek to support bottom-up algorithmic auditing movements conducted by third-party organizations and end-user groups.

When we started the External Audit of VioGén, we had concerns around transparency, independent oversight, accountability, end-user engagement and the transition to ML. After conducting the audit, we can confirm that:

VioGén is not transparent. We could not access any system data or information beyond what has been produced by experts involved in the definition of the system. Neither external auditors nor women groups have any kind of access to the VioGén data. For a publicly-funded, high-impact system like VioGén, this is unacceptable.

VioGén has not been independently assessed or audited. The publicly available resources and surveys regarding the validity and desirability of VioGén have been conducted by individuals who either work for or have vested interests in the ministry and police forces. External auditors or researchers have no official or public path to access the data, and access seems to be provided by the Ministry at their discretion.

VioGén is not accountable. While the Ministry of the Interior sees VioGén as a recommender system, the high rates of prima facie acceptance of the algorithmic results (95%) points to an automated system, which should be held to further scrutiny as per the Régimen Jurídico de la Función Pública.

VioGén does not engage end-users. In our fieldwork we have found that women and women organizations have never been approached about the system, neither in its design phase nor later on during the different decisions on how to alter the VioGén system. Also, we have found that 80% of the women who have used the system have negative comments about it. They are not informed of what it does or how it works, which leads to distrust.

The VioGén transition to ML raises new questions. Even though the literature explores the process of transitioning to a ML version for VioGén, the nature and extent of the collaboration between SAS and the ministry has not been publicly disclosed. While the lack of a public and open debate on this process would in itself be concerning, the fact that the technical evolution of the system is being decoupled from state of the art research and oversight is bound to lead to further problems.

The auditing process has also allowed us to go beyond our initial concerns to identify new issues that deserve attention.

Firstly, we want to highlight that through this audit, we have found that the VioGén system adapts the clustering of risk assessments to the resources available. This means that the system only gives the number of “extreme” risk scores it can afford, and so funding cuts have a direct and quantifiable impact on the chances that women will receive effective protection after seeking police protection. As the number of VioGén cases is growing each year, there are more women receiving police protection. While in 2015 around 3,000 women received police protection -with medium, high, and extreme risk scores-, in 2021 this number rose up to almost 9,000 women. Yet, there is still a big gap between women who receive police protection over those who do not, despite reporting the case of gender violence to the police. In terms of calibration, we are concerned by the number of cases that the VioGén system “discards” by giving them an “unappreciated” risk score. As it is currently designed, the risk score given by VioGén is not only determined by the objective facts that the questionnaire intends to unearth, but also the overall distribution of gender-violence cases, which is determined by the available resources. Therefore, in 2021, only 1 out of 7 women who reached out to the police for protection actually received it.

This is even more serious if we take into account the barriers we have identified to accessing VioGén, which are one of the reasons why only 21.7% of women victims of domestic violence seek protection. These figures mean that only 3% of the women who are victims of gender violence receive a risk score of “medium” or above and, therefore, effective police protection.

Secondly, we have identified that not having children has a significant negative impact on how extreme risk cases are perceived. Our data analysis shows that women who were killed by their partners and did not have children were systematically assigned lower risk scores than those who did, with a recall difference between groups of 44%.

We would also like to call into question the representativity of the AUC value of the H-scale claimed by the lead researchers of VioGén. While it is true that with the data available the H-scale is capable of identifying extreme risk cases that can lead to homicide, the fact that only 1 in 4 cases of homicide occur after a previous police report indicates how the majority of homicide victims will remain unprotected, even with the deployment of VPR5.0-H. This means that even though VioGén is now better equipped to identify certain cases of extreme risk, most homicide cases will remain unaddressed.

Fourthly, we have observed that VioGén is, in practice, an automated system with minimal and inconsistent human oversight. Police officers only increase the risk observed in 5% of cases, a figure that goes down when they feel overworked. This is highly problematic, as a non-accountable implementation of human oversight (“human in the loop”) can lead to explainability and transparency problems. If police officers do not have clear instructions on when and how to intervene, their role can re-introduce bias into the system, and women may receive different scores depending on who files their case. Assessing the role of human oversight over time should be part of any audit and transparency efforts.

While our sample is not representative of the broader population of victims and lawyers and therefore our findings are not generalizable, our fieldwork raises important questions that need to be studied more systematically, and ideally addressed at the institutional level.

The External Audit of the VioGen System

Lack of transparency, underestimation of psychological violence, “rigid” questions that do not allow explanations or evaluations that take into account the police resources available when assigning a level of risk to the victim. These are some of the "problems" that an external audit has found in the VioGén algorithm, an automatic system used in police stations to rate the personal risk that each complainant of gender violence has.

“One of the main concerns about the VioGén algorithm is that approximately 45% of cases receive a risk rating of 'unappreciated'. In the context of gender-based violence, the category of "without risk" is already a very controversial issue, since only the fact of taking the step of denouncing can lead to violent reactions on the part of the aggressor", details the investigation, Prepared by the Ethics Foundation, specialized in algorithmic auditing, and the Ana Bella Foundation, a network of women survivors of sexist violence. "This lack of appreciation of the risk leads us to think that there are factors that are not yet being taken into account," they warn.

The audit includes an analysis of public data from the Ministry of the Interior, that VioGén has been carrying out since 2007. Added to this is a qualitative study with interviews with 31 whistleblowers and 7 specialized lawyers, as well as a quantitative approach that deepens VioGén's risk ratings with 126 women who were murdered after filing a complaint against their aggressor.

This second part of the investigation has been carried out externally given the Interior's refusal to provide access to VioGén's databases to carry out an internal audit, something that Eticas points out that it offered to do for free "on several occasions since 2018”. Sources from the Ministry have confirmed to that these requests were denied and affirm that the report “lacks academic rigor by basing its study and its conclusions on an insignificant statistical universe of only 31 interviews compared to more than five million evaluations. of risk carried out since 2007”.

Risk rating for gender-based violence

VioGén (acronym for Comprehensive Monitoring System in Cases of Gender Violence) comes into action when a victim of sexist violence goes to the police station to file a complaint. It is based on a predetermined questionnaire that the agents fill out with the answers of the complainant. Based on them, the algorithm evaluates the risk that her aggressor will attack her, classifying it as "unappreciated", "low", "medium", "high" or "extreme". From the “medium” risk, the complainant has the right to police protection. Agents can modify the assigned risk level, but only to increase it. However, the audit reflects that in 95% of cases they do not.

It is in this first evaluation where the investigation detects many of the system's problems. “80% of the interviewees reported different problems with the VioGén questionnaire. This means that the quality of the data entered into the algorithmic system could be compromised at this time, generating sources of bias and misrepresentation within the system. Among the failures pointed out by the complainants is the lack of transparency, since many were not informed of the risk assessment that VioGén assigned them.

The context surrounding the completion of the questionnaire is also problematic. One of the participants remembers it as "a murky moment with absurd questions where mistakes are made when filling out the questionnaire", describing the situation she experienced as "surreal". Respondents state that it was difficult for them to remember everything that happened at the time of the interview, organize their thoughts and provide detailed answers to VioGén's questions. The foundations recall that "many women who suffer gender violence arrive at the police station and file the complaint right after a violent incident", so "they are in a state of shock".

Added to this is that the system only supports binary responses. If the complainant is unable to offer one, it is the agents who must interpret what they should enter into the system. The interviewed lawyers specializing in gender violence agree that the questions are “rigid” and do not allow explanations. One of them pointed out that the way in which they are formulated means that the educational level of the complainant is also key: “The level of education makes it easier to understand what is asked for and explain how she feels and suffers, as long as it is not serious abuse. and is blocked or terrified.”

Of the 31 women interviewed for the research, 15 negatively evaluated their experience with vioGén; 10 indicated both negative and positive aspects and 6 positively rated their overall experience. Another point on which victims and lawyers agree is that "VioGén underestimates psychological violence and the newer forms of non-physical violence (such as harassment through social networks), placing the emphasis on physical violence." “There is no need for a beating, or an assault, for the risk to exist. It seems that the parameters forgot the psychological abuse”, explains another of the lawyers.

Interior denies that the questionnaire presents problems and reminds that it is constantly being improved. “The police risk assessment system (VPR) in the VioGén field is an important support instrument in the fight against gender-based violence that has proven its usefulness since its implementation 15 years ago, thanks to the scientific validation of each of the its indicators”, sources from the Ministry expose to this medium. "It is necessary to specify that it is a support for the police evaluation work carried out by an expert in the field, also based on another set of factors and criteria," they add.

Insufficient protection in 55 cases

The quantitative analysis of the 126 cases of murdered women has detected that 55 of them "received a protection order that turned out to be insufficient" by VioGén. At this point, the investigation detects a bias in its parameters, since "the murdered women who did not have children had automatically been assigned a lower level of risk than those who did."

The audit of the automatic part of the system has also revealed that VioGén adapts the estimated risk for each victim based on the police resources available. "This means the system only gives the number of 'extreme' risk assessments it can afford, so funding cuts have a direct and measurable impact on the chances of women receiving effective protection after seeking police protection." .

The system only gives the number of “extreme” risk assessments it can afford, so funding cuts have a direct and measurable impact on the chances of women receiving effective protection

The foundations highlight that despite the active cases in VioGén growing year after year, in 2021, "only 1 in 7 women who went to the police in search of protection obtained it." Since 2015, only 3% of victims obtained a risk score of “medium” or higher and, therefore, police protection.

Algorithmic transparency

VioGén was developed in 2007 by the Ministry of the Interior and has been refined ever since. It was a pioneering system for integrating all the information on victims of sexist violence into a single platform, as well as unifying their protection throughout the territory. This has made it "the risk assessment system with the most registered cases in the world" with more than 3 million, collects the audit.

However, the details about how it works or why it assigns one level of protection and not another are opaque. This situation often goes unnoticed due to the apparent mathematical neutrality of Artificial Intelligence, despite warnings from experts: "Algorithms sometimes give a false impression of objectivity that crushes people's rights," he explained in an interview with this medium Carlos Preciado, magistrate of the TSJ of Catalonia. “Most of the VioGén studies have been carried out by the same researchers who contributed to its development and by people who work and/or have vested interests in the Ministry and the police forces,” Eticas highlights in this case.

Most of the VioGén studies have been carried out by the same researchers who contributed to its development and by people who work and/or have vested interests in the Ministry

The Government has committed to promoting the transparency of this type of Artificial Intelligence systems used in the administration. “The quality of the data provided and its accessibility will be improved, promoting a data-oriented culture, using transparent and explainable algorithms, strengthening the relationship between the Administration and the citizenry,” it states in the National Artificial Intelligence Strategy approved in 2020 . The general budgets for 2022 include an item of 5 million euros for the creation of an [agency specialized in supervising algorithms]( .html) and its possible biases.

However, the workings of most of the algorithms it uses remain secret. In addition to VioGén, another example is BOSCO, which regulates access to social assistance for paying electricity to users in a vulnerable situation. In this case, the Government is suing the Civio pro-transparency foundation after [refusing to give access to its source code]( -of-the-source-code-of-the-application-that-grants-the-social-bonus/) after a resolution of the Transparency Council that urged him to reveal its details.

Victims denounce failures in VioGén, the algorithm against gender violence

Spanish algorithm leaves domestic violence victims out in the cold: For the past 15 years, Spain has been using an algorithmic system called VioGén to help the police assess the risk women face when they file complaints of abuse. But the system has severe flaws that lead to women’s risk being ranked too low and without the appropriate response from the authorities, according to a new report by Eticas Consulting, an algorithmic auditing company, and the Ana Bella Foundation, an organization campaigning against gender violence.

Meet VioGén: Launched in 2007, the system was created by the Spanish Ministry of Interior. The “Integral Monitoring System in Cases of Gender Violence” (the VioGén System) is a web application designed to help authorities coordinate actions to protect women and their children from gender violence. The idea was to standardize a set of questions that the authorities ask women when they make an official complaint in an effort to treat them fairly and offer better treatment.

How it works: VioGén uses classical statistical models to perform a risk evaluation and offers women a risk score, which determines how much help they will receive.

Here’s where it goes all wrong: “We could see in the media that every so often you’ll get a woman that was killed and their VioGén risk had been ranked low,” said Gemma Galdon-Clavell, Eticas Consulting’s CEO. Her audit found that in 2021, only 1 out of 7 women who reached out to the police for protection received help, and only a small minority of women received a risk score of “medium” or higher, which would qualify them for police protection. “If you don’t have children, you may get a lower risk score, which is also concerning,” Galdon-Clavell added.

A high bar: The proportion of women who report their partners for abuse is likely only the tip of the iceberg of all domestic abuse and intimate partner violence cases. Women risk a lot by going to the authorities, and often do so in situations of extreme duress. “I was trembling, I couldn’t find the words to explain years of abuse, mockery, scorn and contempt from him, it wasn’t easy and the police officer who took my statement was not trained to deal with a person who had suffered gender violence,” one women who participated in Eticas’s audit said.

The VioGén questionnaire was a baffling experience. “I was totally lost in these questionnaires between nerves and crying. There was no one with me to explain the questionnaires. The policeman gave me the paper and left me alone and every now and then he came to pressure me to finish quickly,” the woman said

‘Nightmare’ scenario: “It’s like a bad dream. A nightmare. And to this day I don’t remember those questions, I just remember that I felt very bad and unprotected. I didn’t even know there was a score,” another woman who participated in the audit said.

Humans not in the loop: Etica’s audit found that VioGén is not very transparent and has very little human oversight or accountability. In 95 percent of cases, police officers stuck with the risk score the system offered.

Machine-learning red flag: The ministry has said it is considering introducing machine learning to VioGén. This is a big red flag to Galdon-Clavell. “It’s quite clear that human oversight is not working,” she said. Using machine-learning would lead to the discrimination patterns to be “made even worse without any kind of oversight,” she added.

Spain’s interior ministry responds: The risk assessment system “is based on the study of 600,000 real cases and has been subject to a constant and permanent process of evaluation and improvement since 2007, with the collaboration of universities and study centers of recognized academic prestige,” a spokesperson for the Spanish interior ministry said. The Eticas report “lacks academic rigor by basing its study and its conclusions on an insignificant statistical universe of only 31 interviews compared to the more than five million risk assessments carried out since 2007,” they continued.

How to fix the system: Involving women in the process and explaining how the system works would be a good start. Women who used the system told Decoded that they would have liked support from a specialist in gender violence, psychologists or doctors. “Give security to the victim. Give empathy. Don’t judge. Be kind and supportive,” a second woman said. Read more from me here.

AI: Decoded: Spain’s flawed domestic abuse algorithm — Ban debate heats up — Holding the police accountable