Associated Incidents
"I saw a naked photo of you."
Those were the chilling words a boy told a 14-year-old girl after she arrived on the first day of school this year in Spain, Spanish newspaper El Pais reports. AI-generated, deepfake nudes of her and other female classmates were being circulated online across four schools, stoking despair among the victims and their parents. There even was one case of alleged blackmail.
"Mom, they say there’s a naked photo of me going around," the girl told her mother, as quoted by El Pais. "That they did it with an artificial intelligence app. I’m scared. Some girls have also received it.”
The case sparked a round of parental outrage in the town of Almendralejo, and the local Juvenile Prosecutor’s Office is now handling the investigation, according to the report_,_ with 20 victims impacted so far and a group of suspected culprits behind the deepfakes already identified.
The deepfake app the culprits allegedly used is free to use and accessible to anybody with a smartphone — a dystopian use of AI tech that could continue to cause mayhem in schools and elsewhere.
Deepfaked
One of the girls, according to El Pais, was approached on Instagram by a boy who demanded money. When she turned him down, he sent her a deepfake naked photo of herself.
As the tech becomes more accessible, we should expect this kind of stuff to happen more and more. Meanwhile, in the US, the FBI sent out a warning earlier this year about deepfakes being used for extortion.
"The FBI continues to receive reports from victims, including minor children and non-consenting adults, whose photos or videos were altered into explicit content," the agency said in its statement.
To stem this tide, attorney generals from every single state in the US sent a letter to Congress earlier this month, urging a commission and action against the increase of AI-generated child sexual abuse material (CSAM).
The Washington Post reported in June that experts had noticed an increase in the dissemination of AI-generated CSAM, which is already hampering efforts to identify the many victims.
It remains to be seen if any upcoming regulations will help forestall the tide of deeply disturbing deepfake images, especially considering the proliferation of open-source generative AI and how long it takes for governments to agree on a solution.