Students at a New Jersey high school are terrified after learning that one of their classmates allegedly used AI to create sexually explicit images of them without their permission.
Westfield High School student Francesca Mani said that she learned that a male student had used an original picture of her and an artificial intelligence app to create the pornographic images, New York Fox affiliate WNYW reported. The images were allegedly created over the summer, but students only learned about them in October.
“I never thought being like a student, AI didn’t even come to my mind,” Mani told the station. “I just thought it would be like creeps on the internet.”
She reported the incident to the school principal and has learned that it was a group of boys using around a dozen images of girls to make the AI-generated pornography, according to WNYW. The boy was reportedly suspended for a few days but has since returned to the school.
“All the other girls agree with me, they don’t want him in this school,” Mani said. “They are very scared.”
In an email, the school principal said an investigation was ongoing, according to CBS New York affiliate WCBS.
“At this time, we believe that any created images have been deleted and are not being circulated,” the email said. “This is a very serious incident. We are continuing to investigate and will inform individuals and families of students involved once the investigation is complete.”
The school has reportedly made counseling available, and the Westfield Police Department is also involved, the email said. The principal also included a warning to parents.
“I wanted to make you aware of the situation, as, in addition to harming the students involved and disrupting the school day, it is critically important to talk with your children about their use of technology and what they are posting, saving and sharing on social media,” the email said. “New technologies have made it possible to falsify images and students need to know the impact and damage those actions can cause to others.”
CBS also reported that Westfield Superintendent Dr. Raymond González released a statement:
All school districts are grappling with the challenges and impact of artificial intelligence and other technology available to students at any time and anywhere. The Westfield Public School District has safeguards in place to prevent this from happening on our network and school-issued devices. We continue to strengthen our efforts by educating our students and establishing clear guidelines to ensure that these new technologies are used responsibly in our schools and beyond.
The law surrounding pornographic “deepfakes” such as the ones reported by Mani is not settled. Rep. Joe Morelle, D-N.Y., proposed legislation in May that would make non-consensual sharing of altered or “deepfake” explicit images online illegal, and create legal recourse for victims targeted by such images.
Justice Department officials have said that AI-generated sexually explicit images of children are illegal under federal law.
Neither school officials nor representatives for Westfield police immediately responded to Law&Crime’s request for comment.