Deepfake porn is running rampant, and a New Jersey high school is one of the latest epicenters.
Police are investigating after boys at Westfield High School circulated AI-generated images that showed the faces of their female classmates on naked bodies. It’s unclear if the perpetrators will be punished due to questions around the legality of their transgression: Neither federal law nor most state laws have anything on the books about deepfake porn. Plus, it may not be divulged if they are punished due to student confidentiality. In the meantime, a New Jersey state senator asked county prosecutors to take notice.
Widespread and rising: More than 90% of online deepfakes are pornographic, according to the image-detection firm Sensity AI. As of September, creeps had already uploaded 54% more deepfake porn videos this year than in all of 2022, Wired reported.
In other recent instances, a stranger virtually undressed a woman’s Instagram mirror selfie, a Wisconsin man threatened minors with nude deepfakes, and more than 30 girls in Spain were targeted by classmates with fake nudes.
How is this happening? Though big-name image generators like OpenAI’s Dall-E and Adobe’s Firefly have guardrails that stop users from making porn, countless online face-swapping and undressing tools that use publicly available software are still out there, making it hard to shut them down.—ML
Morning Brew delivers quick and insightful updates about the business world every day of the week from Wall St. to Silicon Valley.