Associated Incidents
OpenAI's latest image-generating 4o model is surprisingly good at generating text inside images, a feat that had proved particularly difficult for its many predecessors.
And that makes it a powerful tool for generating images of fraudulent documents, as users have found.
Case in point, Menlo Ventures principal Deedy Das tweeted a photo of a fake receipt for a lavish meal at a real San Francisco steakhouse, as spotted by TechCrunch.
"You can use 4o to generate fake receipts," Das wrote. "There are too many real-world verification flows that rely on 'real images' as proof. That era is over."
Doc Holiday
The image itself, at first blush, is pretty convincing and includes a breakdown of a multicourse meal, a correct subtotal, and even a tip calculation.
Another user even managed to edit the image further by adding a realistic filter and food stains — the perfect way to commit expense fraud, if you were so inclined.
And that's the tip of the iceberg. Das also found that 4o was happy to generate fraudulent prescriptions for controlled substances like Zoloft.
Fraud Squad
The development highlights how far AI-powered image generators have come. Previous models infamously struggled with recreating letters, often resulting in garbled shapes and unintentionally hilarious phrases.
Beyond faking expenses for lavish meals, OpenAI's increasingly canny ability to generate fake documents could open up the door for everything from phony tax forms and bank cheques to fake IDs and birth certificates.
Whether our ability to detect this storm of faked documents remains to be seen. But given AI companies' current efforts, it's not looking good. Guardrails like appended metadata or watermarks that divulge whether an image was generated by an AI are easily overcome.
Even before the advent of powerful AI-powered image generators, a 2015 survey found that 85 percent of respondents admitted to lying to get reimbursed with more money. Many of these kinds of fraud cases fall through the cracks due to a lack of internal controls and flawed accounts payable processes.
In other words, you can no longer believe anything you see online.