Associated Incidents
Apple's "Apple Intelligence" update, introduced in late 2024, is under fire for making scam messages appear legitimate. The AI-powered features, designed to summarize and prioritize notifications, have flagged fraudulent messages as priority alerts on iPhones, iPads, and Mac computers.
Apple Intelligence condenses multiple notifications into a single summary and highlights urgent messages, such as emails from colleagues or airlines. However, users have reported that the AI struggles to differentiate scams from genuine alerts.
For instance, an iPhone user saw a scam email claiming he owed $465, prioritized by the system. "Having an AI tool onboard millions of devices enabling scams like this is going to catch people out and cost them money," he said.
Other users on social media have shared similar examples, with UQ professor Jeremy Howard criticizing Apple for flagging scams as a priority. Another user highlighted a fake USPS message summarized by the AI to appear legitimate, as shown in the screenshot below.
These incidents show how the feature could inadvertently increase the effectiveness of scams.
Experts warn that the feature's reliance on summarization removes important context, making it harder for users to identify fraudulent messages. Daswin De Silva, an AI expert at La Trobe University, pointed out that the lack of detail forces users to reevaluate messages, creating more work. He also criticized tech companies for rushing AI features without addressing potential risks.
Apple's Planned Update
Apple told the BBC it plans to update the feature to clarify when summaries are AI-generated, aiming to reduce confusion. Until then, experts recommend manually verifying flagged messages and approaching AI summaries with caution.