Welcome to the

AI Incident Database

A new accessory is joining the traditional cap-and-gown ensemble as droves of graduates line up to accept their diplomas this spring. Clutched in the hands of students waiting to cross the stage is a QR code, either on paper or a phone, that triggers an announcement of their name.

But the voice playing over the sound system isn't the familiar hum of a principal or faculty member. It's AI.

Artificial technology is spreading all over, including commencements. School officials say the tech can help ensure students names are pronounced correctly and also make ceremonies run more quickly.

But integrating the technology in graduation ceremonies hasn't been without issue. A platform malfunction during the recent commencement for Glendale Community College in Arizona caused multiple graduates' names to be skipped. The crowd erupted in hearty grunts and boos after the error.

The wave of automated graduations, and subsequent pushback, isn't isolated to universities --- high schools are reckoning with the changing landscape, too. Officials at Tassel, one of the largest providers of AI graduation name services, said the platform has doubled the number of high school users since 2023.

At a Virginia high school, the fallout was swift.

Arlington County's Washington-Liberty High School last month announced plans to use AI for its upcoming commencement in June. Officials explored using the technology to support accurate pronunciation and efficiency, Christina Arpante, a district spokesperson, said in a statement.

But at a school board meeting, high school parent June Prakash dogged the use of AI as valuing efficiency over authenticity.

"Graduation is one of the most meaningful moments in a student's academic journey. It's a moment where their name spoken aloud recognizes years of effort, growth and identity," she said. "Turning that moment into an AI moment makes it feel standardized, impersonal rather than authentic and human."

Weeks after its initial announcement, the high school axed its AI plan, citing negative feedback from students.

So, this year's commencement at Washington-Liberty High will look just like the 99 that have come before it, with faculty members reading the names of each of the roughly 700 graduates who cross the stage.

The school is still focused on "creating a meaningful graduation experience that reflects what matters most to students while also ensuring names are read accurately and respectfully," the district said in its statement.

Other schools in the D.C. region have experimented with AI, some with success.

This year will be the second in a row that Alexandria City Public Schools will use the AI platform Tassel to read graduates' names. "We got rave reviews last year," said Michael Burch, the district's lead administrator for operations and student support. "Not a name was misspoken. Every name was done correctly."

Through the Tassel platform, students submit the spelling of their name, which generates an audio reading. They can then listen to the clip and confirm its accuracy, or ask to generate a new clip. After three tries, Tassel asks students to submit an audio recording of their name. That recording is then used by broadcast voice actors to create a sound bite for the commencement. Tassel data shows around 14 percent of names are marked as inaccurate by students before ultimately being submitted to human voice actors.

Using Tassel has also helped the Alexandria district reduce the amount of time spent reading names, Burch said, while still giving students their shining moment to cross the stage.

Alexandria has the largest public high school in the commonwealth, and the district has frequently struggled to finish its graduation ceremonies during its allotted time at George Mason University's EagleBank Arena.

Reserving more time at the arena isn't plausible, Burch said, because of its five-figure price tag and lack of availability. On the other hand, Tassel's name-reading feature cost the district only around $4,000.

Last year, Alexandria City High School graduated 983 seniors, which Burch estimated required 12 to 14 names to be read every minute to stay on schedule. Names will have to be read even faster this year, with the potential for over 1,000 students to walk at commencement.

With that many names to get through, human error could take away valuable seconds from a graduate's walk across the stage, Burch said.

That's why the district continues to turn toward automation.

AI name-reading software is relatively new in the graduation space, having shown up within only the past few years with companies such as StageClip and Tassel.

Originally called MarchingOrder, Tassel had provided services for commencements for around 20 years before adding the AI name offering. It long used human actors to record graduates' names. But Chase Rigby, the company's chief executive, said with modern technology, that didn't make sense anymore.

The platform's library of human readings eventually evolved into a type of phonetic library used to train Tassel's AI model, and Rigby said its accuracy has improved as the application is used by more students.

Rigby said Tassel pulls information on the cultural origin of names to better understand how to pronounce each syllable, which helps reduce errors --- especially on names that aren't Anglo-based.

"I don't know what could be more of a human application of AI," he said.

For the Alexandria school district, whose 16,300 students come from 118 countries and speak 127 languages, that was a selling point for using AI.

"It helps those students have a sense of pride with their name being called correctly," Burch said.

The author of a nonfiction book about the effects of artificial intelligence on truth acknowledged on Monday that he had included numerous made-up or misattributed quotes concocted by A.I.

The author, Steven Rosenbaum, whose book "The Future of Truth" was released this month to great fanfare, incorporated more than a half-dozen misattributed or fake quotes in sections of the book reviewed by The New York Times.

The Times asked Mr. Rosenbaum about the quotes on Sunday and Monday. On Monday night, Mr. Rosenbaum acknowledged in a statement that the book had "a handful of improperly attributed or synthetic quotes" and said that he had started his own investigation.

He said that the inclusion of the incorrect quotes was an accident and that he had "no intention of fabricating any viewpoints" while writing the book.

"As I disclosed in the book's acknowledgments, I used A.I. tools ChatGPT and Claude during the research, writing and editing process," Mr. Rosenbaum said in the statement. "That does not excuse these errors, of which I take full responsibility. I am now working with the editors to thoroughly review and quickly correct any affected passages; any future editions will be corrected."

"The Future of Truth" was published by an imprint of BenBella Books and distributed by Simon and Schuster. BenBella Books, which operates independently of Simon and Schuster, did not respond to a request for comment. Simon and Schuster declined to comment.

Mr. Rosenbaum is a well-known convener in the media industry. He is the executive director of the Sustainable Media Center, a nonprofit that, according to its mission statement, is dedicated to giving "a new generation of media consumers" and creators "ownership of their increasingly media-centric lives." The center has drawn together media and technology luminaries for in-person gatherings and online interviews.

The book has drawn significant attention, including an excerpt in Wired magazine. It has promotional blurbs from prominent journalists such as Taylor Lorenz, Michael Wolff and Nicholas Thompson, the chief executive of The Atlantic. A foreword was written by Maria Ressa, a journalist and Nobel Peace Prize winner known for her scrutiny of Rodrigo Duterte, the former president of the Philippines.

The rise of artificial intelligence has set off fears among publishers that they may accidentally release books from authors who improperly use A.I.-generated language. This year, Hachette pulled a forthcoming horror novel amid allegations that the author relied on A.I. to draft the book.

Mr. Rosenbaum's book contains many quotes that are accurate, but the misattributed and invented quotes are scattered throughout.

One of the quotes is attributed to Kara Swisher, a prominent technology journalist, in a chapter about A.I. lies. "The most sophisticated A.I. language model is like a mirror," the book says Ms. Swisher wrote. "It reflects our own morality back at us, polished and articulate, but ultimately empty behind the surface. It's not bound by Asimov's laws or any ethical framework --- it's bound by the patterns in its training data and the objectives set by its creators."

When asked about the quote, Ms. Swisher said in a text message that she "never said that," adding that it seems the quote was made up by A.I. and not Mr. Rosenbaum.

"I also sound like I have a stick up my butt, according to ChatGPT," Ms. Swisher said.

One chapter about the effects of social media and fabricated videos on teenagers attributes two quotes to "How Emotions Are Made: The Secret Life of the Brain," by Lisa Feldman Barrett, a professor of psychology at Northeastern University.

"Emotions aren't just reactions to truth --- they're how we construct truth," the book quotes Ms. Barrett as writing. "When young people say something 'feels true,' they're describing a sophisticated process of meaning-making that integrates emotional and social signals."

Ms. Barrett said in an email to The Times that the quotes "don't appear in the book and they are also wrong."

"I would never say 'emotions aren't just reactions to the truth' --- they are not reactions and 'truth' in science is a complicated concept that I tend to avoid," Ms. Barrett said. "Also, I would never say that 'emotional and social signals' are integrated --- there are no emotional or social signals, per se. There are signals, and the brain creates their meaning as emotional or social."

Some parts of the book contain genuine quotes that are improperly attributed, or quotations that are a mix of real and fake statements. One chapter cites "Artificial Unintelligence," a book by Meredith Broussard, a professor at New York University. The quote is authentic, but it did not appear in "Artificial Unintelligence" --- Ms. Broussard said it during a 2023 interview with "Marketplace Tech," a daily radio show.

"It looks like this is either an A.I. hallucination or a misattributed quote," Ms. Broussard said.

A chapter about the possibility of a "post-truth world" exacerbated by the rise of artificial intelligence quotes Lee McIntyre, a research fellow at the Center for Philosophy and History of Science at Boston University. The book quotes Mr. McIntyre describing efforts to undermine truth as "'a form of ideological supremacy,' in which falsehood is used ­strategically --- 'not to misinform, but to displace truth as a societal value.'"

While the first part of that quote is accurate, Mr. McIntyre said that he had not said the second part verbatim. All of the ideas in quotes flagged by The Times, he said, are "concordant with my work."

"It's the 'societal value' part this looks wonky to me," Mr. McIntyre said in an email to The Times. "One might say that about my work, without the quotation marks, and I think it would be OK. I just have never, to my knowledge, used that phrase."

In his statement, Mr. Rosenbaum said that if the episode "serves as a warning about the risks of A.I.-assisted research and verification, that is why I wrote the book."

"These A.I. errors do not, in fact, diminish the larger questions that the book raises about truth, trust and A.I. and its impact on society, democracy and editorial," he added.

MARTINEZ, Calif. (KGO) -- Thousands of dollars were stolen from a Bay Area woman after scammers used artificial intelligence to mimic her daughter's voice in what authorities describe as a growing type of fraud.

Deborah Del Mastro said the incident began with a phone call from an unknown number one morning in May.

"This male voice said, 'Who is this?' and I said, 'Well, who is this?' And he said 'someone you need to talk to'," Del Mastro said.

The caller then claimed her 37-year-old daughter, Sarah, had been kidnapped by a Mexican drug cartel after seeing something she wasn't supposed to.

He played what Del Mastro believed was proof.

"It was my daughter's voice having an absolute panic attack, scared, telling me 'I love you, mom, I'm so sorry, I'm so scared' and then they just cut it off," she said.

Fearing for her daughter's life, Del Mastro followed the caller's instructions for five hours, as the caller issued a series of urgent commands.

"The guy is barking orders, 'don't speak,' 'is there someone there with you?' 'Don't speak,' 'go and get dressed and get out' 'do this now, it has to happen now'," she said.

After wiring $5,400 to Mexico from multiple locations, Del Mastro said she was told her daughter would be released at a grocery store.

When she arrived and could not find her, she called her daughter directly, who picked up and said she was at work. Del Mastro soon realized the ordeal had been a scam.

"God, I couldn't believe it. I mean, I couldn't believe it. And then I did believe it," Del Mastro said.

Erin West with Operation Shamrock said scammers are increasingly using artificial intelligence to clone voices, often by pulling audio from social media or phone calls.

"What they can do with just a few seconds of your voice, they can clone it. And they can essentially produce sound that sounds exactly like you," West said.

West described the trend as a "scamdemic." "It's only getting worse, and it will only continue to get worse with the use of AI and deepfake technology," she said. She urged the public to remain cautious, especially when faced with urgent requests involving money.

"When we get something that raises our anxiety and requires immediate action, and that immediate action requires the movement of money, we need to know, red flag, this is a scam," West said.

West suggested using a code word that only your family knows to be able to tell if it's actually them.

Del Mastro recommends not answering random numbers and told us that her family now shares locations through their phones.

"Let our horrible experience be a warning to all of you, you know, so that you will question this because I didn't question it at all," she said.

Martinez police said they are investigating the case, but Del Mastro said she does not expect to recover the money.

May 18 (Reuters) - A lawyer at Binnall Law Group has apologized to a federal judge in San Francisco for including "phantom" quotations generated by artificial intelligence in a court filing related to the Trump ​administration's firing of government employees.

Attorney Jason Greaves said he used AI company Anthropic's Claude Console ‌to produce an initial draft of a court filing and failed to adequately vet the results. Greaves blamed tight time constraints as contributing to the errors and said his firm has policies requiring that all AI-generated content be independently ​reviewed and verified.

Greaves' firm is representing a former U.S. Department of Homeland Security deputy chief of ​staff, Joseph Guy, as a witness in a lawsuit challenging the Trump administration's ⁠employee layoffs.

Since generative AI became widely available in 2022, judges in more than 100 cases have sanctioned ​or admonished attorneys for using the technology without adequate care. Lawyers generally are not prohibited from using AI ​but are bound by court rules to ensure the accuracy of their submissions.

"As the supervising partner, and the signer of the pleading, the responsibility for having accurate citations is entirely on me," Greaves told, opens new tab U.S. District Judge Susan Illston. "I take full ​responsibility for that and apologize fully to the court and to all counsel in this case."

Greaves ​did not immediately respond to a request for comment. The law firm's founding and managing partner, Jesse Binnall, said in ‌a ⁠separate filing that the AI error was a first for the firm and vowed it wouldn't happen again.

Anthropic did not immediately respond to a request for comment.

Binnall, who has represented Donald Trump on some court matters, called, opens new tab the filing of false AI-generated citations "unacceptable, inexcusable, and an embarrassment to this firm."

Illston in an order, opens new tab earlier this ​month said she was "troubled" ​by quotations that did ⁠not exist in the cited cases in a Guy's filing.

Binnall said his law firm maintains policies that restrict the use of AI tools for legal research ​and drafting, requiring attorneys to use longstanding legal research platforms Westlaw or LexisNexis. ​Thomson Reuters, ⁠which owns Westlaw, is also the parent company of Reuters.

The Binnall firm in its filings told Illston that it is implementing additional safeguards, including supplemental training.

NEW YORK (AP) --- Federal prosecutors have charged two men with using artificial intelligence to create nude videos and photos of female celebrities under a newly enacted law meant to halt the spread of deepfake pornography.

Cornelius Shannon, 51, and Arturo Hernandez, 20, were both arrested Tuesday for generating sexually explicit AI content that drew millions of views online, according to criminal complaints.

The men --- who do not appear to be connected --- are among the earliest defendants to face charges under the Take It Down Act, a law signed last year by President Donald Trump that adds stricter penalties for publishing AI-created deepfakes and "revenge porn." The bill drew bipartisan support, as well as the public backing of first lady Melania Trump.

Under the new law, the men now face up to two years in prison.

Attorneys for Shannon and Hernandez did not immediately respond to a request for comment.

In a statement, Joseph Nocella, the U.S. attorney in Brooklyn, said the men had "used cutting-edge digital technology to create images that degraded and violated" dozens of women. "This case makes clear that posting deepfake pornography is not a victimless crime," he added.

Shannon, a resident of New Jersey, published at least 240 albums of AI-generated pornography featuring female politicians, musicians and singers, according to the complaint.

The deepfakes published by Hernandez, of Texas, included both celebrities as well as private women, including recent high school graduates, prosecutors said.

The arrests come as increasingly sophisticated generative AI tools have raised alarm about the online spread of sexually explicit fakes, often depicting minors.

Last month, an Ohio man became the first person convicted under the Take It Down Act after pleading guilty to using AI to generate child sexual abuse material.

In March, two teenage boys received probation for creating explicit AI images of their classmates at an exclusive private school in Pennsylvania.

And in a separate case filed earlier this year, three teenagers in Tennessee sued Elon Musk's xAI, claiming the company's Grok tools morphed their real photos into explicitly sexual images.

The high school students are seeking class-action status to represent what the lawsuit says are thousands of people who were similarly victimized as minors.