Citation record for Incident 202

Suggested citation format

AIAAIC. (2021-12-06) Incident Number 202. in Lam, K. (ed.) Artificial Intelligence Incident Database. Responsible AI Collaborative.

Incident Stats

Incident ID
Report Count
Incident Date
Editors
202
5
2021-12-06
Khoa Lam

Incidents Reports

Presidential candidates show off their digital engagement with AI avatars.

At the kickoff ceremony of an election committee for the main opposition party’s presidential candidate Yoon Suk-yeol, a familiar face showed up to encourage support for the nominee, calling himself “AI Yoon Suk-yeol.”

“Hello. I am AI Yoon Suk-yeol. Are you surprised because we look alike?” the digitally-created character resembling the People Power Party’s presidential candidate said in a video clip revealed at the ceremony on Monday.

“AI Yoon Suk-yeol is a first in the political arena, and it symbolizes Korea’s new future that Yoon will create (when he is elected president),” it added.

While not perfect, presidential candidates running for the March election are bringing in advanced technologies to show off their digital engagement to appeal to voters.

The AI Yoon Suk-yeol was slightly awkward when making some gestures, but the character perfectly recreated the voice of the candidate.

The digitalized Yoon also lacked some of the distinct habits of the nominee, who has been criticized for shaking his head sideways too often and for opening his legs too wide in public events.

The People Power Party election committee is also ramping up efforts in digital canvassing, announcing that it will utilize Namuwiki, a local online encyclopedia, to collect public opinions on election pledges.

On Tuesday, Kim Dong-yeon, a former Finance Minister who threw his hat in the ring for the presidency in August, introduced an artificial intelligence spokesperson named Aidy as the first hired employee of his election committee.

He also presented an avatar character named Windy, who is his second hired employee.

“I want to be chosen by the people with a new way of campaigning that fits the new era, instead of the usual time-consuming activities that wastes taxpayer money,” Kim said in the announcement, adding that the two major political parties receive over 100 billion won ($85 million) as a state subsidy for election campaigning.

“Korean politics has been a burden to the people for a long time, costing an astronomical amount of money,” Kim said. “Adopting an AI spokesman is an attempt to cut down on the cost of campaigning.”

The AI spokesperson appeared in a short video to introduce himself.

“Hello, I am Kim Dong-yeon’s spokesman Aidy. I will be working hard to support Kim’s philosophy and for a better world,” Aidy said.

Taking on the usual task of a spokesperson, Aidy also criticized the ruling and main opposition parties for hiring campaign employees only for show and that lack genuineness.

“Kim Dong-yeon (on the other hand) is actually recruiting me and Windy as his first and second campaign employees in his attempt to introduce the right figures, in the era of the fourth industrial revolution and digitalization,” the AI spokesman added.

Kim, who served the ministerial position for the incumbent Moon Jae-in Administration, formed a new party dubbed “New Wave” in October.

According to Kim, his election camp plans to utilize Aidy and Windy to better approach voters online. Aidy will be delivering various messages from the candidate together with spokeswoman Song Moon-hee.

Windy, the avatar, would be working as “the other half” of the candidate, presenting his policies and appearing in places on behalf of Kim, he added.

Lee Jae-myung, the presidential candidate from the ruling Democratic Party of Korea, operated an election camp headquarters inside a metaverse to communicate with the users during the party’s primary. There, he presented an AI-based chat bot, which automatically gave answers to questions.

Other candidates, including Ahn Cheol-soo of the minor opposition People’s Party and Sim Sang-jung of the progressive Justice Party, are also preparing digitalized electioneering activities, party officials said.

Amid the efforts to highlight that the candidates are “up-to-date,” there are some concerns voices that these AI and deepfake technologies may be abused to manipulate campaign efforts.

“Thinking about the political ethics, I think the People Power Party has gone too far,” Ko Sam-seog, former standing member at the Korea Communications Commission said on his Facebook on Tuesday. He served his post at the communications commission at the recommendation of the ruling Democratic Party.

“It is obvious why PPP adopted Yoon Suk-yeol’s avatar. They wanted a makeover of the candidate’s bad image, with his head-shaking, wide-legged standing and poor speaking ability,” Ko said.

“Whether it is with good intentions or bad, deep fake technology should be used sparingly, and very carefully.”

In 2017, a video showing former US president Barack Obama appearing to curse and calling then-president Donald Trump names went viral in the United States. The video turned out to be fake, created with the deepfake technology.

[Election 2022] AI spokesman, avatars enter election campaigns

Korea's presidential candidates are capitalizing on digital technologies to win more support by having virtual characters powered by artificial intelligence (AI) substitute for themselves in election campaigns.

The attempts, however, are sparking debate over the political ethics of the practice due to the potential deceptiveness of AI-created characters showcasing prepared audiovisual content only, as well as concerns over "deepfakes," video clips using fabricated images which are extremely difficult to identify as real or fake.

On Dec. 6, main opposition People Power Party (PPP) presidential candidate Yoon Suk-yeol introduced "AI Yoon Suk-yeol" during the inauguration ceremony of his election camp.

Appearing in a video clip played during the ceremony, Yoon's avatar said, "Are you surprised because I look the same as candidate Yoon?"

An artificial intelligence-generated image of main opposition People Power Party presidential candidate Yoon Suk-yeol speaks in a video clip which was played during the inauguration ceremony of Yoon's election camp, Monday. Captured from People Power Party YouTube channel

"AI Yoon Suk-yeol is the beginning of innovation in election, and I will visit every corner of the country," the artificial character continued. A facsimile of Yoon, his AI version impersonated the candidate's voice, accent and even words of choices based on deep learning, which is a branch of AI technology.

The PPP said it plans to use the technology for campaign areas where Yoon cannot attend in person by showing clips on screens set up on vehicles.

An artificial intelligence-generated image of main opposition People Power Party presidential candidate Yoon Suk-yeol speaks in a video clip which was played during the inauguration ceremony of Yoon's election camp, Monday. Captured from People Power Party YouTube channel

On Tuesday, former deputy prime minister and independent presidential candidate Kim Dong-yeon also introduced AI-powered cyber spokesperson, named AiDY. He also showcased his own avatar, WinDY.

AiDY resembled a polygon character that could be seen in video game or metaverse platforms, but WinDY is a realistic rendering of Kim.

"AI spokesperson AiDY is an attempt to profoundly reduce the cost of election campaigning," Kim said. "An AI spokesperson may not be satisfying for now, but it will make progress down the road, replacing conventional election campaigns which consume tax payers' money."

Though the avatars have now made flashy debuts in Korean politics, their roles are anticipated to be limited ― reading prepared scripts or showing simple gestures ― because the characters are yet to learn candidates' language characteristics to a deep level. Kim's AI spokesperson is also expected to function as a chatbot, rather than playing a creative role in proposing a political agenda and disseminating the candidate's messages independently.

As presidential candidates are depending on technologies which are still new in the world of politics, questions are arising over the ethics of AI-powered campaigns.

"We should think about the political ethics," Ko Sam-seog, a chair professor at Dongguk University and former Korea Communications Commission member, wrote on his Facebook page.

"Even if a small company hires an employee, it requires the job seeker to appear at an interview to see how he or she speaks and writes. A person who seeks to be the country's president asks voters to elect them through an avatar is something close to a fake."

Ko, who served at the presidential office for three liberal presidents ― Kim Dae-jung, Roh Moo-hyun and Moon Jae-in ― also claimed that AI Yoon Suk-yeol is the PPP's attempt to "disguise Yoon's personal habits such as rude sitting posture or excessive use of filler while speaking." Yoon has been criticized for such habits, and his AI character did not reflect those traits.

Concerns over fake news exploiting deepfake are also being raised. In 2018, a video clip in which former U.S. President Barack Obama cursing then-President Donald Trump went viral, was revealed to be an educational video created to alert the public to the risks of synthetic media. Following subsequent debates, Facebook and Twitter in January published new guidelines banning deepfake clips.

Since deepfake technology can be used to create whatever scene the maker desires, experts said it is difficult to rule out the chance that fake clips in which either Yoon or Kim promising nonsense policies will appear. This would eventually lower the public's trust on other forms of media which present candidates in their true form.

Against this backdrop, the ruling Democratic Party of Korea (DPK) presidential candidate Lee Jae-myung pledged to strengthen laws to punish those who make or are in possession of malicious deepfake clips, with the DPK criticizing Yoon's AI campaigning saying, "Yoon should stop hiding behind his avatar."

While debates are heating up on AI campaigning, the National Election Commission (NEC) is yet to determine whether it is legitimate or not. "It is difficult to make a finding on whether it is against the laws governing campaigning or not because it is uncertain how the technologies will be used in the campaign," an NEC official said.

Presidential candidates' computer-generated avatars heat up debate

In a crowded campaign office in Seoul, young, trendy staffers are using deepfake technology to try to achieve the near-impossible: make a middle-aged, establishment South Korean presidential candidate cool.

Armed with hours of specially-recorded footage of opposition People Power Party candidate Yoon Suk-yeol, the team has created a digital avatar of the front runner – and set “AI Yoon” loose on the campaign trail ahead of a March 9 election.

From a deepfake video of Barack Obama insulting Donald Trump to failed New York mayoral candidate Andrew Yang campaigning in the metaverse, AI technology has been used in elections before.

But AI Yoon’s creators believe he is the world’s first official deepfake candidate – a concept gaining traction in South Korea, which has the world’s fastest average internet speeds.

With neatly-combed black hair and a smart suit, the avatar looks near-identical to the real South Korean candidate but uses salty language and meme-ready quips in a bid to engage younger voters who get their news online.

It’s been a huge hit. AI Yoon has attracted millions of views since making his debut January 1.

Tens of thousands of people have asked questions, but it’s not the usual policy-related fare.

“President Moon Jae-in and (rival presidential candidate) Lee Jae-myung are drowning. Who do you save?” one user asks AI Yoon.

“I’d wish them both good luck,” the avatar snaps back.

At first glance, AI Yoon could pass for an actual candidate – an apt demonstration of how far artificially generated videos, known as deepfakes, have come in the last few years.

The real Yoon recorded more than 3,000 sentences – 20 hours of audio and video – to provide enough data for a local deepfake technology company to create the avatar.

“Words that are often spoken by Yoon are better reflected in AI Yoon,” said Baik Kyeong-hoon, the director of the AI Yoon team.

What the avatar actually says is written by his campaign team, not by the candidate himself.

“We try to come up with humorous and satirical answers,” Baik said.

The approach has paid off. AI Yoon’s pronouncements have made headlines in South Korean media, and seven million people have visited the “Wiki Yoon” website to question the avatar.

“If we had only produced politically correct statements, we would not have this reaction,” Baik said.

“The political establishment has been too slow in the face of a fast-changing society,” he added.

When answering questions posed by users, AI Yoon mockingly refers to President Moon and his rival Lee as “Moon Ding Dong” and “Lee Ding Dong”.

“I want to ask Moon Ding Dong this question: who is our real enemy?” AI Yoon says, in a thinly-veiled swipe at what his critics say is the president’s more conciliatory approach towards Pyongyang.

North and South Korea remain technically at war and Moon has met with Pyongyang’s leader Kim Jong-un four times since taking office – an approach candidate Yoon rejects as too soft.

The avatar politician has also used humour to try and deflect attention from Yoon’s past scandals, for example claims he received inappropriate fruit gifts from a construction company when he was a senior prosecutor.

“I am not beholden to persimmons and melons. I am only beholden to the people,” AI Yoon said – although his campaign was later forced to acknowledge he had accepted some gifts.

The kind of script used by the campaign for AI Yoon draws on the language used in the online gaming world, Kim Myuhng-joo, professor of information security at Seoul Women’s University, told local media.

“AI Yoon reads off the scripts compiled by its creators, who do not mince words,” Kim said.

Ko Sam-seog, a staffer for Yoon’s main opponent Lee, accuses the cyber-candidate of “downgrading political decorum”.

But the snark is working: while polling for the March 9 election remains neck-and-neck, Yoon has pulled ahead of rival Lee Jae-myung with voters in their 20s.

Tech-savvy Baik and his two other team members – all in their 20s and 30s – are some of the youngest staffers in the sprawling Yoon campaign.

They come with AI Yoon’s responses in rapid-fire brainstorming sessions, which can take as little as 30 minutes, in contrast to the carefully-honed rhetoric usually found in public policy debates.

South Korea’s election monitor allows AI candidates to campaign on the condition it is clearly identified as deepfake technology, and does not spread misinformation.

The technology has more often been flagged as harmful – the 2018 deepfake video of Obama was produced by Oscar-winning filmmaker Jordan Peele to warn viewers about trusting material they encounter online.

But Baik thinks AI is the future of election campaigns.

“It’s so easy to create huge amounts of content with deepfake technology,” he told said.

“It is inevitable that this will be used more and more.”

Is South Korean the world’s first official deepfake candidate?

South Korean presidential front-runners court younger voters with avatars—who are wittier and more likable

SEOUL—Wearing a dark suit and red tie, Yoon Suk-yeol, the main opposition party’s candidate for South Korean president, stared into the camera with a stern look in his eyes. He prepared to answer a question: If he had to save President Moon Jae-in or the ruling party’s nominee from drowning, whom would he pick?

“I am AI Yoon Suk-yeol,” he begins his response, thanking the questioner in a video posted online. He paused a moment before finishing his thought.

“From far away,” he says, cracking a small grin. “I will cheer them both on from far, far away.”

The so-called AI Yoon—as in Artificial Intelligence Yoon—sounds, looks and gestures much like the real-life, conservative politician who is in a close race for South Korea’s presidential election on Wednesday—although with much more mischievous humor.

A sharp-tongued former prosecutor, the 61-year-old Mr. Yoon is new to politics and wanted an efficient way to reach out to the electorate. He needed to pursue young voters and sought a softer public image, and had just roughly three weeks to officially campaign by law.

“We want voters to see the human side of Yoon—not the stern image he projects on television,” said Baik Kyeong-hoon, head of the campaign’s AI Yoon team.

More than 80 clips of Mr. Yoon’s digital self have been shared on social media, attracting more than 70,000 comments since making a debut in January. The daily videos are typically 30 seconds or less. Mr. Yoon’s campaign staffers choose a voter question to answer and write a script for the avatar.

AI Yoon addresses topics including North Korean missile launches and fake news. The character also delves into the K-pop girl group Blackpink (one of their songs is AI Yoon’s “karaoke go-to”) and his grocery shopping list that day (eggs, green onions, anchovies and beans).

Though the real Mr. Yoon hasn’t shied away from campaign mudslinging, his AI alter ego dispenses the insults in more meme-friendly vernacular. He slams his progressive rival, Lee Jae-myung, of the ruling Democratic Party, belittling him as, “Lee something something.” The AI character said one debate performance by Mr. Lee showed that “he lost his will to fight.”

“Yoon is learning from his AI Yoon messages,” said Lee Jun-seok, head of the candidate’s People Power Party, who came up with the idea of the AI version of the candidate. Mr. Yoon has adapted what he brings up on the campaign trail, and how he says it, based on the popularity of the online videos, he said.

“Human Yoon is way more interesting than AI Yoon—if you get to meet him in casual places,” added Mr. Lee, who is 36 years old and the conservative party’s youngest-ever leader.

The AI technology is often called “deepfake.” It generally refers to videos or images that use technology to falsely portray people saying or doing things.

The South Korean incarnation is more sophisticated than the traditional tactics of dropping in face swaps or voice impersonations. Mr. Yoon’s digital twin is controlled by his campaign team, drawn from hours of audio and video he recorded himself.

Democratic Party officials initially blasted AI Yoon, calling the deepfake fraudulent and a threat to democracy. But soon after, an AI version of Lee Jae-myung emerged. His party said it was different because Mr. Lee’s avatar was made from real footage of his actual comments, such as reciting his election pledges and slogans. In contrast, Mr. Yoon’s computer-generated remarks “purposefully hid Mr. Yoon’s flaws,” a Democratic Party official said.

“I am a realist and a pragmatist,” said Mr. Lee, the candidate, in a written interview.

South Korea’s presidential race is a tight showdown between Messrs. Lee and Yoon. Unlike prior elections, people in their 20s and 30s have become swing voters, expressing dissatisfaction with the state of the economy, soaring real-estate prices and a variety of ruling party political scandals. More than half of the electorate wants a change from the current Moon administration, according to Gallup Korea polling.

That has given Mr. Yoon—who has especially targeted young men and taken swipes at feminism—a shot with a younger demographic that has historically shunned conservatives.

Lee Seong-yoon, a 23-year-old college student, first thought AI Yoon was real after viewing a video online. Watching Mr. Yoon talk at debates or on the campaign trail can be dull, he said. But he now finds himself consuming AI Yoon videos in his spare time, finding the digital version of the candidate more likable and relatable, in part because he speaks like someone his own age. He said he is voting for Mr. Yoon.

“I’m not so worried about deepfake technology because any technology can be used for good or bad,” he said.

To build AI Yoon, the politician late last year spent two days at a recording studio trying to perfectly enunciate about 3,000 sentences, such as, “To the people of Jeju, it is nice to meet you” and “It is a tremendous honor for me to be here.” He was recorded in front of a green screen, a camera capturing his blinks, lip movement and expressions. He switched outfits.

From those elements, the technology can produce Mr. Yoon saying nearly anything and making a variety of movements.

The underlying technology is provided by the Seoul-based DeepBrain AI Inc., which synthesizes voice and video to produce a human-looking avatar that can hold down real-time conversation. “It’s a bit creepy, but the best way to explain it is we clone the person,” said John Son, who heads Asia-Pacific business development at DeepBrain.

The digital version of Mr. Yoon launched on Jan. 1. It tanked. The AI version of Mr. Yoon, which was controlled by campaign staffers typing in answers to policy questions, was too serious—much like his public persona, which tends to be bombastic and gruff.

Then, the staffers pivoted to fielding lighter questions from voters and sprinkling in humor. “What’s your MBTI?” one voter asked, the acronym for the workplace personality test.

“My personality type is ENFJ, the same as Barack Obama,” AI Yoon responds, referring to a personality type common among outgoing, warm leaders. “Have an energetic day, Barack Obama!”

Some videos take just 30 minutes to make, while others take half a day, especially when specific policy knowledge is needed. The script writers try to make each response funny and understandable even to a middle schooler, said Mr. Baik, the head of the AI Yoon team.

AI Yoon has some limitations that Mr. Yoon doesn’t. His voice, rearranged by the technology, produces the smooth delivery of a news anchor, so he can’t laugh, express excitement or be sad. Even spelling out “ha ha” produces a flat sound because it merges together awkwardly, Mr. Baik said. The real-life Mr. Yoon is a head bobber, side to side, when he talks. But the software can’t replicate that movement given how it splices together the sound and visuals.

Should Mr. Yoon win on Wednesday, the career of AI Yoon could be extended, too, said Lee Jun-seok, the party leader. He envisions one of DeepBrain’s kiosks greeting visitors at the presidential Blue House.

“I look forward to having AI Yoon Suk-yeol tell me where the toilet is,” he said.

These Campaigns Hope ‘Deepfake’ Candidates Help Get Out the Vote

As a star prosecutor in South Korea, Yoon Suk-yeol, the leading conservative candidate in the country’s presidential election, helped imprison two former presidents as well as the head of Samsung and a former chief justice of South Korea’s Supreme Court on charges of corruption. Today, as citizens cast their votes, Yoon hopes to become president himself by appealing to South Koreans who are deeply dissatisfied with the outgoing president, Moon Jae-in.

And Yoon has called on a secret weapon to boost his popularity among younger audiences—artificial intelligence, more precisely, deepfakes. The so-called ‘AI Yoon’ stood in as a replacement for the ‘real deal’ in short video clips. Why? Because AI Yoon looks and gestures much like the real-life politician it is based on—only its answers are wittier and more likeable.

“A sharp-tongued former prosecutor, the 61-year-old Mr. Yoon is new to politics and wanted an efficient way to reach out to the electorate. He needed to pursue young voters and sought a softer public image, and had just roughly three weeks to officially campaign by law,” explained the Wall Street Journal (WSJ) on the logic behind AI Yoon.

“We want voters to see the human side of Yoon—not the stern image he projects on television,” Baik Kyeong-hoon, head of the campaign’s AI Yoon team, further told the publication. Over 80 clips of Yoon’s deepfake have been shared on social media platforms, attracting more than 70,000 comments since making its debut in January 2022. The videos are typically 30 seconds or less, posted on a daily basis and often show AI Yoon answering voters’ questions picked beforehand by the candidate’s campaign staffers. Once a question is selected for the day, its answer is redacted and delivered by the digital avatar.

AI Yoon addresses a myriad of topics including North Korean missile launches and fake news, the K-pop girl band Blackpink—according to the WSJ, one of their songs is AI Yoon’s “karaoke go-to”—and his grocery shopping list that day (eggs, green onions, anchovies and beans).

In turn, “Yoon is learning from his AI Yoon messages,” said Lee Jun-seok, head of the candidate’s People Power Party, who came up with the idea of the deepfake version. Following the success of his digital self, Yoon has adapted what he brings up on the campaign trail and how he says it. Considering the fact that AI-generated fake faces have been proven to be more trustworthy than real ones, it comes as no surprise that AI Yoon has played a crucial role in the candidate’s presidential campaign.

In an attempt to slow down the deepfake’s rise in popularity, Democratic Party officials initially blasted AI Yoon, calling the technology fraudulent and a threat to democracy. Yet soon after, an AI version of Lee Jae-myung, Yoon’s progressive rival, emerged too. His party justified the move by saying that it was different because Lee’s avatar was made from real footage of his actual comments, such as reciting his election pledges and slogans. In contrast, Yoon’s computer-generated remarks “purposefully hid Mr. Yoon’s flaws,” a Democratic Party official said.

Currently, South Korea’s presidential race is a tight showdown between Lee and Yoon. People in their 20s and 30s have now become swing voters, expressing dissatisfaction with the state of the economy and soaring real-estate prices. Outgoing president Moon and his Democratic Party have also been rocked by a series of scandals that exposed ethical lapses and policy failures. As a result, more than half of the electorate wants a change from the current Moon administration, according to a Gallup Korea polling.

That has given Yoon—who has especially targeted young men and taken a stab at feminism—a shot with a younger demographic that has historically avoided conservatives. Lee Seong-yoon, a 23-year-old college student, first thought AI Yoon was real after viewing a video online. Watching Yoon talk at debates or on the campaign trail can be dull, he explained to the WSJ. But he now watches AI Yoon videos in his spare time, finding the digital version of the candidate more relatable, in part because “he speaks like someone his own age.” He said he is voting for the candidate.

“I’m not so worried about deepfake technology because any technology can be used for good or bad,” he further told the publication. The technology behind AI Yoon is provided by the Seoul-based DeepBrain AI, which synthesises voice and video to produce a human-looking avatar that can hold down real-time conversation. “It’s a bit creepy, but the best way to explain it is we clone the person,” said John Son, who heads Asia-Pacific business development at DeepBrain AI, to the WSJ.

But what’s really to be acclaimed here is the work Yoon’s campaign team did. When the digital version of the candidate first launched, it completely tanked. The deepfake was too serious and unapproachable as it answered policy questions—much like its real-life counterpart. Then, the staffers thought of changing the type of questions AI Yoon would be answering, which is when questions from voters started appearing along with some humorous answers from the deepfake.

“What’s your MBTI?” one voter asked, the acronym for the workplace personality test. “My personality type is ENFJ, the same as Barack Obama,” AI Yoon responded, referring to a personality type common among outgoing and warm leaders. “Have an energetic day, Barack Obama!” he added.

No matter the question, script writers try to make each response funny and understandable even to a middle schooler, Baik, head of the AI Yoon team, told the WSJ. If Yoon wins today’s election, the career of AI Yoon could be extended too, said Lee Jun-seok, leader of the People Power Party. He envisions one of DeepBrain AI’s kiosks greeting visitors at the presidential Blue House.

In South Korea, AI deepfakes of politicians are being used to win the hearts of the people