Viral Jennie clip tests limits of South Korea’s AI deepfake laws

For a few seconds during Blackpink member Jennie’s headline set at Tokyo’s Summer Sonic festival last month, it looked like something had come undone. She turned away from the crowd, adjusted herself and then, the moment was gone. But online, the clip never disappeared – and more than 10 million views later, it has become the latest test of South Korea’s fight against AI-enabled sexual abuse. Fans were quick to cry deepfake, dissecting the footage frame by frame for signs of possible AI manipulation and calling for it to be taken down. The K-pop star’s agency, OA Entertainment, neither confirmed nor denied the clip’s authenticity, merely saying in a statement on August 18 that it had “recently identified ongoing misconduct targeting the artist [Jennie] across online communities and social media platforms”. It added that it was working with lawyers to pursue civil and criminal action, vowing “no compromise or leniency whatsoever”. The account believed to have first posted the video has now been suspended, but no official police or prosecutorial follow-up has yet been announced. If pursued, the case is set to become a high-profile test of South Korea’s revised deepfake laws. Enacted in 2024, their scope has since been broadened to punish not only those who create or distribute sexually exploitative AI-manipulated material, but also those who possess, save or watch it. Yet legal experts say a viral clip of disputed authenticity also exposes the practical limits of enforcement, from proving who made it to who knowingly spread it around. South Korea’s legal framework is “relatively proactive in establishing strong criminal penalties for this kind of abuse”, according to Bang Suk-ho, a senior legal adviser and head of the AI industry centre at Seoul-based law firm Lin LLC. “But simply strengthening the law does not make crime disappear,” he told This Week in Asia. “If anything, we’re in a phase where technology has led to more crime while making it harder to detect criminals.” Evolving abuse South Korea overhauled its legal arsenal against deepfakes two years ago in the wake of public fury over Telegram-based sex crime rings that distributed doctored images of women and girls, including students and servicewomen, via encrypted chat rooms. The scandal carried echoes of the notorious Nth Room case of 2020, which first exposed how platforms based overseas could be used to produce, sell and blackmail victims with sexually exploitative material. But the newer deepfake cases were different in one crucial aspect: demonstrating how convincing abuse material can now be conjured from an ordinary photo, then rapidly circulated and viewed by large numbers of people, before anyone – victim or investigator – knew who had made it. Under amendments to South Korea’s Act on Special Cases Concerning the Punishment of Sexual Crimes that came into effect in 2024, creating sexually explicit deepfakes carries penalties ranging up to seven years in prison or a fine of 50 million won (US$36,800). Possessing, purchasing, storing or knowingly viewing such material can carry up to three years in prison or a fine of up to 30 million won. Campaigners got some of the harsher penalties they were calling for, but the new law also raised difficult questions about proving intent – and how to actually enforce it. Tougher sentencing rules had pushed prosecutors to seek “increasingly harsher sentences”, which in turn had influenced court decisions, said An Gab-chul, a lawyer at Gammyung Law Firm in Seoul who specialises in sexual crimes. Yet An said the new measures “often fall short in practice and do not function as effectively as they should”, further arguing that procedures for removing illegal material should be simplified. The Jennie case shows why. Before any prosecution can begin, investigators need to establish whether the footage was indeed fabricated, trace who posted it first and prove whether those who shared it knew it was fake. “The apprehension and punishment of offenders is another major problem, as it’s difficult to identify the operators of platforms based overseas,” An said. “Extraditing them may raise conflicts with foreign laws, and overseas servers are outside the jurisdiction of South Korean law.” That difficulty was borne out in police data presented late last month at a forum hosted by Amnesty International Korea, which showed that authorities had filed 26,697 data requests with foreign messaging services and platform operators but received responses to just 15,466 of them, or 58 per cent. The rest went unanswered. For An, the more immediate concern is often more basic: identifying who is behind an anonymous account before the evidence disappears. “The biggest obstacle in online sexual crimes today is preventing suspects from destroying or tampering with evidence,” he said. “In the case of anonymous accounts, it can take a long time to identify the actual user. Investigators often have to contact operators of overseas servers to determine who is behind an account before the person behind the account shuts it down or destroys evidence.” In viral celebrity cases like Jennie’s, that delay can be especially damaging, as reposts and screenshots multiply and vanish long before investigators can establish where the material originated. Still, South Korean courts have shown they can impose prison sentences for AI-generated sexual abuse material when investigators do identify the person responsible. In June, a man convicted of creating and distributed deepfakes of K-pop girl group Aespa was sentenced to two and a half years in prison, alongside an 80-hour sexual violence treatment programme and a seven-year ban from working with children and adolescents. Moral disconnect Criminal penalties alone cannot solve the issue, according to Bang, who finds the psychological side of these offences more unsettling. Whereas committing a crime in person forces the offender to confront their victim, he argues that screens remove this human element. “In the virtual world, that awareness of wrongdoing … becomes much weaker, especially among younger people,” Bang said. “The younger the users, the wider the gap tends to be between their awareness of wrongdoing and the severity of the punishment. As a result, their awareness of the seriousness of these crimes, and of the harm caused to victims, is becoming weaker.” Last year, deepfake-related offences accounted for 35.2 per cent of all cybersex crimes uncovered by police, or 1,553 cases in total – the single largest category, according to statistics presented at the Amnesty forum. Of the 3,557 people apprehended in 2025, almost half – 1,761 – were teenagers and over 90 per cent of deepfake suspects were in their teens or twenties. One case last month involved a teenager being referred to prosecutors after allegedly creating sexually explicit deepfakes using photographs of about 20 girls his age and sharing them on social media. “So this is not something that can be solved simply by strengthening the law, like an endless game of cat and mouse,” Bang said. “We’ve already seen that from experience.” This is not something that can be solved simply by strengthening the law, like an endless game of cat and mouse Bang Suk-ho, AI legal expert Won Eun-ji has seen it too. As executive director of Team Flame, the investigative group that helped expose the Nth Room digital sex-crimes and blackmail scandal in 2020, she has watched high-profile cases like Jennie’s capture the kind of public attention that ordinary victims rarely receive. “The government, media and society as a whole are not paying enough attention to this issue,” she said, noting that even today, many investigations only began after victims forced the issue. “The police and authorities need to further specialise and strengthen their capabilities in tracking and investigating such cases.” Her final point was one she wanted readers to reflect on: that viral moments like the Jennie clip showed the public had a role to play too. “Most of us may not be perpetrators or victims, but that does not mean we are unrelated to the issue,” Won said. “Whenever we come across deepfake sexual abuse material, it is important to report it to the platform, and remember that when someone opens up about being victimised, it is important to support them. We shouldn’t just be bystanders.” Amnesty International Korea framed the challenge in still broader terms, saying the real task ahead was for society to ask “why this violence continues to occur and how we can change the environment in which violence occurs and spreads”. “Digital sexual violence should be understood as a form of gender-based violence,” the organisation said in a statement. “We need to address the structural causes that allow such violence to persist, as well as creating change in the digital environment that provides it a platform.”

Viral Jennie clip tests limits of South Korea’s AI deepfake laws

.css-1c6uqr6{color:inherit;font-weight:inherit;font-size:inherit;font-family:inherit;line-height:inherit;overflow-wrap:break-word;}Severe legal threats fail to deter a wave of teenage offenders who see digital victims as nothing more than pixels

Severe legal threats fail to deter a wave of teenage offenders who see digital victims as nothing more than pixels