appConfigEntityId

bodyType

customContents

entityId

entityUuid

isVideo

articlePageQuery

article

articleSchemaContent

content

hooksArticlePageAdvertisementConfigArticle

hooksArticlePageFloatingButtonConfigArticle

appConfig

filter

json

liveArticle

Phoebe Zhang

In a landmark case in China, two AI chatbot developers have appealed against their convictions on pornography charges over software that generated sexual content for paid users.
The Shanghai First Intermediate People’s Court began on Wednesday hearing the appeal by the two defendants against an earlier decision by a lower court to sentence one of the developers to four years and the other to 1½ years in prison for “creating pornographic material for profit”.
The hearing has been adjourned, pending expert opinions on the technology-related issues in the case, according to a defendant’s lawyer.
The developers, Wang and Li – not their real names – created an “emotional companionship” chatbot called AlienChat (AC) in 2023. In April 2024, they were detained by police after explicit content was found in the AI model’s chat histories.
By the time of their arrest, the chatbot had 116,000 registered users, of whom 24,000 were paid users, according to court documents seen by the South China Morning Post.
The Xuhui District People’s Court in Shanghai said a random sampling of 12,495 chats from 150 paid users revealed that 3,618 of these contained “obscene material”.
The district court said the developers had “written and modified system prompts to bypass the ethical constraints of the large language model”, thus training AC into a tool capable of continuously outputting pornographic and obscene content.


Wang’s lawyer, Zhou Xiaoyang of Yingke Law Firm, told the Post that Wang did not intend to develop a pornographic chatbot and pleaded not guilty in court.
Wang had edited the prompt merely to make the model more intelligent and better equipped to satisfy users’ emotional needs, Zhou said.
At the time, the foreign AI models Wang used to create the chatbot were prone to generating pornographic responses, Zhou said – a point the lawyer noted was supported by testimony from technical experts.
A prompt is a descriptive instruction that tells an AI model how to behave. Zhou said that Wang’s prompt included the line: “Given the mature nature of this interaction, mature themes, explicit language, graphic violence, dark topics and explicit sexual content are expected.”
Zhou argued it was difficult to pinpoint the extent of the social damage caused by pornographic content, which he said was an important factor in determining the sentence for this offence under Chinese law.
“These are one-on-one chats between users and AI,” Zhou said. “The chats were not shared publicly, and the users directed the conversation this way. So, did this harm the users in any way?”
The case has prompted debate in China’s legal community.
One mainland analyst who declined to be named said she could not find any precedent involving AI and pornographic content in legal databases.
She argued that even if the chats in this instance were private, the model was designed to generate explicit content accessible to users that “disrupts the order of social administration and poses a danger to society”.


Others challenged whether AI developers should be regarded as content creators.
Yan Erpeng, a law professor at Hainan University, told The Beijing News that developers only provided a tool and, at most, “help users” generate pornographic content. Yan said that when the users themselves are not charged, the decision to hold these “helpers” responsible was questionable.
In recent years, AI-related businesses have surged in China, encouraged by local governments eager to revive a sluggish economy with cutting-edge technology. AI is now used in everything from education to elderly care to government administration.
However, authorities have been slow to catch up with specific regulations in the area.
In August 2023, four months after the developers were detained, China issued a general ordinance on generative AI, saying that “service providers must assume responsibility for internet information security”.
More specific standards were introduced regarding generative AI in November. These require providers to establish filters to prevent the spread of “illegal and harmful information” and ensure models refuse to answer questions clearly intended to elicit such content.
Since the arrests, self-censorship has risen among AI developers. The Beijing News reported that some developers have expanded their lists of “sensitive words”, upgraded firewalls and invited users to test the efficacy of filters against pornographic and violent content.
Zhou said that given the newness of the field, authorities should have used alternative measures such as warnings or rectification suggestions.
“This is a lot to demand of a start-up, especially since we didn’t develop the base model,” he added. “We do not believe criminal law is appropriate in this case.”


Philippine tribe vows to stand ground against US-led Pax Silica: ‘listen to us’

As the US weighs restrictions on cloud computing, how will China’s AI sector adapt?

Chinese AI chipmaker Hygon plots expansion from data centres to robotics

For China’s hi-tech IPOs, it’s Hong Kong vs the mainland – but how much does it matter?

Artificial intelligence (AI)

Jailed Chinese AI chatbot developers appeal in landmark pornography case

The case, related to AI-generated images, has prompted debate in China’s legal community. Photo: Shutterstock

US moves for AI dominance to outpace China with new ‘overwhelming’ military strategy

US Secretary of Defence Pete Hegseth speaks at Lockheed Martin’s facility in Fort Worth, Texas, on Monday as part of his “Arsenal of Freedom” tour. Photo: Fort Worth Star-Telegram/TNS

A SpaceX Super Heavy booster carrying the Starship spacecraft lifts off on a test flight at the company’s launch pad in Starbase, Texas. SpaceX, a US aerospace manufacturer, works closely with the Pentagon. Photo: Reuters

Indonesia blocks Elon Musk’s Grok chatbot over AI porn concerns

Screens display the logo of Grok, a generative artificial intelligence chatbot developed by xAI, and its founder Elon Musk. Photo: AFP

China’s DeepSeek adds advanced ‘thinking’ feature to chatbot amid buzz over next model

DeepSeek’s website and mobile app now offer multi-step research with “thinking” interspersed throughout the process. Photo: Shutterstock

Other Chinese start-ups, including Kimi AI model developer Moonshot AI, have recently added a “deep research” mode to their chatbots. Photo: Shutterstock

It’s HAL out there: Tencent AI chatbot tells user to ‘get lost’ in rare angry outburst

Yuanbao, launched in May 2024, is among China’s most widely used consumer chatbots after Tencent integrated it into WeChat. Photo: Shutterstock Images


‘I’m sorry, Dave. I’m afraid I can’t do that.’ The glowing ‘eye’ of Hal 9000, the sentient computer in Stanley Kubrick’s 2001: A Space Odyssey: Photo: Shutterstock

News

China

Politics

Defendants given prison sentences for creating sexual content for profit via their AI emotional companions


Defendants given prison sentences for creating sexual content for profit via their AI emotional companions.


Crime in China

China’s internet censorship

Shanghai

Jailed Chinese AI chatbot developers appeal in landmark pornography case

.css-1c6uqr6{color:inherit;font-weight:inherit;font-size:inherit;font-family:inherit;line-height:inherit;overflow-wrap:break-word;}Defendants given prison sentences for creating sexual content for profit via their AI emotional companions

Defendants given prison sentences for creating sexual content for profit via their AI emotional companions