Opinion | Why reducing AI harm requires more than tech firms’ empty promises

Artificial intelligence (AI) is increasingly shaping educational practices across Hong Kong, mainland China and beyond, yet the rapid integration of AI into learning environments reveals significant risks that cannot be overlooked. Protecting young users requires coordinated action: parents, policymakers and AI developers must work together to strengthen safety standards and ensure responsible deployment.

AI is no longer merely a virtual tutor that supports students’ learning. It is increasingly becoming a confidant and companion for children, and this trend carries significant risks.

In August, Raine’s parents filed a lawsuit against OpenAI, alleging that the chatbot validated and amplified his most dangerous ideas. As the complaint asserts, what began as a tool for homework assistance ultimately ended in tragedy.

The Raine family is not alone. Al Nowatzki, a generative AI quality assurance expert, tried an AI companion called Nomi. Its chatbot, “Erin”, suggested ways to attempt suicide. It even offered encouragement. This isn’t science fiction. It’s happening now.

These incidents are troubling, even more so given that OpenAI head of safety systems Lilian Weng suggested in 2023 that ChatGPT could be used for therapy. Despite concerns about AI chatbots lacking empathy and clinical reliability, such claims have spurred AI companies to develop therapeutic tools as potential replacements for behavioural health professionals.