Baidu’s Ernie Bot tops Chinese large language model rankings by Xinhua think tank, but lags OpenAI’s ChatGPT

Ernie Bot performs better in a series of tasks than competing services from Alibaba, iFlyTek and SenseTime, according to Xinhua Institute

A different LLM test in China finds Qihoo 360’s Smart Brain to be the best-performing Chinese model, followed by iFlytek’s SparkDesk

Reading Time:2 minutes

The logos of Ernie Bot and Baidu are seen in this arranged photograph. Photo: Shutterstock

Published: 7:00pm, 12 Jun 2023

Baidu’s Ernie Bot ranked top among ChatGPT alternatives offered by Chinese tech companies, according to a recent test conducted by Xinhua Institute, a think tank affiliated with the Xinhua state news agency.

The internet search giant’s chatbot, launched in March, led Alibaba Group Holding’s Tongyi Qianwen, voice recognition firm iFlytek’s SparkDesk, and image recognition company SenseTime’s SenseChat in Xinhua’s test of large language models (LLMs). Alibaba owns the South China Morning Post.

However, Baidu’s LLM still trailed US rival OpenAI’s GPT-4 and GPT-3.5, which are officially unavailable in mainland China. GPT-3.5 powers the original version of ChatGPT that was released to the public in November last year.

08:54

Is China’s technology falling behind in the race for its own ChatGPT?

Xinhua’s test measured general capabilities, including basic language skills – such as the ability to understand prompts and generate responses – and intelligence, which involves common sense, logical reasoning and subject knowledge in fields like mathematics, physics, finance and literature.

The test also gauges the chatbots’ ability to enhance the productivity of professionals such as journalists, painters, designers, marketing specialists, lawyers and researchers.

Chinese technology firms have been rushing to introduce competing chatbots after the launch of ChatGPT sparked renewed interest in the potential of artificial intelligence (AI), in particular LLMs – deep-learning algorithms that can recognise and generate text, images and videos.

A different LLM test by Clue, a Chinese website that follows AI research development, found cybersecurity firm Qihoo 360’s Smart Brain to be the top-performing Chinese model, followed by iFlytek’s SparkDesk. Both ranked below GPT-4, GPT-3.5 and US start-up Anthropic’s Claude.

Baidu’s Ernie Bot tops Chinese large language model rankings by Xinhua think tank, but lags OpenAI’s ChatGPT

Ernie Bot performs better in a series of tasks than competing services from Alibaba, iFlyTek and SenseTime, according to Xinhua Institute A different LLM test in China finds Qihoo 360’s Smart Brain to be the best-performing Chinese model, followed by iFlytek’s SparkDesk

Ernie Bot performs better in a series of tasks than competing services from Alibaba, iFlyTek and SenseTime, according to Xinhua Institute

A different LLM test in China finds Qihoo 360’s Smart Brain to be the best-performing Chinese model, followed by iFlytek’s SparkDesk