- tech360.tv
Baidu and Zhipu AI's Large Language Models Lead Chinese Rankings, but Overseas Competitors Remain Ahead in Overall Performance
Baidu's Ernie Bot 4.0 and Zhipu AI's GLM-4 lead Chinese large language models (LLMs) rankings. Overseas models, such as OpenAI's GPT-4 and Anthropic's Claude-3, outperform Chinese models in multiple capabilities. Chinese LLMs show better performance in Chinese text-language tasks.
In a recent assessment conducted by Tsinghua University in Beijing, Baidu's Ernie Bot 4.0 and start-up Zhipu AI's GLM-4 emerged as the top-performing Chinese large language models (LLMs). However, when it comes to overall capabilities, foreign models such as OpenAI's GPT-4 and Anthropic's Claude-3 still hold the lead.
The SuperBench assessment report evaluated 14 representative LLMs, which are the backbone of generative artificial intelligence (AI) chatbots. The findings revealed that overseas models outperformed their Chinese counterparts in various areas, including semantic comprehension, coding abilities, and alignment with human commands. The researchers noted "obvious gaps" in code-writing and operative abilities between domestic and foreign models in real-world scenarios.
The objective of the report was to provide a scientific evaluation of the growing number of LLMs that have emerged recently. Tsinghua's Basic Model Research Centre, in collaboration with the state-backed Zhongguancun Laboratory, conducted the assessment.
Chinese tech giants and start-ups have been striving to enhance their LLMs since the launch of innovative tools by US start-up OpenAI, backed by Microsoft. However, despite the efforts, the Tsinghua report aligns with Alibaba Group Holding co-founder and chairman Joe Tsai's recent comment that China is lagging behind US companies in the global AI race. Tsai highlighted OpenAI's significant advancements in AI innovation, which have surpassed the rest of the tech industry.
The report also sheds light on the challenges faced by Chinese LLM developers. Revisions to US export controls have made it more difficult for China to access advanced AI processors and semiconductor-manufacturing equipment. This has further contributed to the gap between Chinese and foreign LLM capabilities.
Despite these challenges, the report indicates that Baidu's Ernie Bot 4.0 and Zhipu AI's GLM-4 have made significant progress in narrowing the gap with the world's best models in terms of overall performance. Notably, Chinese LLMs performed better in Chinese text-language tasks, with Moonshot AI's Kimi chatbot, Alibaba's Tongyi Qianwen 2.1, GLM-4, and Ernie Bot 4.0 ranking among the top four in that category. However, OpenAI's GPT-4 still claimed the first position in Chinese text-language reasoning.
Zhipu AI, along with Moonshot AI, Baichuan, and MiniMax, are recognised as the "four new AI tigers" in China, representing some of the country's most promising generative AI start-ups. Zhipu AI, founded by a Tsinghua graduate in 2019, has raised 2.5 billion yuan (US$347 million) from investors including state-affiliated entities, venture capitalists, and major tech companies like Alibaba, Tencent Holdings, and Meituan. Similarly, Moonshot AI, based in Beijing, secured a funding round of US$1 billion in February, according to Chinese media reports.
In conclusion, while Baidu and Zhipu AI's large language models have achieved top rankings among Chinese LLMs, foreign competitors like OpenAI and Anthropic still maintain an edge in overall performance. The Tsinghua University assessment highlights the need for continued development and improvement in Chinese LLMs to bridge the gap with global leaders in the AI industry.
Baidu's Ernie Bot 4.0 and Zhipu AI's GLM-4 lead Chinese large language models (LLMs) rankings.
Overseas models, such as OpenAI's GPT-4 and Anthropic's Claude-3, outperform Chinese models in multiple capabilities.
Chinese LLMs show better performance in Chinese text-language tasks.
Source: SCMP