OpenAI spent $80M to $100M training GPT-4; Chinese firm claims it trained its rival AI model for $3 million using just 2,000 GPUs | TechRadar

Source: original

OpenAI spent $80M to $100M training GPT-4; Chinese firm claims it trained its rival AI model for $3 million using just 2,000 GPUs

News

By Wayne Williams published 2 December 2024

01.ai’s Yi-Lightning AI model is ranked sixth globally in performance

A hand reaching out to touch a futuristic rendering of an AI processor. (Image credit: Shutterstock / NicoElNino)

Share this article

Join the conversation

Add us as a preferred source on Google

Newsletter

Subscribe to our newsletter * 01.ai trained an AI model for $3 million using 2000 unnamed GPUS * “Efficient engineering” allows 01.ai to compete globally, company claims * 01.ai reduced inference costs to 10 cents per million tokens Tech companies in China face a number of challenges due to the American export ban, which restricts access to advanced hardware from US manufacturers.

This includes cutting-edge GPUs from Nvidia, critical for training large-scale AI models, forcing Chinese firms to rely on older or less efficient alternatives, making it difficult to compete globally in the rapidly evolving AI industry.

However, as we’ve seen time and again, these seemingly insurmountable challenges are increasingly being overcome through innovative solutions and Chinese ingenuity. Kai-Fu Lee, founder and CEO of 01.ai, recently revealed that his team successfully trained its high-performing model, Yi-Lightning, with a budget of just $3 million and 2,000 GPUs. In comparison, OpenAI reportedly spent $80-$100 million to train GPT-4 and is rumored to have allocated up to $1 billion for GPT-5.

Making inference fast too

“The thing that shocks my friends in the Silicon Valley is not just our performance, but that we trained the model with only $3 million," Lee said (via @tsarnick).

"We believe in scaling law, but when you do excellent detailed engineering, it is not the case you have to spend a billion dollars to train a great model. As a company in China, first, we have limited access to GPUs due to the US regulations, and secondly, Chinese companies are not valued what the American companies are. So when we have less money and difficulty to get GPUs, I truly believe that necessity is the mother of invention."

Lee explained the company’s innovations include reducing computational bottlenecks, developing multi-layer caching, and designing a specialized inference engine. These advancements, he claims, result in more efficient memory usage and optimized training processes.

“When we only have 2,000 GPUs, the team has to figure out how to use it,” Kai-Fu Lee said, without disclosing the type of GPUs used. “I, as the CEO, have to figure out how to prioritize it, and then not only do we have to make training fast, we have to make inference fast... The bottom line is our inference cost is 10 cents per million tokens.”

For context, that’s about 1/30th of the typical rate charged by comparable models, highlighting the efficiency of 01.ai's approach.

Some people may be skeptical about the claims that you can train an AI model with limited resources and "excellent engineering", but according to UC Berkeley’s LMSIS, Yi-Lightning is ranked sixth globally in performance, suggesting that however it has done it, 01.ai has indeed found a way to be competitive with a minuscule budget and limited GPU access.

TOPICS

Wayne Williams

Editor

Wayne Williams is a freelancer writing news for TechRadar Pro. He has been writing about computers, technology, and the web for 30 years. In that time he wrote for most of the UK’s PC magazines, and launched, edited and published a number of them too.

Skymizer HTX301 GPU Pro Tiny company steals AMD's thunder and challenges Nvidia with old-tech PCIe AI accelerator

Razer April Fool 3D campaign Pro Razer generates 11,000 AI images without cloud subscriptions using decentralized marketplace

An Nvidia GB200 server rack in a data center with one rack open and exposed. Pro 'It was awesome to see it work': OpenAI deploys GPT-5.5 Codex across Nvidia Blackwell systems – 50x efficiency boost and 35x cost reduction makes AI ‘viable at enterprise scale’

Nvidia GTC 2025 Jensen Huang keynote Pro Nvidia CEO Jensen Huang warns of "enormous" China compute capacity