NVIDIA chip rose by 70,000 yuan in a week! GPT drove the price increase, and the gap in the main chip reached 300,000

Source of this article: Times Finance Author: Xie Silin

Image source: Pixabay

The explosion of ChatGPT has driven the demand for AI chips to soar.

As of Friday, Nvidia's latest flagship AI chip, the H100, sold for more than $40,000 a piece on eBay. Compared with the previous retailer's offer of $36,000, the price has been significantly raised.

This price continues to rise. On April 19, when Times Finance searched eBay, it found that a total of 5 stores are currently selling H100 chips, and the price is generally about 45,000 US dollars, of which the highest price is more than 50,000 US dollars. In renminbi terms, this is equivalent to a price increase of nearly 70,000 yuan in less than a week.

In China, the main chip for AI applications, the A100 released by NVIDIA in 2020, its trading price has also soared rapidly in a very short period of time. Wen Qiang (pseudonym), who is engaged in GPU server sales, told Times Finance that in just over three months from the beginning of the year to now, the price of A100 has risen from about 60,000 yuan to 90,000 yuan, and even exceeded 100,000 yuan at one point, an increase of more than 50%.

In Wen Qiang's view, the price of A100 will continue to rise. In August last year, the US government issued a policy banning the sale of two AI chips, Nvidia A100 and H100, to China. At present, the products sold in China are all previously in stock. The industry estimates that there are only 30,000 domestic A100 chips in stock, and odd goods can live.

In order to comply with the US government's export control policy, NVIDIA then announced the launch of the "reduced version" A800 and H800 chips that have been castrated in performance and bandwidth. Limited by production capacity problems, these two chips are also out of stock, and the price has also risen.

Wen Qiang told Times Finance that the price of A800 in his hand has reached 87,000 yuan, and there are only 16 left, while stocks last. If the demand is large and orders are required, the quotation will rise by another 5,000 yuan to 92,000 yuan, approaching the price of the A100 chip.

"One price a day, more expensive every day."

The A100 chip gap is 300,000

Computing power, algorithms, and data constitute the three elements of the AI era, and computing power is infrastructure. How many GPUs and how much computing power are considered to be one of the most critical factors for the development of large voice models and the success or failure of entrepreneurship.

It is precisely because of this that at the moment when a number of giants and entrepreneurs at home and abroad are pouring into the research and development and training of large language models, the market demand for AI chips has soared.

According to Chen Wei, former chief scientist of artificial intelligence NLP enterprises and chairman of Qianxin Technology, if you want to directly train a GPT-3-level large model in China, you need at least 3,000 to 5,000 A100-level AI chips.

This means that there is a huge supply gap in the domestic market. At present, in addition to Alibaba, Baidu and other Internet giants, SenseTime, Kunlun Wanwei, Milli Zhixing, 360, Zhihu and other companies have also officially announced their own large models, if you add Wang Xiaochuan, Wang Huiwen, Lee Kaifu and other entrepreneurs, the number of large models to be launched in China this year has exceeded 10. To meet the needs of these companies alone, 30,000 to 50,000 A10,000-level AI chips are required.

This is only the demand at the level of R&D training. Chen Wei told Times Finance that if you want to actually deploy, the demand for computing power will only be greater. According to conservative estimates, the gap of domestic A100-level AI chips is about 300,000.

"We calculated that if Baidu wanted to access a conversation model like ChatGPT in its search engine, it might need 100,000 A100-level AI chips, which is just a conservative calculation assuming that everyone only talks once."

It is precisely because of this that A800 and H800 chips have been robbed by companies including servers and Internet manufacturers.

A circulating "Ali AI Expert Exchange Minutes" shows that Baidu urgently placed an order for 3,000 A800 servers with 8 chips at the beginning of the year, which means that 24,000 A800s are needed, and Baidu is expected to have a total of 50,000 A800 and H800 demand throughout the year. Alibaba Cloud expects to need about 10,000 this year, of which 6,000 are H800.

Overseas, the giants' snapping up is just as crazy. According to media reports, Musk purchased about 10,000 AI chips to advance the new AIGC project inside Twitter. Since the end of 2022, Microsoft has implemented GPU resource quota supply, but since January this year, the approval time has become longer and longer, and now some applications need to wait days or even weeks to be approved. At the same time, Microsoft ordered tens of thousands of AI chips from NVIDIA with no deadline.

Or slow down the development of domestic large language models

For big language model training, NVIDIA is almost the only winner.

Last year, the BR100 released by the domestic chip company Bicheng Technology has been widely believed to be able to reach the world's leading level in terms of computing power and energy efficiency. However, in terms of supporting product ecology, almost no one can compete with NVIDIA.

An AI chip design related practitioner told Times Finance that the CUDA platform released by NVIDIA in 2006 has long become the most widely used AI development ecosystem, but CUDA almost exclusively supports NVIDIA's Tesla architecture GPU, which makes it difficult for developers to separate from NVIDIA chips for generative AI research and development and training. "There are hardly any chip companies that can shake NVIDIA's dominance."

IDC data shows that domestic GPU servers accounted for more than 88.4% of the domestic server market in 2021, and NVIDIA's products accounted for more than 80%.

However, due to the export control policies of the US government, China's Internet giants and large-language model entrepreneurs cannot obtain the most advanced AI chips in time. In the case that it is difficult to produce alternative products in China, it will undoubtedly affect the development process of domestic large language models.

In Chen Wei's view, if Chinese companies use "reduced-version" chips, the comprehensive bandwidth performance may only be 60% of the flagship model in the same period. This requires more and runs more slowly, and the overall cost is at least 20% to 30% higher. "This is likely to be a watershed between making money and not making money, a watershed between living and not living."