Huang Jenxun put on a Northeast coat to "show goodwill", and China's big factories can hardly hide their AI ambitions

Source of this article: Times Finance Author: Xie Silin Intern Lu Qianying

Image courtesy of Pixabay

The fire of computing power chips will burn from 2023 to 2024.

Shortly after the beginning of the year, Nvidia CEO Jensen Huang visited Chinese mainland for the first time in four years, and also wore a northeast flower coat to dance with domestic employees, and the atmosphere was extremely warm, which seemed to be a response to the rumors that his own chips were cold.

In 2023, Nvidia's AI chips have become the hard currency of the technology industry, which was once hard to find, and Nvidia's performance has also risen, and its financial report for the third quarter of fiscal year 2024 shows that Nvidia recorded revenue of $18.12 billion during the period, an increase of 206% over the same period last year, much higher than the market expectation of $16.2 billion, a record high.

Recently, however, the situation seems to have changed. According to the reference news network, since November last year, China's major Internet manufacturers have begun to test Nvidia's "China Special Edition" AI chip samples, and the results are not ideal - due to insufficient computing power, relevant companies have hinted to Nvidia that the number of Nvidia chips ordered this year will be far less than originally planned.

Therefore, Huang's sudden visit at this special time has sparked a lot of speculation and reverie in the market. Some industry insiders believe that when domestic manufacturers are unwilling to buy NVIDIA special version chips with shrinking performance, Huang Jenxun came to Chinese mainland, most likely to stabilize the domestic team and head customers, and understand the market's demand and feedback for NVIDIA products, so as to make corresponding product and strategy adjustments.

In response, Nvidia also responded to the media that Huang's visit did not involve meetings with government officials or major commercial announcements, and the main purpose was to "have a good time" with Chinese employees.

"This reflects the gradual rise of the status of the mainland's related industries in international competition. Zhang Xiaorong, president of the Deep Science and Technology Research Institute, told the Times Financial Reporter that with the support of the policy side and the continuous efforts of domestic enterprises, the gap between domestic and foreign chip computing power has been narrowing, which has given the domestic head large model manufacturers the confidence to switch to domestic chips, and also sounded the alarm for NVIDIA.

H20 "cold" truth

In October 2023, in order to bypass restrictions and meet the needs of the Chinese mainland market, Nvidia set out to launch three "downgraded" chips based on its AI chip H100: H20, L20 and L2, and plans to mass produce them in the second quarter of this year.

However, this special chip developed for the Chinese market has not been recognized by the market. The lack of performance is generally considered to be the main reason for the cold of Nvidia chips.

According to the comparison of official data, H20 is the best performance among the three chips, and it is more suitable for the training and inference of vertical models. The H20 has up to 96GB of RAM and 296 TOPS of INT8 hashrate, the L20 has 48 GB of RAM and 239 TFLOPS of INT8 hashrate, and the L2 configuration has 24 GB of RAM and 193 TFLOPS of hashrate.

However, even the most powerful of the three chips is the H20 chip, whose computing power has shrunk significantly compared to the H100.

According to the research report of Semianalysis, a semiconductor research institution, the overall computing power of H20 is theoretically about 80% lower than that of NVIDIA H100, but the performance of large language model (LLM) inference is 20% faster than that of H100, and the configuration of HBM3 video memory and NVLink interconnection module at the same time increases the cost.

In addition, a number of industry insiders told the Times Financial Reporter that the performance was not as good as expected, which was not the only reason why H20 was cold in China.

Wu Quan, president of Huaxin Jintong Semiconductor Industry Research Institute, pointed out to the Times Financial Reporter that since the explosion of large models in 2023, a number of domestic cloud computing and large model manufacturers have begun to actively stock up on goods and buy Nvidia chips such as A800 and H800 in large quantities.

Chen Wei, the former chief scientist of artificial intelligence NLP companies and chairman of Qianxin Technology, further revealed to the Times Financial Reporter that there are still a large number of NVIDIA GPU chips hoarded in the domestic trading market waiting to be digested. This also gives domestic large model manufacturers a certain amount of choice.

"It is estimated that 20 per cent of the inventory is still uncleared. Chen Wei said.

Times financial reporters found by asking online channels such as Xiaohongshu and Xianyu APP, as well as offline visits to the Huaqiangbei chip trading market, that compared with the scene of hoarding goods and reluctant to sell half a year ago, there is generally more Nvidia inventory in the hands of dealers, which confirms Chen Wei's statement.

A number of sellers revealed to the Times Financial Reporter that they currently have more than 50 A100 chips in stock, with different configurations and quotations fluctuating slightly, and a single selling price is concentrated in the range of 120,000 to 180,000 yuan, which is stable compared with the second half of last year. There are also sellers who require a minimum of 50-100 pieces, and the delivery cycle is 1 month, which can be traded in Hong Kong, Japan, and Singapore.

In addition, Chen stressed that the current wait-and-see attitude of the market is also closely related to Nvidia's own actions.

In addition to H20, at the end of 2023, Nvidia China released a customized GeForce RTX 4090 D on its official website. Then, in early January this year, the GeForce RTX 40 SUPER series of consumer graphics cards was launched, including the RTX 4080 SUPER, RTX 4070 Ti SUPER, and RTX 4070 SUPER GPUs, all of which have powerful generative AI performance. Justin Walker, vice president of Nvidia, stressed at the press conference that the new graphics cards released by Nvidia meet export requirements and can be provided to consumers in Chinese mainland.

Chen Wei believes that considering that the development trend of most domestic large model manufacturers is to develop small and medium-sized models with billions of parameters or tens of billions of parameters for vertical fields, rather than developing hundreds of billions of large models like GPT-3.5 175B. Therefore, for cloud computing and IDC vendors, consumer-grade graphics cards such as 4090D are more cost-effective in cloud deployment and inference computing, and the attractiveness of H20 has further decreased.

Domestic self-developed AI chip acceleration

At the same time as it was revealed that H20 and other "downgraded" chips were cold in China, the relevant report of the reference news network said, "In the short term, the performance gap between Nvidia's downgraded chips and China's local chips has been narrowing, enhancing the attractiveness of China's self-produced chips." According to people familiar with the matter, Alibaba and Tencent are shifting some orders for high-performance AI chips to local companies and relying more on in-house developed chips. The same goes for China's other two largest chip buyers, ByteDance and Baidu. ”

Through interviews and public information, the financial reporter of the times found that the current domestic AI chips can be divided into three categories: one is the AI chip developed by large technology companies, the second is the state-owned technology company such as Haiguang Information, and the third is the entrepreneurial chip companies such as Biqian Technology, Tiantian Zhixin, Moore Threads, Suiyuan Technology, Cambrian and so on.

According to late statistics, most of the peak computing power of the above chips is equivalent to 41% to 82% of Nvidia A100.

Judging from the actual implementation situation, the vast majority of manufacturers are still in the early stage of commercial application, and they are still stuck in the application of specific scenarios.

At present, there are domestic technology companies with full-stack AI software and hardware products, which can be truly mass-produced and put into business use. The core product developed by it is widely regarded as the domestic AI chip with the closest performance to NVIDIA. Jiang Tao, vice president of iFLYTEK, once revealed at a performance briefing that the chip's capabilities have basically been benchmarked against Nvidia A100.

According to incomplete statistics from Times Finance, many technology companies such as iFLYTEK, Baidu, 360, Yuncong Technology, and Zhipu AI have been using the chip.

In fact, the competition of computing power of AI chips is only the first step, and for domestic manufacturers, the next ecological construction is a new challenge. A number of practitioners and analysts also agreed that from multiple perspectives such as supporting ecology and supply chain, it is still difficult for large model manufacturers to completely get rid of their dependence on NVIDIA GPUs.

This is not a problem for a domestic manufacturer. A practitioner related to AI chip design told Times Financial Reporter that the CUDA platform released by NVIDIA in 2006 has long become the most widely used AI development ecosystem, which makes it difficult for developers to break away from NVIDIA chips for generative AI research and development and training.

"From this perspective, there are few chip companies that can shake Nvidia's dominance. The above-mentioned practitioner said.

There are also executives of listed AI companies who revealed to the Times Financial Reporter that at present, the price of domestic chips is basically the same as that of Nvidia products, and the production capacity and shipment are difficult to meet expectations. "Only by becoming an ecological partner can we barely guarantee supply. ”

"As long as it's incremental, it's a good trend"

For domestic large-scale model manufacturers, increasing the proportion of domestic AI chips is still an inevitable move.

"Ensuring supply has become a key issue. On the one hand, the United States is likely to take various measures to ensure its superiority in the field of artificial intelligence, and on the other hand, the global situation may also affect the stability of supply chains. Based on this expectation, the market needs to find new solutions as soon as possible, which will become an opportunity for the development of domestic AI chips. Chen Wei thinks.

In order to seize this opportunity, a number of domestic AI chip manufacturers are focusing on expanding the market. According to the information provided by Cambrian to the Times Financial Reporter, its Siyuan chip has recently announced that it has completed a comprehensive adaptation with the large models of Baichuan Intelligence, Zhixiang Future and other manufacturers.

"Ecological construction is the goal, the chip is only the foundation, and more importantly, a series of ecosystems such as architecture platforms, software tools, operating systems and application scenarios are built on top of the chip. Nvidia started early, and domestic manufacturers need to catch up quickly. Wu Quan analyzed to the financial reporter of the times.

According to data disclosed by the International Data Corporation (IDC), in the first half of 2023, the market size of China's acceleration chips will exceed 500,000. In Chen's view, this is an opportunity to increase the market share of that domestic market. "Just gradual is a good trend. AI chips need to be used in practical applications to find problems and speed up iteration. We generally believe that only after the market share exceeds 15% can ecological construction really run. ”

Wu Quan vividly compared the development of domestic AI chips to the growth of trees, "At present, we have only planted a small sapling and completed the most basic cultivation work. Domestic enterprises still need to make more efforts in technological innovation, talent training, and industrial chain improvement. ”