With the continuous and rapid development of the artificial intelligence industry, AI has been continuously landed in several major industries such as intelligent security, unmanned driving, smart phones, smart retail retail, and intelligent robots, and the Ministry of Industry and Information Technology has issued 5G commercial licenses in advance, and artificial intelligence is detonating a new round of intelligent boom. #人工智能 #
In the long-standing "AI boom", the large models and AI applications launched by various companies have further boosted the demand for computing power fever.
NVIDIA A100 and H100 are currently the most scarce strategic materials, and NVIDIA has begun to significantly increase orders since March.
This week, it has been reported that NVIDIA's subsequent demand for AI top-specification chips has increased significantly, and it has urgently booked additional advanced packaging capacity from TSMC.
Under the new wave of artificial intelligence, the domestic AI intelligent computing center and other digital infrastructure continue to improve, the complexity of AI models and the number of parameters are rapidly increasing, the requirements for computing power are constantly improving, the high-performance artificial intelligence chip market will maintain rapid growth, and all links of the industrial chain are expected to usher in high-speed growth opportunities.
According to iResearch, the market size of Chinese smart chips will reach 39.6 billion yuan in 2022, and the market size is expected to reach 216.4 billion yuan in 2027, with a CAGR of 40.5%. #芯片 #
Overview of the AI chip industry
AI chips are the hardware cornerstone and core of computing power, also known as computing cards or AI accelerators.
Broadly speaking, as long as the chip can run artificial intelligence algorithms, it is called an AI chip. However, AI chips in the usual sense refer to chips that have made special acceleration designs for artificial intelligence algorithms.
At this stage, these artificial intelligence algorithms are generally based on deep learning algorithms, but can also include other machine learning algorithms. Modules dedicated to handling a large number of computing tasks in AI applications.
Pay attention to Leqing Think Tank and gain insight into the industrial pattern!
AI chip industry chain:
Three mainstream routes for artificial intelligence chips
The chips used to execute AI algorithms can take different technical routes.
At present, GPU (general-purpose), FPGA (semi-customized), ASIC (fully customized) have become the mainstream technical route of the AI chip industry. Different types of chips have their own advantages, and show a parallel development trend of multiple technology paths in different fields.
GPU
The design and production of GPUs (Graphics Processing Unit) have matured, occupying the main market share of AI chips.
GPUs are good at massively parallel computing and can process massive amounts of information in parallel, making them the first choice for AI chips.
Processor chips for servers and data centers are themselves growing, and computing still mainly relies on CPUs, with the increase in AI computing power demand, data center computing power shows a trend of diversification, and the proportion of GPUs mainly used for AI computing continues to increase.
According to IDC data, GPUs account for up to 75% of the cloud training market.
Line check | Industry research database data shows that globally, NVIDIA and AMD form a duopoly, especially NVIDIA accounts for 70%-80% of the GPU market share.
Due to the rich chip design experience and technology precipitation of foreign GPU giants, and at the same time strong financial strength, China cannot shake the market pattern of GPU chips in the short term.
GPUs do not involve the complex branch operations commonly encountered by CPUs, becoming the best choice for AI chips:
Source: Ofweek
The market share of domestic manufacturers in the GPU market is less than 1%, and the US sanctions against China accelerate the domestic replacement of GPUs.
In the past few years, it was mainly the United States that pulled China's supercomputing center and related GPU chip companies into the entity list to limit the development of China's AI and supercomputers, but the scope of restrictions was limited to a single scenario of supercomputing.
In September 2022, the United States issued restrictions on high-end GPUs used in AI, HPC and data center research and development, and NVIDIA's A100 and H100 and AMD's MI250 chips suspended sales to Chinese customers.
In October 2022, the United States upgraded the scope of the ban to limit specific parameters such as the connection speed and number of operations per second of high-computing power chips, in addition to NVIDIA and AMD, some products of domestic manufacturers Haiguang Information were also added to the limit.
The United States has expanded the scope of sanctions from application scenarios to the chip and product level, which actually represents that the development of domestic related GPU products or downstream applications exceeds the expectations of the US government.
The United States continues to increase export restrictions on China's high-end chips, and the localization process of high-speed computing-related GPUs, CPUs and other chips is bound to accelerate.
From the perspective of domestic alternatives, manufacturers such as Jingjiawei, Haiguang Information, Holly Technology, and Bicheng Technology are expected to benefit.
Domestic GPU industry chain:
FPGA
FPGA (Field Programmable Gate Array) chip is mainly used for hardware acceleration in the data center field, with the advantages of hardware programming, high configuration flexibility and low energy consumption, and is widely used in cloud computing models released after 2016.
FPGA as an AI acceleration chip:
Source: Muxi Technology
Since November 2022, with the release and open testing of domestic and foreign language models such as Chatgpt-3&4, "Wen Xin Yiyan" and "Pangu Big Model", it is expected to have a strong pull on the AI chip market.
Compared with ASICs, FPGA chips can achieve an excellent balance of performance, flexibility, homogeneity, cost and power consumption.
Compared to CPUs, FPGA chips can provide both powerful computing power and sufficient flexibility due to their instruction-free, memory-free architecture.
Since the data center uses FPGA chips instead of traditional CPU solutions, it can achieve significant acceleration effects when processing its custom algorithms, so that FPGA chips have been widely deployed in Microsoft Azure, Amazon AWS, and Alibaba Cloud servers.
Compared with GPUs, FPGA chips have the advantages of low latency and high throughput in the data center field.
FPGA technical barriers are high, the market is a duopoly: Xilinx and Intel together account for nearly 90% of the market share, of which Xilinx has a market share of more than 50%, and has always maintained a global FPGA supremacy.
Xilinx has the world's leading FPGA products, as well as adaptive SoCs, accelerators and SmartNIC solutions, which complement AMD's capabilities in the data center and edge areas.
Intel invested heavily in Altera to make FPGA technology contribute to Intel's development. It is manifested in the technology roadmap, that is, from the current discrete CPU chip + discrete FPGA acceleration chip, to the CPU chip + FPGA chip in the same package, to the final integrated CPU + FPGA system chip. These product forms are expected to coexist for a long time, as the discrete CPU and FPGA are slightly less performant but more flexible.
Intel FPGA Technology Roadmap:
The largest downstream application field of FPGA is the communication industry, accounting for more than 40%, and the domestic civilian FPGA leaders are Unigroup Tongchuang (Unigroup Guowei holds 30%) and Anlu Technology, which has accelerated verification in the field of communication and continued to grow rapidly.
Under the background of the release of generative pre-trained language models at home and abroad, the FPGA market will be driven by higher and faster.
According to Semico Research, the global market size of FPGAs in the field of artificial intelligence is expected to reach $5.2 billion in 2023, with a five-year compound growth rate of 38.4%.
ASIC
With the trend of chip specialization gradually emerged, more and more chip manufacturers began to try other technical routes to achieve the calculation of artificial intelligence algorithms, such as FPGA and ASIC, of which the most optimistic, and the most open competitive landscape is ASIC.
ASICs (Application Specific Integrated Circuits) are custom chips designed for specific user needs to meet a variety of terminal applications.
Although ASICs require significant physical design, time, money, and verification, they outperform GPUs and FPGAs in performance, power, cost, and reliability after mass production.
Unlike GPUs and FPGAs, ASICs are only a technical route or solution, focusing on solving outstanding problems and management needs in various application fields.
At present, the ASIC chip market competition pattern is stable and fragmented.
The technology gap between China and the United States on ASIC chips is small, which is a good breakthrough for Chinese enterprises. The gap between the mainland's ASIC technology and the world's leading level is small, and some fields are in the forefront of the world.
For companies that have just been involved in the chip field, the technical threshold of ASICs is lower than that of GPUs and FPGAs, because they do not have to pursue high versatility and flexibility, but only need to design for application scenarios, and achieve good performance results by giving up flexibility.
Overseas, Google TPU is the dominant player; Domestic start-up chip companies (such as Cambrian, Bitmain and Horizon), Internet giants (such as Baidu, Huawei HiSilicon and Alibaba) have also made achievements in subdivided fields, and have achieved certain results.
FPGA chip and ASIC cost comparison:
Source: Toupanard Research Institute
AI cloud and terminal chip
According to the deployment location, AI chips can be divided into cloud (such as server side such as data center) and terminal (application scenarios covering electronic terminal products such as mobile phones, automobiles, and security cameras) chips.
At present, NVIDIA is the largest company in the cloud AI chip market, especially in the cloud training end, mainly because NVIDIA GPU has a rich product line, mature programming environment, and products that support major development frameworks and languages in the market.
However, at the same time, its products also have problems such as large power consumption and high price (V100 chip price of 100,000 yuan, DGX series server price of more than one million yuan).
NVIDIA's rich product line and good ecological environment are its core competitiveness:
Based on this, major cloud vendors have proposed their own AI chip development plans to get rid of the monopoly market situation of an upstream AI chip supplier.
In addition, according to the data, the future growth rate and space of the inferred market will be higher than that of the training end market, and GPU chips are not good at inferring tasks.
Therefore, in the current situation that the penetration rate of intelligent servers is still low and GPU products are not perfect solutions, there is still a large market space for other AI chip manufacturers to enter the cloud computing center market.
There are hundreds of schools of thought in the terminal AI chip market, and there is no universal solution.
At present, the application fields of the edge end mainly include security, smart home, intelligent robots, automatic driving, etc.
Because such applications have greater pressure on power consumption, cost and even size, mainly including some Internet of Things and wearable applications, the demand for dedicated processors is more significant. The total number of such applications is large, but the differentiation is obvious, the needs are varied, there are many variables, and it is technically difficult to implement with one architecture.
At the same time, compared with pan-mobile phone chips, the capital investment threshold of such chips is not high. With Nvidia open source DLA, the barrier to entry may be further lowered if combined with open source RSIC-V CPU.
AI training and inference chips
According to the functions undertaken and the goals in practice, AI chips can be divided into training and inference chips.
There are only a few manufacturers of AI training chips in the world, and related chips are mainly used in the cloud in the mode of servers, clusters, accelerator cards, etc., and will gradually appear at the edge.
The threshold for AI inference chips is lower than that of training, and some excellent startups rely on the Fabless model to enter the market. Chip companies can provide dedicated AI chips/IP and development tools.
Training chips are for customers in data centers, automobiles and other industries, and there are also a large number of cloud companies that design their own chips for internal use.
Some inference chips are integrated in hardware terminals, such as mobile phones, cameras, automobiles, mining machines, etc., and some are used to provide services in the cloud.
Some domestic companies are also worthy of attention under the general trend of localization. These companies have greater opportunities in the inference market in edge computing centers, automotive, IoT, and other fields, and industry-specific inference chips may be the way to break the game.
At present, a large number of customized reasoning chips have appeared in the fields of autonomous driving, cameras, and digital currency mining, and the development of the industry will be more clear in the future. The development of the data center training market needs to rely on continuous research and development to narrow the gap.
The overall market structure of AI chips
The complete industrial layout has become an important force to support American AI chip start-ups, so that these start-ups can concentrate their main resources in a certain link, so as to make breakthroughs and achieve commercialization faster.
In the current AI chip market, NVIDIA has a high market share by virtue of its hardware advantages and software ecology.
In terms of algorithms, the complexity of the model continues to increase; In terms of software, the use of AI frameworks is becoming more and more concentrated, chips with large-scale model training capabilities will be more concentrated, and the types of chips supported by software tools will be more concentrated.
Therefore, NVIDIA's advantages are difficult to shake in the short term.
Compared with the wave of mergers and acquisitions of leading companies in overseas markets, a large number of startups have emerged in China to focus on AI chip design in the data center and automotive fields, and have formed an evolution roadmap that is comparable to that of overseas companies. #5月财经新势力 #
In general, Europe, the United States, Japan and South Korea basically monopolize high-end cloud chips, and the domestic layout is mainly concentrated in terminal ASIC chips, and some fields are in the forefront of the world, but most of them are mainly start-ups, and have not yet formed an influential "chip-platform-application" ecology, and do not have the strength to compete with traditional chip giants (such as NVIDIA and Xilinx); In the field of GPU and FPGA, China is still catching up, and high-end chips rely on overseas imports.
Pay attention to Leqing Think Tank and gain insight into the industrial pattern!