The future of computing has three pillars, CPU and GPU have been seated in two seats, who is finally eligible to be selected? NVIDIA CEO Jen-Hsun Huang gave his own answer: "The third main chip" is the DPU. [1]
The three pillars of computing have different roles. The CPU is equivalent to a housekeeper, responsible for thinking about and processing various businesses; the GPU is equivalent to an artist, specializing in various image processing work; and the DPU is equivalent to the front desk, responsible for packaging and unpacking various data[2]. Although DPU is a descendant of the chip industry, due to the scale of the market, the boom in the past two years has surged and new players around the world who have entered the game madly want to take a spoonful from this sea.
This article is the fifth article in the "domestic substitution" series planned by "Fruit Shell Hard Technology", focusing on the domestic substitution of DPUs. In this article, you will learn: what is a DPU, what a DPU can do, the history of the development of this concept, and the opportunity for domestic DPUs.
Fu Bin | author
Li Tuo | edit
Hull hard technology | mastermind
Crazy DPU track
DPU (Data Processing Unit) is a smart network card with computing power that is used to share tasks such as networking, storage, and security that are inefficient to run THE CPU and difficult for GPUs to handle, in order to improve the efficiency of computing systems and reduce the total cost of ownership (TCO). [3]
How crazy is the domestic DPU market? The catch-up and preemptive publicity of startups is the most direct manifestation.
DPU related domestic manufacturers investment and financing, watchmaking 丨 fruit shell hard technology
References 丨Public information
Due to the relatively new concept of DPU and the time for market introduction, domestic DPU start-ups pay great attention to product design and development and mass production speed.
DPU related domestic manufacturers, watchmaking 丨 fruit shell hard technology
"Rushing" is not impetuous, but because the existing overseas products themselves are expensive, coupled with the small shipment volume, agent purchase will further increase the cost. More importantly, the products of overseas companies in the exploration stage are not very perfect, and the opportunity for domestic DPU is very large.
Lu Sheng, chairman and CEO of Xin Qiyuan, once introduced to the author that DPU is a place where soldiers must compete, and the world's top five giants are actively laying out, who occupies this territory, it occupies the entire data center and network center platform.
Not only that, the demand for DPUs continues to rise, and the global DPU market is expected to reach 10 billion US dollars in 2024[4], of which the Chinese DPU market will grow from more than 1 billion US dollars in 2022 to 3.741 billion US dollars in 2025, with a compound growth rate of more than 50%. [5]
Major investment institutions are also optimistic about the DPU track, believing that it is the hub of the next generation of data centers and is just needed, with tens of billions of dollars of incremental value. [6]
Why it must be DPU
How did the DPU catch fire?
This is because cpu performance increases cannot keep up with the speed of data generation: according to CBN data, network bandwidth increased by about 30% annually before 2010, and has now increased to 45%, while CPU performance growth has now dropped from 23% 10 years ago to 3% [7]. This gives the DPU a chance.
Initially, network-related interventions were done by the Network Interface Card (NIC), but in order to handle network protocols, it squeezed out some CPU power, and it only had some very basic task offloading functions. [8]
In order to unload these crowded resources, smart network cards (SmartNIC) came into being. [10] The network card becomes smarter and requires the loading of running code, which is equipped with additional computing power and on-board memory,[9] to reduce the consumption of CPU resources such as network interconnection, security, virtualization, and memory access, and provide more CPU resources for applications.
This smart network card has also been deployed in the data center for a long time. Since 2015, Amazon[11] and Microsoft[12] have deployed smart network cards in their own businesses.
Since you have a smart network card, why do you need a DPU? Because DPUs support more features, including bandwidth compression, secure encryption, and virtualization of network functions, although these functions have a thin sense of existence for ordinary users, they are more efficient, safer, and more real-time guarantees for existing daily applications [13]. DPU equipped with these functions can better solve the problems of network protocol processing, data security, algorithm acceleration and other issues than smart network cards, and these functions are not good for THE CPU and the GPU for it. [14]
Three stages of DPU development
Image source丨 Special Data Processor (DPU) Technology White Paper, Institute of Computing Technology, Chinese Academy of Sciences, Yan Guihai, etc
Although the DPU is so powerful, it is just an assistant to the CPU.
Since the development of digital chips, "IT has been difficult for CPUs to do everything well" has become the consensus of the industry, but it is indispensable. Chip engineer Lao Shi said that in the paper in the field of algorithm acceleration, the CPU is the existence of sadness, and devices that are several times, dozens of times, and hundreds of times stronger than the CPU abound. But let time flow, the CPU still stands on the throne of the data center, but there are more GPUs next to it, and there is a DPU that is ready to move. [15]
Huang Chaobo, CEO of Shanghai Matrix Technology and former head of UCloud chip and hardware research and development, analyzed that CPUs, DPU and GPU are both collaborative and competitive with each other. The traditional view is that the DPU only acts as a task offload of the CPU, but according to the perspective of software and hardware fusion, the DPU and the embedded CPU/GPU constitute a super-heterogeneous computing platform, and the independent CPU/GPU is the expansion of the DPU. [16]
The role of DPU, image source 丨Socnoc[17]
The existing DPU market is very extensive
Behind the birth of DPU is a tortuous and tossing naming tug-of-war, which leads to the current DPU market is very extensive, the standard is uneven, and the positioning and market strategy are in full bloom.
In 2019, Fungible defined the DPU as a new type of data processing unit, but it didn't make a splash at the time. Subsequently, Nvidia acquired Mellanox in June 2020, repackaged its original IPU series (products with different names for the same function) as DPU, and began to publicize it with great fanfare in October 2020,[18] which made DPUs known to everyone. In June 2021, Intel released its own DPU, but renamed it "Infrastructure Processor" (IPU). [19]
The history of the development of DPU, watchmaking 丨 fruit shell hard technology
After that, the industry has three new interpretations of the "D" in DPU, including Data, Datacenter, and Data-centric, focusing on data, data centers, and data centers, respectively. [20]
Different interpretations of DPUs
Inconsistencies in the naming of DPUs have led to different understandings of vendors' marketing strategies: NVIDIA (i.e., Mellanox) for data security, network security, and storage offload, Fungible for networking, storage, and virtualization, Intel, Broadcom for switches and router chips, Marvall for 5G baseband, and Pensando for P4-enabled software-defined networking (SDN). [21]
DPU Competitive Landscape, Source丨 CCID Consultants[22]
This difference in positioning has led to different understandings of different manufacturers when choosing architectures, and is currently divided into three main technical forms: SoC (Arm and ASIC collaborative architecture), FPGA, and ASIC. FPGA-based DPUs have the best flexibility, but throughput and power consumption are limited to some extent; ASIC-based DPUs will increase as bandwidth becomes higher; and For more complex and broader use cases, SoC-based DPUs have good price/performance, ease of programming, and high flexibility.
Comparison of different technical implementation parameters of smart network card, source丨 CCID consultant[22]
This difference will lead to two problems, one is what the DPU should do, how to connect with the computing platform and understand the business scenario, and the other is how to choose the technical solution. [23]
How to break through the domestic DPU
Since DPU is a new thing, the different needs of the entire industry and the closed system will not change in the short term.[21] At the same time, DPU products have not really achieved scale and commercialization in the global market, and this highly integrated device is very difficult to build, and the existing products cannot meet the industry's expectations for DPUs in the general field [24]. In addition, the dispersion and non-singularity of customer needs in the DPU market further exacerbates the differentiation of application scenarios and architectures. [25]
Intel once said to this writer: "DPU/IPU is obviously a big market, but it is gradually no longer a market that startups can dominate, but a market dominated by large companies." "Intel, which has always been very confident in CPUs, has also entered this market, indicating that the era when all computing was handled by CPUs is gone, and it is enough to illustrate how important this field really is."
Since DPU is a concept proposed by international giants and gives priority to mass production products, the existing market is mainly covered by them, in this case, how should domestic DPU start-ups survive?
First of all, there should be a deeper understanding of the application scenario. "The DPU needs a brand new unit, not a patchwork of original switches or CPUs. Li Shuang, CEO of Dayu Zhixin, believes that the DPU cannot be simply defined as a chip, but should be regarded as a system, which is an architectural transfer. Designing a competitive and easy-to-use DPU is key to a deep understanding of the scene. [26]
Coincidentally, Li Xiaojun, a partner at IDG Capital, also said that the DPU field needs to fully understand the business needs of large-scale cloud vendors, define architectural characteristics, and have the engineering implementation capabilities of large chips. [27]
Second, use different solutions to adapt to different customers. Zhang Binlei, senior analyst of Xinmou, believes that domestic DPU companies are more suitable for entering the small and medium-sized customer market through FPGA and NP and other schemes with good applicability. [25]
Finally, differentiate competition, rapid iteration. Sun Zhuoyi, an analyst at CCID Consulting Integrated Circuit Industry Research Center, suggested that domestic DPUs adopt the idea of differentiated competition, accelerate the process of independent research and development, accelerate the iteration speed of products, and find subdivision product angle innovation in business models. [25]
References：
[1] NVIDIA：What Is a DPU?. 2020.5.20.https://blogs.nvidia.com/blog/2020/05/20/whats-a-dpu-data-processing-unit/
[2] Semiconductor Industry Watch: Promising DPU Chip.2021.4.8.https://mp.weixin.qq.com/s/tBP2HYK0nfYY6ZC0afJb-A
[3] Institute of Computing Technology, Chinese Academy of Sciences: The Institute of Computing Sciences, Chinese Academy of Sciences, led the release of the DPU Technical White Paper.2021.10.21.http://www.ict.cas.cn/xwgg/jssxw/202110/t20211021_6226618.html
[4] Jaguarmicro: Clouded Leopard Intelligent released a full-featured Cloud DPU network card, leading the new trend of data center.2021.12.15.https://mp.weixin.qq.com/s/y5gpY6_3vrtUvU08-c1v8Q
[5] Open Source Securities Asset Management: Industry Observation丨Summary Demand Improvement in the Cloud Computing Era-DPU Track Beginning to Emerge.2022.3.9.https://mp.weixin.qq.com/s/EaZ-3fr4ZMoky3pfG1tUQQ
[6] Daily IC: Capital-driven DPU "core" battlefield, hundreds of millions of yuan has become the investment standard?.2021.8.21.https://mp.weixin.qq.com/s/t2VIA2CAsvqZ3H29O77cMA
[7] Lai Salsa. DPU: The Tens of Billions of Tracks[N].CBN Daily, 2021-12-30(A10)
[8] Hardware-software convergence: How to distinguish between the definitions of SmartNIC and DPU? Introduction to the STH NIC Classification Framework.2021.8.31.https://mp.weixin.qq.com/s/0wohV_OIvi3Tubn1q5daEw
[9] Xilinx Xilinx: What are the significant differences between SmartNIC and ordinary NICs?.2020.10.https://mp.weixin.qq.com/s/c9yh1EnF0FpA-c-uStTPcg
[10] CIIC Watch: AWS/Alibaba Cloud have self-researched, Intel/Nvidia compete for layout, what is the magic of DPU?.2021.8.18.https://mp.weixin.qq.com/s/MyqEt_kCSFbdYhvXviyMfA
[11] International electronics offer: DPU market, a scuffle.2021.12.20.https://mp.weixin.qq.com/s/BPOGI1MpvtnUaauB5L2Tmg
[12] 21ic electronic network: 80% of cloud service providers face a "networking crisis", why must be FPGA?.2020.3.9.https://www.21ic.com/article/697539.html
ZHAO Guangli. Computing chip "upstart" DPU, road ahead geometry[N].China Science Daily, 2021-11-04(003)
ZHANG Xinyi. Data center "the third main chip" DPU to start a card slot war[N].China Electronic News, 2022-01-18(007)
[15] Lao Shi Tanxin: DPU in the Spotlight: How to Open an Era.2021.11.05.https://mp.weixin.qq.com/s/ObRQDIpFdFtWQQ2EhC06Rg
[16] iTalk Salon: Another hundred million yuan financing! .2021.12.28.https://mp.weixin.qq.com/s/tZZB8TOpzPCom7HIwsNHCg
[17] Socnoc Official Website: https://www.socnoc.ai/
[18] Qubits: NVIDIA's big wave of hardware is coming: Today GTC released professional GPUs and DPUs, and will build supercomputing in the UK for the acquisition of ARM.2020.10.6.https://mp.weixin.qq.com/s/aQYgo5IWh9Aub5ooGjettg
[19] Intel introduces a new Infrastructure Processor (IPU).2021.6.15.https://mp.weixin.qq.com/s/7cxb-CyCmSo7Yq9oaOP9fg
ZHAO Guangli. Computing chip "upstart" DPU, road ahead geometry[N].China Science Daily, 2021-11-04(003)
[21] Yan Guihai et al. DPU: Data-centric dedicated processor[J].Communications of China Computer Society, 2021,17(10)
[22] CCID CONSULTING CO., LTD.: China DPU Industry Development White Paper.2021.12.9.https://microsite-wx-industries.nvidia.cn/Uploads/20211209/61b1809e68fba.pdf
[23] Zhongke Chuangxing: "Dayu Zhixin" completed tens of millions of yuan pre-A round of financing, and the first 25G DPU product in China entered the commercialization | Chuangxing Portfolio.2021.7.9.https://mp.weixin.qq.com/s/sVIsuTHlGMJou26tva4clw
[24] Pencil said: "Dayu Zhixin" raised tens of millions of dollars to develop the first 25G DPU product in China into commercialization.2021.7.9.https://www.pencilnews.cn/p/39233.html
Xu Zihao. A number of different entry routes DPU staged differentiated competition[N].China Electronic News, 2021-08-13(008)
[26] Da Yu Zhixin: DPU: Why is it the need of the times? .2021.11.22.https://mp.weixin.qq.com/s/1NGM-Ran0NqSQ9GiZO1M4A
[27] YunmaiXinlian: Cloud-network integration of "core" forces, Yunmaixinlian received hundreds of millions of yuan of angel round investment.2021.10.22.https://mp.weixin.qq.com/s/rtr_lYLSHIOVBzGwpRDWaw