Generating meeting minutes with one click, creating an AI application in one sentence, a task that used to be time-consuming and laborious, is now a breeze with the help of AI. The "magic" of AI is not only reflected in these daily office scenes, it is also changing the way we live. From intelligent recommendation systems to provide us with personalized entertainment options, to autonomous driving technology to make travel safer and more convenient, AI is undoubtedly bringing unprecedented convenience and efficiency to our lives.
However, behind all this "magic", there is an indispensable "magician" - AI chips. It's like the "brain" of AI, processing massive amounts of data and driving complex algorithms. In response to the higher requirements for hardware performance of AI applications, tech giants have launched their own AI chips, such as NVIDIA's H200, Google's TPU, and NPUs integrated into computer products by Intel, AMD, Qualcomm, and Apple.
AI chips have now become a key product for technology giants to compete in research and development, and there are a lot of news and latest progress reports every day.
GPU：NVIDIA H100/200、AMD Instinct MI300、Intel Gaudi 3
The mention of AI makes us think of GPUs, as if GPUs are deeply bound to AI! Indeed, in the field of artificial intelligence (AI), graphics processing units (GPUs) have an inherent advantage. The efficient parallel processing capability of GPUs makes them ideal for mathematical calculations in AI algorithms, especially for processing a large number of matrix operations and deep learning tasks in AI. GPUs are able to perform complex computing tasks faster than central processing units (CPUs), greatly improving the training and inference speed of AI models.
In addition, the parallel architecture of the GPU also enables it to process multiple data blocks at the same time, which is essential for processing large-scale datasets in AI. In the field of deep learning, this capability of GPUs is widely used to accelerate the training and inference process of neural networks, thereby significantly improving the performance and efficiency of AI applications.
Not only that, but GPUs also provide highly optimized libraries and tools that make it easier for developers to implement efficient AI algorithms. These libraries and tools provide powerful support for AI researchers and engineers, enabling them to develop efficient AI applications faster.
At present, representative products include NVIDIA H100/200, AMD Instinct MI300, etc. As large technology companies such as Google and Microsoft have deployed a large number of GPUs in the cloud to provide powerful computing power support, NVIDIA has successfully entered the trillion-dollar market value club with GPUs.
The New Age: The Edge of the Edge
Compared with CPUs and GPUs, FPGAs seem unfamiliar to ordinary users, but in simple terms, FPGAs are equivalent to a kind of "all-purpose" chips, which can be reprogrammed as needed to perform a variety of different tasks. This means that if you need to perform a specific computing task or process a specific kind of data, you can program the hardware logic on the FPGA to do the job in the most efficient way. It's like you can change the tools on your Swiss Army knife whenever you want.
As a result, FPGAs excel in applications that require a high degree of customization and flexibility, enabling them to adapt to different AI needs and improve performance through hardware-level optimizations, such as deep learning acceleration, video image processing, natural language processing, and more.
The advantages of FPGAs are their high performance, low latency, and powerful parallel computing capabilities, which make them excellent when working with complex AI algorithms and large-scale data. At the same time, FPGAs are cost-effective and power-efficient, making them ideal for power-sensitive applications or applications that require long-term operation.
ASIC：Google TPU、AWS Trainium、Azure Maia
In the field of AI, major tech giants such as Google, AWS, and Azure have developed their own dedicated AI acceleration chips, such as Google's TPU (Tensor Processing Unit), AWS's Trainium, and Azure's Maia. These chips are all a type of ASIC (application-specific integrated circuit) that is custom-developed for their respective companies' AI services to provide efficient computing power and optimized performance. These chips are usually not sold separately, but are used as part of a company's internal services to improve its own AI processing power and service quality.
In contrast, companies such as NVIDIA, AMD, and Intel are focused on producing AI-accelerated hardware for a wide range of markets, such as GPUs (graphics processing units) optimized for AI. These companies' products can be purchased and used by a variety of different companies to meet their needs in AI applications. In short, the former is a "private chip" customized by a technology company for its own services, while the latter is a "mass chip" for the market.
类脑芯片：IBM TrueNorth、Intel Loihi 2
A brain-like chip is a completely new processor that is inspired by the design that mimics the structure and function of the human brain's nervous system. Different from the traditional CPU/GPU chips based on the von Neumann architecture, the brain-like chip draws on the concepts of neuroscience and bionics to achieve efficient parallel computing and adaptive learning capabilities by simulating the connection and information transmission of neurons.
In the field of artificial intelligence, brain-like chips have shown many unique advantages. First of all, the massively parallel neuron structure makes it far more powerful than traditional chips, and can efficiently process massive amounts of data at the same time. Secondly, brain-like chips are neuroplastic, which can independently optimize network weights according to application scenarios and continuously improve the level of intelligence. In addition, the low-power event-driven design also makes brain-inspired chips particularly suitable for energy-sensitive fields such as mobile and IoT.
At present, IBM and Intel are representative companies in the research and development of brain-inspired chips. IBM's TrueNorth is one of the trailblazers in the field of neuromorphic computing. Intel's latest Hala Point system is based on the Loihi 2 neuromorphic processor, integrating 140544 neuromorphic processing cores, simulating a total of 1.15 billion neurons and 128 billion synaptic connections, achieving an extremely high performance of 20 quadrillion operations per second, with an energy efficiency ratio far exceeding that of GPU and CPU architectures, opening up a new realm of brain-like computing.
So the question is, what are the NPUs integrated in computer processors such as Intel, AMD, Qualcomm and Apple?
NPU (Neural Network Processor) is an ASIC (application-specific integrated circuit) type of AI chip, and its main advantage is that it is specially tailored for AI inference scenarios, and has natural advantages in computing power density, energy efficiency ratio, and low-latency inference performance. Intel NPU, Apple Neural Engine, and Qualcomm Hexagon AI Engine are all optimized for mobile/IoT and other terminal device scenarios, which are different from the positioning of large-scale AI accelerators in the cloud (such as Google TPU and AWS Trainium).
In contrast, GPUs, as general-purpose parallel computing accelerators, are very suitable for deep learning training due to their flexible architecture and high degree of parallelism. However, GPUs need to be further optimized in terms of inference acceleration, and cannot achieve the ultimate energy efficiency and low latency performance like dedicated NPUs.
In general, NPU is an efficient acceleration chip tailored for the AI inference process, while GPU is a general-purpose accelerator that is more suitable for training. The two form a good division of labor and complement each other in the field of AI acceleration. Therefore, for the AI personal computer of end users, Intel, AMD, Qualcomm, and Microsoft all regard the local heterogeneous computing power of CPU+GPU+NPU as a necessary condition for the definition of AI PC. This hybrid architecture, which combines multiple processing cores, can maximize the advantages of different hardware and bring excellent comprehensive performance to AI computing.
But NVIDIA is not convinced, it believes that with RTX discrete graphics cards, it is a real AI PC!
Of course, NVIDIA definitely has the confidence to challenge the definition of AI PC, because in the AI era, computing power is crucial, and most NPUs are currently designed to be integrated with CPUs and integrated graphics chips, and power consumption is limited to a certain extent, and the use scenarios mainly revolve around continuous low-load AI tasks, such as video conference enhancement, image processing, etc., and the computing power is usually in the range of 10-45TOPS.
In contrast, Nvidia has launched the RTX series graphics cards that can be called the "strongest on the surface" with years of intensive work in GPU research and development. The AI acceleration capability of a single RTX graphics card can easily exceed 100TOPS, and the flagship model is up to 1300+ TOPS, which is fully capable of carrying complex AI workloads such as local large language models.
In addition, NVIDIA has spared no effort in the construction of the AI software ecosystem. There are 125+ RTX-accelerated AI applications worldwide spanning multiple fields such as image/video editing, 3D modeling and rendering, bringing up to 10x faster performance to creators and developers. With NVIDIA's unique AI SDK, developers can maximize the AI acceleration potential of RTX graphics cards.
It is undeniable that NPU does excel in persistent AI tasks with its low-power design, but when it comes to extreme computing power and general-purpose AI application acceleration, the NVIDIA RTX series is still the unrivaled king-level solution. In the era of vigorous development of terminal AI, hardware vendors will not only promote the integration of heterogeneous architectures, but also compete fiercely in the computing power war.