Breaking the hegemony of Nvidia's computing power! Technology giants have made a "new Android"

Over the past year or so, the generative AI revolution in Silicon Valley has swept the world and intensified. And in this revolution there are two very obvious threads:

The first is the evolution of basic technology, with large models represented by GPT and Gemini continuing to iterate towards AGI (General Artificial Intelligence);

The second is the emergence of phenomenal applications, and the application frenzy of generative AI in different fields and scenarios is gradually changing the way people live and work, such as the field of AI hardware, which Lei Technology is particularly concerned about, and new applications such as AI mobile phones, AI PCs, AI TVs, AI home appliances, and AI cleaning have emerged.

Under the two main lines, there is also a "dark line" about computing power.

For the development of AI, the shortage of computing power is no longer a one-day or two-day problem, and to this day, major companies continue to snap up GPUs, more precisely, Nvidia's high-end GPUs:

If "Dune" is "whoever gets the spice gets the world", then the AI industry is "whoever gets the NVIDIA GPU gets the world".

Photo/Nvidia

This has led the industry to both love and hate NVIDIA GPUs, on the one hand, because Nvidia's high-end GPUs based on the CUDA platform can be used for AI training simply and efficiently, and on the other hand, everyone is too dependent on Nvidia as a company.

01 CUDA is the cornerstone of NVIDIA's AI

Much has been done to explain the reasons for the success of NVIDIA GPUs, but the core is the high performance of NVIDIA GPUs (including interconnect performance) and the hardware and software synergies brought by the CUDA platform. NVIDIA's advantage in the hardware itself is not difficult to overcome, the key problem lies in the software, in the CUDA platform.

At the opening speech of GTC 2024 some time ago, Lao Huang also reviewed the development history of NVIDIA.

Huang first emphasized the importance of machine learning in 2014, before AlphaGo beat Lee and deep learning didn't garner global attention. However, at the time, Nvidia had already proposed the concept of CUDA (Universal Computing Platform), and it was at the forefront of the AI revolution when many people were still thinking of Nvidia as a manufacturer of "gaming graphics cards".

However, initially, the application scenarios of CUDA were mainly scientific computing, that is, similar to climate simulation, physical simulation, bioinformatics and other professional research, and the application scenarios were very valuable, but narrow. Because of this, NVIDIA's CUDA has not opened the market, and the returns brought to NVIDIA cannot match the corresponding huge R&D investment. Huang needs to explain to the board every year why Nvidia is sticking to CUDA.

In fact, Lao Huang didn't know at that time that NVIDIA's CUDA would usher in computing scenarios such as blockchain "mining" and AI large model computing in the next few years, which was truly rich and rich.

(Source: NVIDIA)

In just two years, Nvidia has created a trillion-dollar AI empire through H100 and H200 chips, and its market value has surpassed traditional giants such as Amazon.

Now, Nvidia's "cards" are in short supply, not only Chinese tech giants such as Byte and Baidu are rushing to stock up on cards to cope with extreme situations, but Silicon Valley tech giants such as Microsoft and Meta are also looking for Lao Huang to buy cards.

In 2024, which is known as the first year of AI applications, NVIDIA's CUDA (Universal Computing Platform) has become general, as its name suggests, from underlying technologies such as large language models, conversational AI, and edge computing, to application scenarios such as intelligent cockpits, autonomous driving, and humanoid robots, and then to applications such as AI mobile phones, AI PCs, AI home appliances, AI search, and AI painting, as well as future climate prediction, computational lithography and 6G networks”。

While Nvidia's cards, as well as the CUDA platform, are becoming more and more important, other tech giants see a crisis of "dominance":

On the one hand, Nvidia's cards are expensive, and they have absolute pricing power, and manufacturers hoard GPU cards to give Nvidia huge sums of money, and the result is: those who do AI do not necessarily make money (almost none of them make money at present), but Nvidia makes a lot of money.

On the other hand, the tech giants were stuck in the neck by Nvidia. Nvidia can decide who the top cards will be given first, how much, and whether to give them or not. In addition, NVIDIA also relies on the computing resources of GPUs to expand to the upper level of business, and set foot in cloud and other businesses, forming a certain competition with technology giants.

In fact, starting in 2023, many chip manufacturers and large model manufacturers are aware of these problems and hope to build another software platform to counter NVIDIA's CUDA. It cannot be said that all attempts have failed, only that no real challenger has yet emerged.

Now, a new alliance and open-source platform could completely change the dominance of Nvidia's CUDA family — just as Google's OHA alliance and the open-source Android against Apple did in the first place. As fate would have it, Google still played a vital role in this round of the war to break the hegemony of CUDA, and Qualcomm and ARM in the Android camp are also playing their key roles.

Nothing new under the sun.

02 巨头组建联盟反抗CUDA

"The whole industry wants to take out CUDA, including Google, OpenAI, and others are looking for ways to make AI training more open. We believe that CUDA's moat is shallow and narrow. At an event late last year, Intel CEO Pat Gelsinger made a surprising statement about his thoughts on Nvidia's CUDA platform.

But even if Kissinger thinks CUDA's moat is "narrow and shallow," he understands that challenging Nvidia in AI training will not be easy.

According to Reuters, the Linux Foundation, together with Intel, Google, Qualcomm, ARM and Samsung, has formed the Unified Acceleration Foundation (UXL), starting with Intel's oneAPI, and is developing an open-source software suite that will allow AI developers to run their code on any AI chip.

Diagram / UXL

OneAPI is Intel's unified programming model and software development framework that enables developers to develop programs across hardware architectures, including Intel's CPUs, GPUs, and FPGAs, without major code modifications.

To put it simply, UXL is a step further on the basis of oneAPI, to achieve a wider range of cross-architecture and cross-platform support, and to remove the strong binding relationship between chip hardware and software.

UXL, which brings together chip manufacturers, large model manufacturers and wafer foundries, undoubtedly wants to replace NVIDIA's CUDA platform and become the preferred development platform for global AI developers. Vinesh Sukumar, head of artificial intelligence and machine learning at Qualcomm, made it clear:

"We're actually showing developers how to migrate from the NVIDIA platform. 」

Even, UXL will eventually support NVIDIA's hardware and code.

In addition to the initial founding members, UXL has also attracted Amazon AWS, Microsoft Azure, and a number of chip vendors. At the same time, according to the plan, UXL expects to finalize the technical specifications in the first half of this year, and complete the technical details by the end of the year.

As for whether UXL can successfully replace CUDA and become the platform of choice for global AI developers, it is clear that a series of proofs are needed, after all, surpassing CUDA:

It's really hard.

03 What is the difficulty in fighting CUDA?

First of all, we need to understand that CUDA is both a programming language and a compiler.

As a programming language, CUDA is a way for developers to communicate with the underlying hardware (GPU) and invoke computing power through CUDA, and it is not difficult to create a new programming language. As a compiler, CUDA undoubtedly has high performance, which means that developers can execute programs more efficiently on the GPU with CUDA, in easier to understand words:

CUDA makes efficient use of the GPU's peak computing power.

CUDA, Photo/Nvidia

Considering the pursuit of large computing power and high performance in today's AI training, it is no wonder that developers prefer CUDA.

But in fact, Nvidia is by no means invincible in the world on these two levels, especially OpenAI's open-source Triton, which can not only achieve execution efficiency close to CUDA on Nvidia's GPUs, but also incorporate code from platforms such as AMD ROCm (benchmarked against CUDA) to be compatible with more GPUs.

The key to CUDA's hard to shake is that it's still an ecosystem.

At Computex last year, Huang revealed that 4 million developers were using the CUDA computing platform. And over the past decade (CUDA was launched in 2007), CUDA has amassed a large number of high-performance libraries and framework code. That's why, even OpenAI complains about the difficulty of programming GPUs with CUDA, more developers are building on CUDA rather than Triton.

On the other hand, NVIDIA's software and hardware co-design also makes this advantage more unbreakable.

You know, Triton is compatible with Nvidia's GPUs, and other GPUs are also compatible with CUDA, and it's not impossible to catch up even in terms of efficiency. But it takes time for software to adapt to hardware, especially on GPUs.

This means that once Nvidia releases a new GPU and CUDA version, whether it is a CUDA-compatible or NVIDIA-compatible GPU, it will need to catch up with Nvidia again.

(Lao Huang's latest release of Blackwell, Photo/NVIDIA)

So to some extent, the only thing that can beat NVIDIA is to adopt the strategy of software and hardware co-design, and have strong chip capabilities and software capabilities.

04 Use "Android mode" to break the hegemony of NVIDIA's computing power

Google has its own TPU, XLA computing platform, as well as its own large model and a series of computing power "exports". But Google doesn't sell it, so Authropic (Claude's parent company) and Midjourney use this computing solution through Google Cloud, instead of buying Nvidia's GPUs.

From this point of view, although UXL gathers from wafer foundries to chip manufacturers, to cloud computing and large model manufacturers, covering the main upstream and downstream of AI chips, the real challenge lies in the collaboration between different members, which is also the key to the success of UXL.

If there is not enough bundle of interests, it will be difficult for every "alliance" to become a climate, and how high-profile it gathers will be, how quickly it will disperse. The key to the success of the Android ecosystem is that participants such as system platforms, semiconductors, hardware, and developers can take what they need and make the pie bigger together. Can UXL create the same positive cyclic effect? At the moment, we don't know the answer.

At the beginning of the year, OpenAI's Sam Altman revealed that he planned to raise $7 trillion to solve the computing power problem faced by AI. Although this number shocked everyone's jaws, it once again shows that the artificial intelligence industry represented by OpenAI is extremely thirsty for computing power - with the support of Microsoft, OpenAI is also laying out its own chip system.

All in all, Nvidia can't satisfy everyone, and everyone isn't satisfied with just one Nvidia. In other words, regardless of whether UXL succeeds or not, whether Google will change its strategy or not, everyone will continue to challenge Nvidia:

Until the hegemony of computing power is broken.