Who can "dump" NVIDIA first?

OpenAI is also riding a donkey to find a horse, trying to get rid of its dependence on Nvidia as soon as possible.

According to Reuters, since at least last year, OpenAI has been discussing various options in the hope of solving the problem of expensive and short supply of chips. Among them, self-developed chips are one of the solutions, and this scheme has not been completely rejected.

Another option is to buy a chip company outright. People familiar with the matter said OpenAI already has potential acquisition targets and has considered due diligence on them. However, the report could not determine which chip company specifically.

Coincidentally, another news came along with Microsoft at next month's annual developer conference, launching its first AI-designed chip, Athena.

According to The Information, citing people familiar with the matter, Athena will be used for data center servers, designed for training large language models, etc., while supporting inference, which can power all the AI software behind ChatGPT.

The cloud has become an important battleground for big models, and Microsoft's two competitors in the field, Google and Amazon, already have their own AI chips. The launch of Athena will allow Microsoft to make up for its shortcomings.

The progress of Microsoft and OpenAI on the chip issue is quite representative: in terms of role, it is the tripartite cooperation of Microsoft, OpenAI and NVIDIA that makes ChatGPT a reality, which then triggers a new wave of global AIGC; In terms of timing, next month is the point where ChatGPT will launch for an entire year.

The next focus of the big model competition seems to be "who can 'dump' NVIDIA first", and NVIDIA, which has a dominant power in the chip field, has become a shackle that needs to be extricated.

A

In 2016, OpenAI, which was established only one year ago, welcomed a distinguished guest, Nvidia CEO Jensen Huang. He personally gave the first lightweight and small supercomputing DGX-1 to OpenAI, which can perform a year's computing volume with the DGX-1 in a month.

Now, people who are hindsighted look back at Huang Jenxun's signature on DGX-1 "for the future of computing and mankind" and exclaim the vicious eyes of the "leather-clad godmaster".

In 2019, Microsoft joined hands with OpenAI to build tens of thousands of NVIDIA A100 GPUs for its supercomputer. In this way, OpenAI contributed, Microsoft paid, NVIDIA provided infrastructure, supported the research and development of OpenAI large models with amazing computing power, and finally made a miracle, ChatGPT was launched in November 2022, stunning the world.

OpenAI has become a star company, Microsoft has fiercely fought with Google with an AI strategy, and the market value of NVIDIA has soared from more than $300 billion in November last year to more than trillion dollars today. The world has set off a big model fever, NVIDIA as a "water seller", chips do not worry about selling.

In July, Citi research analyst Christopher Danely noted in a report that Nvidia will capture "at least 90 percent" of the AI chip market.

However, in this "win-win-win" game, only Huang Jenxun may be completely happy. For the "water buyers" represented by Microsoft and OpenAI, there are at least two problems with relying on NVIDIA's chips.

The first problem is expensive. For supercomputing built for OpenAI, according to Bloomberg, Microsoft spent hundreds of millions of dollars on the project. Bernstein Research analyst Stacy Rasgon analyzed that ChatGPT costs about 4 cents per query. If ChatGPT's query volume grows to one-tenth of Google's search, it would require about $48.1 billion in GPUs and another $16 billion a year in chips to keep running.

The second problem is scarcity. Just in June, OpenAI CEO Sam Altman said at a conference that the chip shortage is hindering the development of ChatGPT. Faced with customer complaints about API reliability and speed, Altman explained that most of the problems were due to chip shortages.

The newly released NVIDIA H100 this year is currently the hottest AI chip, but it can only meet half of the market demand. Both the Nvidia H100 and A100 are produced by TSMC, and TSMC Chairman Deyin Liu explained last month that the supply constraint was not due to a lack of physical chips, but limited capacity for Advanced Chip Packaging Services (CoWos), a critical step in the manufacturing process.

Liu Deyin also expects that the technology production capacity will be enough to meet customer demand in a year and a half, that is, the tight supply of AI chips may be eased by the end of 2024.

While Athena may not launch until this year, Microsoft has been preparing for it for years. In 2019, when hundreds of millions of dollars were spent on supercomputing for OpenAI, Microsoft's Athena project has been launched. According to the news, Athena will use TSMC's 5nm process to build and directly benchmark the NVIDIA A100, which is expected to reduce the cost of each by one-third.

B

For Nvidia, the selfishness of Microsoft and OpenAI is a red signal.

Microsoft is one of NVIDIA's largest customers, and there has even been news of the annual production capacity of "Baoyuan" H100, and OpenAI is the most important wind vane in the field of AIGC. The heart of the two self-developed chips is a dark cloud above NVIDIA's head.

Google was the first company to purchase GPUs on a large scale for AI computing, but later developed its own AI-specific chips. The first generation TPU (Tensor Processing Unit) was released in 2016, followed by the Google Cloud infrastructure Google TPU in 2017. Google has continued to iterate over the years, and in April this year it released details of TPU v4, saying that it is 1.7 times stronger than Nvidia's A100.

Although Google is still buying Nvidia GPUs in bulk, its cloud services have used its own TPUs. In this AIGC war, AI mapping company Midjourney and Anthropic, a unicorn company with ChatGPT's competitor Cloude, did not purchase chips from NVIDIA to build supercomputing like OpenAI, but used Google's computing power.

Another tech giant, Amazon, was also early on when it acquired Israeli chip startup Annapurna Labs in 2015 to develop custom chips for its cloud infrastructure, and three years later launched Graviton, an Arm-based server chip. Later, Amazon launched the artificial intelligence-focused chip Inferentia, Trainium.

Last month, it was announced that Amazon would invest $4 billion in Anthropic as part of the deal, which will use AWS Trainium and Inferentia chips to build, train and deploy its future base models.

In addition, NVIDIA's other competitors are also attacking the field of AI chips. AMD, Intel, IBM, etc. are successively launching AI chips to try to compete with NVIDIA's products. In June this year, AMD released the Instinct MI300, which is directly benchmarked against the NVIDIA H100, which is an accelerator specifically for AIGC. The number of transistors integrated reached 153 billion, higher than the H100's 80 billion, and it is the largest chip since AMD put into production. AMD even uses a strategy compatible with NVIDIA CUDA to lower the migration threshold for customers.

It is undeniable that at present, NVIDIA still almost monopolizes the AI chip market, no competitors shake its position, and no technology giant can completely get rid of its dependence on it.

But "reducing Nvidia's control" seems to be a consensus, and external challenges are coming in waves. The news of Microsoft's and OpenAI's self-developed chips is a new wave. Can NVIDIA stand firmly on the shore?

Resources:

1, the heart of the machine: "Amazon just invested $4 billion, Google and others want to invest another $2 billion, Anthropic's valuation is skyrocketing"

2, Sina Technology: "AI chip shortage drags down technology companies' revenue, NVIDIA H100 is said to have at least tripled shipments next year"

3, CSDN: "Burning hundreds of millions of dollars and consuming tens of thousands of NVIDIA GPUs, Microsoft reveals the past of supercomputers behind the construction of ChatGPT!" 》

4. Wall Street News: "Let Go of Pride!" How Microsoft Gambles on OpenAI"

5. Interface news: "Microsoft's self-developed AI chip "Athena" surfaces, wanting to break NVIDIA's monopoly on computing power"

6, Yuanchuan Research Institute: "A Crack in the Nvidia Empire"