"Besieged" Nvidia AI chips

Since the explosion of ChatGPT, Nvidia has been recognized as the biggest "shovel seller" in this global AI gold rush wave, and it is also the most discussed AI chip company in major media and social platforms. However, as the AI boom continues to heat up, more and more manufacturers have also begun to make efforts in the field of AI chips: before Intel, AMD and other semiconductor giants announced a new round of AI chip research and development plans, and then OpenAI, Microsoft and other downstream customers promoted self-developed chips to break NVIDIA's monopoly.

Self-developed by Microsoft

On November 15, local time, Microsoft launched two custom chips at the Ignite developer conference held in Seattle to cope with the increasing cost of large model training and try to reduce the cost of providing AI services. Microsoft said the newly released chips will not be sold and will only be used to support its own products and as part of Microsoft's cloud Azure cloud computing services.

Microsoft's two latest chips are the Maia chip, which is used to accelerate AI computing tasks, and the Cobalt chip, which uses the Arm CPU architecture. The Maia chip is designed to run large language models. Scott Guthrie, executive vice president of Microsoft's cloud and artificial intelligence division, said that it hopes to optimize this AI chip to provide a faster, lower-cost, and higher-quality solution. It is reported that both chips will be available in 2024.

Microsoft did not disclose the technical details of how much these chips have improved their capabilities compared to traditional chips. But Rani Borkar, Microsoft's vice president of hardware systems and infrastructure for Azure, said that both chips are made using TSMC's 5nm manufacturing technology. In addition, the Maia chip is built differently from the network connectivity technology used by NVIDIA, where the Maia chip is connected to a standard Ethernet cable.

In fact, there have been early signs that Microsoft is pushing custom chips. In 2010, Microsoft wanted to develop its own AI hardware. According to The information, Microsoft has been developing a new AI chipset codenamed "Athena" since at least 2019, with the aim of providing an alternative to Nvidia's chips for the training and inference of large language models such as ChatGPT. According to Tom's Hardware, Athena uses TSMC's 5nm process, which is designed for large language model training.

According to a person familiar with the matter, during the development of Athena, Microsoft has ordered at least hundreds of thousands of GPUs from Nvidia in order to meet the needs of OpenAI. In September, as the ChatGPT craze flattened, there was constant market news that Microsoft began to reduce the order volume of Nvidia's H100 graphics card. In Microsoft's earnings call in October, the issue of "cutting costs" was also repeatedly emphasized.

However, Microsoft's launch of self-developed AI chips is not intended to replace manufacturers such as Nvidia. Microsoft said it will provide cloud services to Azure customers next year, which run on the latest flagship chips from Nvidia and AMD. Microsoft is testing OpenAI's state-of-the-art model, GPT-4, on AMD's latest flagship chip.

One after another

In the context of the shortage of chips and the surge in demand for AI applications, Microsoft's rapid advancement of the custom chip plan is widely interpreted by the outside world as the only choice for cloud computing giants with resources. The generative AI market is expected to surge from $40 billion in 2022 to $1.3 trillion over the next decade, according to a Bloomberg report, with the influx of services such as OpenAI's ChatGPT.

Prior to this, tech giants such as Amazon and Google had also used self-developed chips and partially offered them to customers. AMD, another major chip manufacturer, also launched the latest accelerator card Instinct MI300X not long ago. At the press conference, a line of words was specially typed on the PPT - special for large language models, which was regarded by the industry as a direct declaration of war against NVIDIA.

It is reported that the MI300X's high-bandwidth memory (HBM) density can be up to 2.4 times that of the NVIDIA H100, and the high-bandwidth memory bandwidth can be up to 1.6 times that of the H100.

On the heels of Microsoft, South Korean wireless carrier SK Telecom hosted SK Tech Summit 2023 in Seoul on Thursday. At the summit, Sapeon Inc., a semiconductor startup under SK Group, announced that the company unveiled its latest AI chip, the X330.

According to the data, Sapeon is headquartered in Santa Clara, California, USA. According to reports, South Korean giants SK Telecom, SK hynix, SK Square, etc. are shareholders of the company. In 2020, SK Telecom developed the Sapeon X220 for the first time, which is the first chip in Korea for data centers needed to provide high-speed, low-power AI services. Two years later, SK Telecom spun off its Sapeon business into a separate entity headquartered in California to accelerate the commercialization of AI chips.

In a statement, the company said that the X330 chip has four times the computing performance and more than two times the energy efficiency of the previous generation X220. Sapeon plans to test the chip for key customers first and start mass production of the chip in the first half of 2024.

The status is still there

For the industry, more and more supply is naturally a good thing, as the shortage of AI chips has been going on for some time. Many industry insiders expect the shortage of AI chips to continue until at least next year. "At present, the visibility of Nvidia's orders has reached 2024, and high-end chips are in short supply. With the current production schedule, even the A800/H800 will not be delivered until the end of this year or next year. In the short term, judging from its popularity, the only thing that will affect Nvidia's high-end GPU sales may be TSMC's production capacity. A chip practitioner told a reporter from Beijing Business Daily.

Some founders and investors said that even if the AI chip is in place, they still have to wait weeks before they can use it. The CEO of an AI startup said, "Even if you've paid upfront, it doesn't mean that the GPU will be delivered to you the next day or the next week, you can only wait." ”

"Nvidia won't have a monopoly in the large-scale training and inference chip market forever. This was Tesla CEO Elon Musk's response to a tweet from Adam D'Angelo, CEO of Quora, a social Q&A site and online knowledge marketplace, who wrote: "One reason the AI boom is underestimated is the GPU/TPU shortage, which has led to various limitations on product launches and model training, but none of these are obvious." Instead, what we're seeing is a surge in Nvidia's share price. Once supply meets demand, things will accelerate. ”

However, it seems that it is not so easy to get rid of dependence on Nvidia. In addition to the real level of custom chips that need to be further investigated, other series of updates released by Microsoft at the conference are still indispensable to NVIDIA. Microsoft, for example, announced that its Azure will provide Nvidia's new generative AI modeler to customer enterprises. In the view of Bu Rixin, a partner at Chuangdao Consulting, this means that the partnership between the two will be expanded, and it also shows that Microsoft does not want to completely destroy the tacit understanding with NVIDIA.

Someone once said frankly that the gap between Nvidia and other chip manufacturers is the difference between academicians and high school students. As Nvidia CEO Jensen Huang said, Nvidia "has been running", and other chip manufacturers who want to surpass the giant can only run wildly. According to Khaveen Investments, Nvidia's data center GPU market share in 2022 is as high as 88%, and AMD and Intel will share the rest.

An Guangyong, an expert of the Credit Management Committee of the China Mergers and Acquisitions Association, also pointed out that the development of these companies is different, but in general, their market share has not been able to shake Nvidia's dominant position. In terms of pain points, there are challenges in chip manufacturing costs, chip energy efficiency ratio, product performance and stability, etc.

Beijing Business Daily reporter Fang Binnan Zhao Tianshu/text

One Map Network/Figure