Selected from IEEE
Machine Heart Compilation
Editor: Du Wei
When the single-chip processor has reached its limit, the chips released by Apple and Nvidia have proved that multi-chip packaging may be the future development direction, but interconnection technology is still a major problem and the main battlefield of giant competition.
On March 10, Apple unveiled an upgraded version of the M1 Max chip, the M1 Ultra, at its spring 2022 conference, innovatively using the package architecture UltraFusion to connect the dies of the two M1 Max chips to create a system-on-chip (SoC) with unprecedented performance and functionality.
On March 23, Nvidia released a similar news release at GTC 2022. Jen-Hsun Huang announced the launch of the first dedicated data center CPU for AI infrastructure and high-performance computing, in which the new Grace Hopper can be connected in parallel with two pieces on the same motherboard to form a 144-core Grace CPU super chip with a memory bandwidth of 1TB/s.
The chips of the two companies have different target markets. Apple is targeting the consumer and professional workstation markets, while Nvidia is trying to make a splash in the high-performance computing market. However, the difference in goals only highlights the broad challenges of a rapid end to the era of single-chip design.
Image source: top10.digital
Chip giants have entered the market
Multi-chip design is not a new concept, but it has only gained popularity in the last five years. Chip giants such as AMD, Apple, Intel and Nvidia are all involved in it to varying degrees.
AMD explores chiplet design with its own EPYC and RYZEN processors. Intel unveiled the next generation of Intel Xeon Scalable processor Sapphire Rapids at the Architecture Day 2021 event, an architecture for the server market built using small chip "tiles".
Now, Apple and Nvidia have also joined the ranks of multi-chip designs, albeit for very different target markets. It should be noted that the shift to multi-chip designs is driven by challenges in modern chip manufacturing. The pace of transistor miniaturization has slowed down, but the growth in the number of transistors in cutting-edge designs shows no signs of slowing down.
Take Apple's M1 Ultra chip, for example, with 114 billion transistors, the largest ever made in a personal computer chip, seven times the size of M1. The chip area of a single M1 Max is 432 square millimeters, which infers that the M1 Ultra has an area of about 860 square millimeters (official figure unknown).
M1 Ultra schematic.
The number of transistors in NVIDIA Grace CPUs is kept secret, but the Hopper H100 GPU released with it has 80 billion transistors and 20 can carry global traffic. In 2019, AMD released a 64-core EYPC Rome processor with 39.5 billion transistors.
Nvidia Grace CPU super chip.
Akshara Bassi, a research analyst at counterpoint Research, a market research firm, said, "As chip areas become larger and wafer yield issues become more important, multi-chip modular package designs can achieve better power consumption and performance performance than single-chip designs."
For now, with the exception of Cerebras, an AI chip startup dedicated to creating a single complete silicon wafer, the chip industry seems to have reached an agreement that single-chip designs are becoming increasingly "overblown."
In April 2021, Cerebras released the WSE 2, a jumbo chip with 2.6 trillion transistors and larger than the iPad.
In addition, the industry's shift to small chips is carried out in tandem with the support of chip manufacturers. In August 2020, TSMC, the world's largest chip foundry, unveiled its 3DFabric Advanced Packaging Technology Family, which includes front-end 3D silicon stacks and back-end packaging technologies.
AMD uses 3DFabric's technology in its EPYC and RYZEN processor designs, and it is almost certain that apple's M1 Ultra chip also uses TSMC-related packaging technology (although Apple has not confirmed it, the M1 Ultra is made by TSMC).
Other chip giants, such as Intel, have their own packaging technologies, such as EMIB and Foveros. Although originally intended for its own use, the company's chip manufacturing technology is becoming relevant to the wider industry as Intel foundry services unfold.
What are the prospects for multi-chip design?
Mark Nossokoff, a senior analyst at Hyperion Research, another market research firm, believes that "the ecosystem around the design, fabrication, and packaging of basic semiconductors has evolved to the point where it can support the 'economically and reliably generation of small chip solutions at design nodes.'" Software design tools that seamlessly integrate diverse chip functions are also mature enough to optimize the performance of the target solution."
Small chips will continue to exist, but for now, the field is an island. AMD, Apple, Intel, and Nvidia are applying their self-developed interconnect designs to specific packaging technologies.
On March 2 this year, Intel, AMD, Arm, Qualcomm, TSMC, Samsung, Sun Moonlight, Google Cloud, Meta, Microsoft and other ten giants announced the establishment of the Chiplet Standards Alliance, launched the Universal Chiplet Interconnection Express (UCIe), hoping to bring the industry together. The standard offers a "standard" 2D package for cost-effective performance and a "premium" package for cutting-edge design.
UCIe also supports off-package connectivity via PCIe and CXL, making it possible to connect multiple chips across multiple machines in high-performance computing environments.
Example of a UCIe encapsulation scheme in the UCIe white paper.
The UCIe standard is a beginning, and its future remains to be seen. Nossokoff said that the founding members who originally launched UCIe represented many outstanding contributors to technology design and manufacturing, but many major organizations did not join, including Apple, AWS, Broadcom, IBM, Nvidia and other silicon foundries and memory chip suppliers.
Bassi noted that Nvidia may be particularly reluctant to join the UCIe alliance. NVIDIA has opened up its self-developed NVLink-C2C interconnect technology for custom silicon integration, making it a potential competitor to UCIe.
While the fate of chip interconnect technologies such as UCIe and NVLink-C2C determines the rules of the industry's game, they are unlikely to change the industry's current situation.