NVIDIA today also released a new generation of Hopper H100 GPU cores, Grace CPU processors, and also put the two on a card, named "Grace Hopper", officially called the highest density to accelerate the computing system.
The two chips are placed side by side and communicate with each other via the NVLink bus, with a bandwidth of 900GB/s.
The specific specifications have not yet been announced, but it should not be full of blood at the same time, especially the Hopper GPU.
Grace CPU is based on ARMv9 architecture instruction set design, 72 cores, integrated 198MB cache, partner LPDDR5X memory, support ECC.
Hopper GH100 GPU18432 CUDA cores, 576 Tensor cores, 60MB L2 cache, 6144-bit HBM2e/HBM3 memory width.
The H100 compute card only has 16896 CUDA cores, 528 Tensor cores, 50MB L2 cache, and 5120-bit HBM3 memory bandwidth, which is also the specification on the Grace Hopper card.
What is the use of this ensemble "monster"? Obviously, HPC and AI computing are also done, and the CPU and GPU can coordinate the distribution of different loads to maximize efficiency.
At the same time, NVIDIA also released the "Grace CPU SuperChip", which integrates two Grace CPUs on a single card.
NVLink C2C interconnect, total 144 cores, 396MB of cache, 1TB/s LPDDR5X ECC memory, power consumption of 500W.
There is also the "H100 CNX", which also integrates H100 GPU and CX-7 SmartNIC network chip.
By eliminating system bandwidth by allowing a GPU direct connection network, PCIe 5.0 is supported, and up to 7 multi-GPU instances are supported concurrently.