Note: This article is from an expert friend Dr. Winnie Shao's original masterpiece, I hope you like it.
Amazon Cloud Technology's Graviton 3
1. Design and innovation of Graviton3
Not only the number of transistors, as shown in the figure below, from the first generation of Graviton in 2018 to this year's Graviton 3, it can be clearly seen that a significant leap in performance from generation to generation.
(Note:)
In fact, whether it is designed with Chiplet, or using DDR5, PCIe Gen5, 5nm process, advanced is advanced, but Ali Pingtou Brother released the Yitian 710 in October, and the subsequent Intel and AMD also have it, so it is a routine operation. However, the power consumption of the Graviton series is significantly lower than that of the several chips mentioned above. Previously, I hadn't figured out this parameter design until I saw the Amazon C7g's server cover. This is to put 3 Graviton3s in one chassis, becoming a high-density server with 3 nodes. In other words, if you put 3 64-core server nodes in a chassis, if converted to a regular 2-way CPU, it is equivalent to 96 cores per CPU. If Graviton2 is a conventional 2-way server design, and then consider the 100Mhz increase on the main frequency, then the graviton3 server is nearly 2 times the performance of Graviton 2. This is very much in line with the nature of Amazon Cloud Technology, they are pursuing the same total power consumption of the full server / the performance of the entire rack maximum improvement, from the chip design stage, from the server node - to the overall reasonable power consumption budget allocation of the rack, and can determine the design plan.
Finally, it must be added that another reason why Amazon Cloud Technology can design a 3-node server in this way is because they have Nitro cards that can support multi-service nodes, otherwise each node needs to be equipped with a separate network card, which will partially offset the advantages of high-density server design.
2, the CPU to do machine learning
In the design of Amazon Graviton 3, there is another point to draw a special focus. It supports bfloat16 and improves machine learning performance by a factor of 3. Intel's cooper lake, AMD's Zen4, and IBM Power10 also support bfloat16, and all of them improve the performance of machine learning by 3 times or even 20 times. The CPU camp has a rare concerted effort to enhance its artificial intelligence business capabilities.
3. The C7g instance synchronized with Graviton3 marks the maturity of the arm server ecosystem
In addition to the various innovations of Graviton3 on chips and servers, there is one thing that is different from the release of Graviton2 in 2019. There is also an instance of Amazon C7g released at the same time as Graviton3, although it is still in preview, but compared to the release of the instance after the release of Graviton 2 3 months later, the AmazonGraviton family's instance development pace is significantly faster. Compared with the peer only chips, there is no news release of the time when the instances are launched, Graviton3 not only has chips, servers, but also instance previews, and even customer endorsements, and product maturity has obvious advantages.
It echoes Graviton3's double floating-point performance and 3x machine learning performance. Amazon C7g is an instance targeted at high-performance workloads. Amazon Cloud Technology recommends applications including high-performance computing, batch processing, electronic design automation (EDA), media coding, scientific modeling, and CPU-based machine learning inference. This is not the same as Graviton1's support for web servers, containerized microservices, data/log processing, and other lightweight workloads that can run on smaller kernels.
In fact, the Amazon Graviton2 processor has achieved a huge leap forward in performance and functionality, becoming the leader of the ARM camp. They support Amazon EC2 general purpose (M6g, M6gd, T4g), compute-optimized (C6g, C6gd, C6gn), and memory-optimized (R6g, R6gd, X2gd) instances with a wide range of workload types. AmazonG5G, a graviton + Nvidia T4G combination, was released at the conference, along with two storage-optimized instances of Im4gn and Is4gen with the latest NitroSSD. Finally, it should be emphasized that SAPHANA, the benchmark product of the database, also supports the Graviton family, which is not only the success of the Graviton family, but also a big step in the ARM server ecosystem.
In fact, since Arm entered the server market, there are doubts about whether Arm can leverage the server ecology. From the three linuxes, to Java OpenJDK, gcc/LLVM, to Ngnix, MySQL, and Redis, arm has great support in the open source software camp. But the support of ISVs (Independent Software Providers) like SAP HANA is an important milestone for the Graviton family to expand the arm server ecosystem.
It is not only ecological maturity, but also rich in examples. The Graviton3 once again improves performance in real-world applications. In the F1 application test showcased by Amazon Cloud Technology, graviton 3 is 40% faster than Graviton2. For Nginx, Graviton3 has a performance improvement of more than 60%, and other applications, there are at least 25% performance improvements.
brief summary
Looking back at the $350 million acquisition of Annapurna Lab, it was a super reward. First the industry-leading Nitro series, followed by the Graviton arm CPU family, followed by the two-piece machine learning inferentia with inferentia and train-enabled Trainium. The three major chips in the data center, DPU, CPU, and AIxPU, are neatly arranged.
When talking about chip innovation in an interview, the CEO of Amazon Cloud Technology said that this is the ability to change the rules of the game. The Graviton family validates everything from agile chip designs with arm CPU IP, to joint chip and server designs, to rapid deployment of Internet speeds to the cloud. This is an era of innovation driving high-quality service.
Note: This article only represents the personal views of the author, and has nothing to do with any organization, if there are errors and deficiencies, please feel free to criticize and correct in the message. If you want to share your own technical dry goods on this public account, you are also welcome to contact me :)
Respect knowledge, please keep the full text when reprinting. Thank you for your reading and support!