The big trend behind Amazon's self-developed chips

Source: Content from the newstack, thanks.

Amazon Web Services expanded its chip presence last month, primarily with the launch of the giant cloud provider's third-generation Arm-based Graviton processor, which will power new cloud instances for compute-intensive workloads such as high-performance computing (HPC), scientific modeling, analytics, and CPU-based machine learning inference.

At AWS's re:Invent conference, the company unveiled the Graviton3 processors currently in preview and the EC2 C7g instances that will run on those processors. At the same time, AWS CEO Adam Selipsky also announced a new instance of Trn1 running on the company's Trainium chip a year ago, targeting machine learning training workloads, and touted the price/performance capabilities of Inf1 instances launched in 2019 and leveraging Inferentia chips for machine learning inference tasks.

The company even announced storage-optimized EC2 instances—Im4gn/Is4gen/I4i—based on its Nitro solid-state drives (SSDs) to improve storage performance for I/O-intensive workloads in the AWS Cloud.

AWS focuses on chips

The launch of the latest processors and EC2 instances is the latest proof of AWS's years of efforts to build its own processors to run in its cloud instances and its Outposts infrastructure, which is designed to provide AWS services and connectivity – deploying data centers at a time when enterprise adoption of a hybrid cloud model is growing rapidly.

It all happened five years after AWS acquired Israeli startup Annapurna Labs in 2016, making it the basis for its chip manufacturing efforts.

Patrick Moorhead, principal analyst at Moor Insights and Strategy, told The New Stack: "AWS has been investing years in its own chips, starting with Nitro and expanding to general graviton, Inferentia for inference, and now Trainium for training. "AWS can pick and choose every feature it wants, and every feature it doesn't need to take advantage of its own software." It can also optimize its I/O for its specific network and storage. At scale, this should allow it to deliver computing at a lower cost and, in some cases, higher performance. ”

Intel, AMD and Nvidia serve a broader market, spanning multiple environments, and some customers aren't using all of the features, Moorhead said. AWS is using native compute to differentiate its instances.

Price/performance ratio is key

In his keynote, Selipsky emphasized that enterprises will see price/performance advantages in running workloads such as ARTIFICIAL intelligence, machine learning, and analytics on instances leveraging AWS chips instead of x86 CPUs from Intel and AMD or GPUs from those vendors and Nvidia.

"With Trainium and Inferentia, customers get the best price/performance for machine learning, from scaling training workloads to accelerating deep learning workloads in production with high-performance inference, giving all customers access to the full power of machine learning," the CEO said. "This has long been our goal, and reducing the cost of training and reasoning is a major step in this journey."

AWS didn't reveal many details about Graviton3. He said that with the new silicon case at 25 percent, it runs faster than graviton2 in general-purpose computing workloads as power cases, and will be better off for some specialized workloads. For example, it is twice as high as the floating-point performance of scientific workloads and cryptographic jobs. Running machine learning applications is also three times faster.

A factor in power efficiency

Graviton3 will use up to 60% of the energy at the same performance, in part due to the use of DDR5 memory, which consumes less power than DDR4 while providing 50% of the bandwidth. The processor will run up to 64 cores, have 50 billion transistors, and have a clock speed of 2.6GHz.

Jeff Barr, vice president of AWS, wrote in a blog post that Graviton3-based C7g cloud instances will be available in a variety of sizes, including bare metal models.

Instances based on Inferentia and Trainium are also designed to reduce the cost of running specific workloads. Selipsky says inference cost per Inf1 instance is 70% lower than similar GPU-based EC instances. Meanwhile, trainium-powered Trn1 instances will provide twice the bandwidth of GPU-based instances for work such as natural language processing and image recognition — up to 800 Gb/s of EFA network throughput.

Enterprises will also be able to deploy Trn1 instances in EC2 UltraClusters, which can scale to tens of thousands of Trainium chips and reach petabytes. These UltraClusters will be 2.5 times larger than the previous EC2 UltraClusters.

"Both Inferentia and Trainium are designed to save money in production-level reasoning and core training," Moorhead said. "AWS has always stood by its stance on saving money, so before I see the Trainium results, I'm very confident that on certain workloads, you'll see significant savings."

The trend towards custom chips

Graviton, Inferentia, and Trainium are part of a broader trend towards dedicated processors in the industry. In a blog post this week, Chris Bergey, senior vice president and general manager of Arm's infrastructure business line, wrote that his company designs chips and licenses those designs to other companies, driving this design trend with its energy efficiency.

"Data center workloads and internet traffic double almost every two years, so performance advantages per watt are critical to preventing computing from increasing its carbon footprint," Bergey wrote, adding that Arm's growth in the cloud "gives developers the option to continue to innovate sustainably by delivering consistent performance and scalability on a per-core basis, enabling a scalable combination of performance and efficiency to deliver industry-leading TCO." ”

AWS isn't the only hyperscale enterprise looking to design its own chips as they seek greater performance and efficiency. Microsoft reportedly decided last year to build Arm-based chips for Azure servers and Google, which has custom chips such as tensor processing units and OpenTitan security chips. Facebook is also building its own data center chips.

The challenge of building your own processor

Rob Enderle, principal analyst at The Enderle Group, told The New Stack that he wasn't sure how that would evolve.

"When companies reach a certain size, they tend to believe that their internal economies of scale will enable them to compete effectively with focused suppliers as peers," Enderle said. "This latest trend is largely the result of Intel missing many key milestones... Forcing most people in the cloud industry to consider this path. ”

However, under the leadership of CEO Pat Gelsinger, Intel's execution is improving. At the same time, AMD's Epyc CPUs and GPUs continue to impress, he said, suggesting that the need for custom chips may be decreasing.

"In times of supply shortages, companies like AMD and Intel may also be more likely to work than act alone, because these companies should not only have better supply redundancy, but should also be better able to shift the blame from internal decision makers if the shortage is even beyond their control," Enderle said. "Cost is still a potential advantage for going it alone, but only if you overlook the value of companies' intellectual property protection and decades of experience that often provides mutually offset reliability, consistency, and performance benefits."

In addition, costs increase over time and internal efforts can become unprofitable and unsustainable. Analysts say part of the reason is that it's difficult to find and retain the talent needed, a particular challenge in times of severe shortage of skilled staff.

"While the past doesn't always predict the future, and COMPANIES THE SIZE OF AWS CAN SUCCESSFULLY DO THINGS THAT EVEN THE LARGEST ENTERPRISES CAN'T," Enderle said, further noting that "as long as the fundamental strengths of specialized companies are executing, they remain valid." ”

★ Click [Read the original] at the end of the article to view the link to the original article!

*Disclaimer: This article is original by the author. The content of the article is the author's personal opinion, semiconductor industry observation reprint is only to convey a different point of view, does not mean that semiconductor industry observation endorses or supports the view, if there is any objection, welcome to contact semiconductor industry observation.

Today is the 2905th content shared by Semiconductor Industry Watch for you, welcome to follow.

Wafer | integrated circuit | devices | automotive chips | storage | TSMC | AI | packages

Link to the original article!