Meta has released details of its next-generation AI chip, MTIA (Meta Training and Inference Accelerator). The chip was designed in-house by Meta with the company's unique AI workloads in mind, specifically the deep learning recommendation models used to improve the experience of many of its products
As a series of dedicated chips independently developed by Meta, MTIA is an important investment in the company's AI infrastructure construction. The new generation of MTIA has achieved a number of major breakthroughs on the basis of the previous generation: the computing performance is increased by 3.5 times, and the sparse computing performance is improved by 7 times; The SRAM capacity is doubled and the memory bandwidth is increased by 3.5 times. The chip frequency is increased from 800MHz to 1.35GHz, and the power consumption is increased from 25W to 90W. This allows MTIA to serve Meta's sorting and recommendation models more efficiently and provide users with a better experience.
In addition to hardware upgrades, Meta has also worked the optimization of the software stack. The next-generation MTIA is fully compatible with its predecessor and seamlessly integrates with PyTorch 2.0 and its children, TorchDynamo and TorchInductor, without the need for developers to re-adapt. In addition, Meta has also independently developed the Triton-MTIA compiler backend, which can generate high-performance computing cores for MTIA hardware, greatly improving development efficiency
Through deep collaborative optimization of hardware and software, Meta's latest generation MTIA system has been greatly improved in terms of performance and energy efficiency: compared with the previous generation, the performance of a single chip has been increased by 3 times, the throughput of model services at the entire platform level has been increased by 6 times, and the power efficiency has been improved by 1.5 times. This has laid a solid technical foundation for Meta's future development in generative AI, recommender systems and other fields
Meta said the MTIA is just a major milestone for the company in the field of customized AI hardware. In the future, MTIA will continue to expand its support and work with hardware such as commercial GPUs to build a more robust and efficient AI infrastructure to meet the increasingly complex computing needs of the metaverse era. This also reflects Meta's long-term planning and determination in the construction of AI infrastructure
epilogue
The design, manufacturing process and subsequent deployment of MTIA are all under the control and guidance of Meta, reflecting Meta's control over its core technologies. Therefore, it can be said that MTIA is Meta's autonomous controllable chip
MTIA and NVIDIA GPU chips differ in terms of design philosophy, workload adaptability, and software ecosystem, with the former focusing on meeting the needs of Meta's specific AI tasks with a high level of customization and optimization, while the latter is characterized by versatility and wide application
It's not that Nvidia can't afford it, but that MTIA is more cost-effective?