At the Huawei Developer Conference 2023, Huawei not only demonstrated the powerful capabilities of Pangu Model 3.0, but also unveiled a series of remarkable results. The excitement of this press conference is dizzying. However, the most striking focus is undoubtedly the revolutionary breakthrough of Pangu Model 3.0 in the field of weather prediction.
The revolutionary nature of the Pangu Grand Model lies in its application in weather forecasting. Previous weather forecasting has relied mainly on models based on 2D neural networks, but the complexity of meteorological systems limits the effectiveness of this approach. To make matters worse, previous AI models accumulated errors in the prediction process, which affected the accuracy of the results, so they have not been widely used. However, the Pangu Grand Model revolutionized this by using 3DEST's three-dimensional neural network to process meteorological data.
The 3DEST network training and inference strategy adopts a hierarchical time-domain aggregation strategy, which fundamentally reduces the iterative error and improves the accuracy of weather prediction. For example, traditional AI weather prediction models usually predict the arrival of a typhoon 6 hours in advance, and then calculate the arrival time of the typhoon several times within these 6 hours. This method may lead to different calculation results, error accumulation, and affect the accuracy of predictions. The Pangu model trains four models with different forecast intervals, which are 1 hour, 3 hours, 6 hours and 24 hours, and selects appropriate models for iteration according to needs. This strategy effectively reduces the error and takes weather forecasting to the next level.
The Pangu model is also so great thanks to its unique architecture. Huawei Pangu Model 3.0 adopts a three-layer architecture of 5+N+X, enabling it to be quickly applied to various industries. This architecture cleverly solves the data acquisition problem faced by AI landing in the industry. First of all, the first layer of Pangu L0 contains 5 basic large models, which learn massive encyclopedic knowledge, literary works, program code and other text data, as well as billions of Internet images with text labels, establishing a basic understanding for the model. Then, the model in the second layer L1 allows one of the basic large models in L0 to learn data from N related industries, similar to the undergraduate level of the university, and need to choose different majors for study. The final L2 is further refined to a specific scenario, similar to the graduate level, and the model is customized according to the needs of different industries.
Huawei also added a feedback session, according to them, in the past, it took 5 months to develop a GPT-3 scale industry model, but with this architecture, the development cycle can be shortened to 1/5 of the original. This also allows the smaller limitations of many industry datasets to be solved, bringing more possibilities to all walks of life.
Not only that, Huawei also proposed the concept of localization of computing power, which solved the shortcomings of AI computing power. Their Ascend 910 processor has surpassed the NVIDIA A100 in performance, and although there is still a gap in practical applications, this move shows Huawei's determination in the field of AI. At the same time, Huawei also provides a full set of application packages to enable users to train large models more efficiently.
On the whole, Huawei's layout in the field of AI is profound and impressive. They not only focus on the basic research of AI, but also actively explore how to apply AI to different industries. Huawei's Pangu Model 3.0 and the localization of computing power have brought new vitality to the AI industry and made people see the great potential of the AI field in the future. As Huawei founder Ren Zhengfei said, the real era in the field of AI is still to come, and we have reason to expect more innovations and breakthroughs.