Since the mid-20th century, numerical weather prediction models based on mathematical physics equations have dominated the field of weather forecasting. These models enable computer forecasts of future weather by simulating motion and state changes in the atmosphere. Numerical prediction has greatly improved the accuracy of forecasting and is regarded as a milestone in the development of human science and technology.
In the 21st century, the development of numerical prediction models is facing bottlenecks. Climate change has made atmospheric movements more complex and variable, and traditional models have slowed to improve the accuracy of forecasts. At the same time, artificial intelligence technologies such as deep learning have developed rapidly and are widely used in various fields. Meteorologists are also experimenting with using machine learning for weather prediction and breakthroughs.
In 2021, NVIDIA's FourCastNet used deep learning in weather forecasting for the first time to achieve a faster forecasting speed. In March 2022, Huawei released a Pangea Meteorological Model based on a 3D convolutional network. The experimental results show that the prediction accuracy of the Pangu model from 1 hour to 7 days exceeds the traditional numerical prediction system of the European Medium-Range Forecasting Center for the first time. At the same time, the prediction speed is tens of thousands of times faster than that of numerical models. The Huawei team made model adjustments and open source, and published papers in Nature.
The article immediately aroused strong repercussions in the industry, and industry experts generally gave positive comments, believing that Pangu opened a new forecasting paradigm. The verification of the European ECMWF confirmed the predictive advantages of the Pangu model, but some industry experts pointed out that the AI model has problems such as relying on reanalysis data, insufficient explanatory properties, and judging extreme weather deviations, including:
- AI models rely on specific datasets for training and cannot be applied in real time. Numerical models are based on mathematical physics equations and are not subject to this limitation.
- AI models lack physical constraints and explanatory properties, and have limited room for improvement. Numerical models, on the other hand, are based on theory.
- AI models are inferior to numerical models in determining extreme weather and typhoon paths.
- AI models lack technical means such as data assimilation and ensemble forecasting, and have limited application to new observation data.
- Single data-driven models are more susceptible to data constraints and uncertainties in the face of climate change. Numerical models have strong theoretical forecasting ability.
In general, although there is still controversy, the AI large model represented by the Pangu meteorological model has created a new era of deep learning technology application in the field of weather prediction. It brings a new approach to forecasting ideas and paradigms for the industry. This model of integrating artificial intelligence and physical models is expected to eventually become the new mainstream of weather forecasting by breaking through data and physical constraints.
Can the new method of predicting weather through artificial intelligence AI large model, represented by Huawei's Pangu Meteorological Model, replace the traditional numerical prediction model based on mathematical and physical equations that the current Meteorological Administration relies on? I believe that this is a question in the minds of many small partners, in fact, as early as October last year, industry experts issued an article to give the answer.
In October 2022, the journal Earth Sciences, hosted by China University of Geosciences, published an article published by Professor Li Shuanglin, doctoral supervisor of the Institute of Atmospheric Physics of the Chinese Academy of Sciences, who has long been engaged in climate dynamics and climate change research, "The Future of Numerical Weather Prediction Is the Integration of Artificial Intelligence and Mathematical Physics Model?" The title seems to have given the answer, but a big question mark does not seem to be so convinced. Small partners with academic skills can go to CNKI to download this article to study it, the little sea ape does not make a long talk today, only pick up the core views, Professor Li Shuanglin believes:
- Traditional numerical weather prediction models have encountered bottlenecks in the current development, and the speed of improving forecast accuracy has slowed down, which cannot meet the demand for higher precision forecasting. In recent years, the rapid development of artificial intelligence technology provides an opportunity to solve the problems faced by numerical weather prediction.
- Deep learning can bring progress to numerical prediction in many aspects, such as the determination of initial model values, the expression of physical processes, and the correction of results. The numerical prediction model enhanced by artificial intelligence technology has obvious advantages in extreme weather forecasting. AI-integrated forecasting models can reduce reliance on climate context and observational data.
- Artificial intelligence technology has the advantage of discovering potential relationships in data, and mathematical physical modeling has the ability to explain theory, and the two can complement each other. The deep integration of mathematical physical models and artificial intelligence technology is a new direction for the development of numerical weather prediction. The integration of mathematical physics and artificial intelligence can give play to their respective advantages and promote the improvement of numerical prediction.
The "Artificial Intelligence Meteorological Application Work Plan (2023-2030)" issued by the China Meteorological Administration on July 20, 2023 seems to give the same answer, pointing out that artificial intelligence has shown certain advantages in the field of weak prediction capabilities of traditional numerical prediction models, and through further production practice, it will certainly be able to explore a path of coordinated development and complementary advantages between AI and traditional numerical prediction models.
The latter series of cooperation seems to further confirm the attitude of the meteorological system, that is, AI models and traditional models have their own advantages, the advantages of the two sides complement each other, and there is no job grabbing. The ending seems to be developing in the direction of hello, hello everyone, complementary advantages, and synergistic development.
However, from some clues, it can still be seen that the truth does not seem to be completely overdue. In December 2022, Huawei's Pangu Meteorological Model was invited to the China Meteorological Administration to carry out academic exchange activities, and the official registration of the Meteorological Bureau only used the phrase "After the report, the participants had a heated discussion" to describe the scene, but some details revealed in the "White Paper on Accelerating Industry Intelligence" released by Huawei on September 20 this year made it clear that there was a hint of "gunpowder" (see another article for details> From "dissatisfied" to "real fragrance", Huawei Pangu Meteorological Large Model is connected to the business system of the Meteorological Bureau).
After this exchange, the two sides agreed to carry out business verification, to put it bluntly, it is a competition, the Central Meteorological Bureau chose the traditional strength of typhoon path forecasting, the specific result white paper did not expand, only used a sentence: "AI model on a single server efficiently generates forecast results comparable to traditional numerical models, and with meteorological business system to achieve docking, provide forecasters with reference, and use in consultations" description.
In the white paper, Huawei showed off a wave of muscles and self-evaluation of the Pangu Meteorological Model:
- Low resource consumption and high computing efficiency: Compared with the slow calculation speed (5-6 hours), large resource consumption (tens of thousands of CPU cores), and difficulty in improving forecast accuracy encountered in the current traditional numerical weather forecasting, the AI weather forecasting scheme provides strong support for business personnel, and the statistical perspective analysis of forecast accuracy is higher than that of traditional numerical methods (operational IFS of the European Meteorological Center), and the prediction speed is increased by 10,000 times, which can provide global weather forecasting in seconds. The computing resources have dropped from tens of thousands of CPU core computing hours to tens of seconds of computing on a single AI card.
- Ensemble forecasting ability: Due to the exponential improvement of reasoning efficiency, it is possible to carry out collective forecasting of thousands of ensemble members, and large-scale ensemble members can cover possible weather evolution more completely, gain insight into various possible extreme weather conditions in advance, and provide effective early warning information for the public, which has great social value.
- Continuous optimization: With the continuous accumulation of high-quality reanalysis data, the AI model is periodically iteratively optimized by the training platform to continuously improve the forecast accuracy of the model.
Are the Pangea meteorological models and traditional numerical prediction models complementary or substituted for each other? Or are they complementary and inevitably replace and compete in some areas? The answer may only be known over time.