Since the advent of ChatGPT and the re-explosion of artificial intelligence (AI), many people have a question: I hope that artificial intelligence can help me sweep the floor and wash dishes, so that I can be in the mood to write poetry and paint. But why are the current artificial intelligence writing poems and paintings, and we humans are still sweeping the floor and washing dishes?
Image taken from social media
The situation complained by netizens is related to the development path of artificial intelligence technology, and it is also related to the commercialization process of technology, and it is not "deliberate" by researchers. At the same time, countless AI developers are working hard to make the technology help us achieve some more practical goals.
Recently, a large AI model published by the HUAWEI CLOUD team in the journal Nature attempts to solve a very real problem that plagues all mankind: weather forecasting.
Copyright pictures in the gallery, reprinting and using may cause copyright disputes
One of the biggest difficulties in weather forecasting is the chaotic nature of the atmospheric system that determines how weather occurs.
Although significant progress has been made in current NWP methods, there are still some limitations. First, NWP models require significant computational resources to solve complex physical equations. This can be a "fatal" problem for large-scale global weather forecasting. Second, NWP models often require detailed initial conditions, which are often provided by satellite and ground-based measurements, but data may have uncertainty or error. This affects the accuracy of the forecast.
So while we have been able to make fairly accurate short-term weather forecasts (such as forecasts for hours or days in the future), the accuracy of medium- and long-term forecasts (such as forecasts for the coming weeks or months) remains a challenge. Today, the emergence of AI large models may allow us to take a step closer to more accurate medium- and long-term forecasting.
In the official issue of Nature on July 5, 2023, the HUAWEI CLOUD team reported on a medium-term weather forecast based on HUAWEI CLOUD's Pangu-Weather model. This work solves for the first time the worldwide problem that AI forecasts weather with less accuracy than traditional numerical forecasting, and the prediction speed is 10,000 times faster, achieving "second-level" global weather prediction.
The core of HUAWEI's cloud Pangea Meteorological Model is a 3D Earth-Specific Transformer that can capture complex patterns in weather data and is trained using about 40 years of global weather data. At the same time, the team adopted a hierarchical time-domain aggregation strategy to reduce cumulative errors in medium-term forecasts. As a result, the Pangea Meteorological Model in some cases surpasses the world's best NWP systems in terms of accuracy and speed.
The HUAWEI CLOUD R&D team found that the accuracy of the previous AI weather prediction model was insufficient for two main reasons: first, the original AI weather prediction model was based on 2D neural networks, which could not handle uneven 3D weather data well. Second, AI methods lack mathematical and physical constraints, so iteration errors will continue to accumulate in the process of iteration.
A key breakthrough in HUAWEI CLOUD's Pangu weather model is the understanding of weather patterns on Earth. By integrating altitude information into new dimensions, the system can understand weather patterns in three dimensions, allowing for more accurate weather predictions. In addition, the hierarchical time-domain aggregation strategy is an important technological breakthrough, which greatly reduces the number of iterations required for medium-term weather forecasting, thereby reducing the cumulative error.
Copyright pictures in the gallery, reprinting and using may cause copyright disputes
The industry spoke highly of Huawei's research. Relevant experts generally believe that the emergence of Pangea meteorological large model represents a major breakthrough in the field of artificial intelligence in the field of weather forecasting. The National Meteorological Center of China and the European Center for Medium-Range Weather Forecasting (ECMWF) have confirmed the superiority of Pangu Grand Model prediction in actual measurements, and its results include temperature, humidity, wind speed, sea level pressure, etc., which can be directly applied to multiple meteorological research subdivision scenarios.
However, although the Pangea model opens up new forecasting avenues in a sense, it still relies on NWP for training, and the results do not crush NWP. Therefore, from an objective point of view, peer experts have also put forward some deficiencies and improvement directions for the Pangu meteorological model, which are still the research team needs to further study and verify.
In May 2023, Typhoon Mawa attracted the world's attention as the strongest tropical cyclone so far this year. The National Meteorological Center uses artificial intelligence to rapidly enhance recognition technology to achieve trend forecasting 12 hours in advance. China Meteorological News said in a related report that HUAWEI CLOUD Pangu model performed well in predicting the Mawa trajectory, predicting its turn trajectory in the eastern waters of Taiwan Island 5 days in advance. At the 19th World Meteorological Congress, the European Center for Medium-Range Forecasting also pointed out that HUAWEI CLOUD's Pangea Meteorological Model has undeniable ability in accuracy, and the pure data-driven AI weather forecast model shows forecasting strength comparable to numerical models.
The HUAWEI CLOUD R&D team also proposed an adaptive learning strategy that enables the model to make real-time adjustments when making predictions on new data. The successful implementation of this technology will further improve the prediction accuracy of Pangea Meteorological Large Model, making it more valuable in practical applications.
In agriculture, aviation, energy, disaster warning and other fields, accurate weather forecasts have great social and economic value. For example, in agricultural applications, accurate precipitation forecasting will help farmers rationalize agricultural activities and improve agricultural production efficiency. In the field of air transportation, accurate wind speed forecasting will help airlines to rationalize routes and reduce operating costs.
Over the past few decades, meteorological scientists have struggled to improve the accuracy of weather forecasts, but there are still many challenges in the field. In the future, AI technology has the potential to be key to addressing these challenges.
exegesis
[1] https://www.nature.com/articles/s41586-023-06185-3
[2] The abstract describes 39 years. Huawei's official website news report is 43 years.
[3] https://www.cma.gov.cn/en2014/news/News/202306/t20230607_5560758.html
[4] https://www.ecmwf.int/en/about/media-centre/science-blog/2023/rise-machine-learning-weather-forecasting
Planning production
The author is not a science journalist
Review丨Yu Yang Head of Tencent Xuanwu Lab