AI plays romance, and it is the weather station that is injured?

If the forecast says that 25% of the sky will be covered by clouds, can you tell the weather tomorrow? I'm afraid it's hard. Maybe those 25% of the clouds will come together to bring a downpour, or maybe they're just lots of lovely scattered clouds in the sky that add to your joy on a sunny day. To predict weather from clouds, we need to know more.

Clouds greatly influence the weather

Surprisingly, the most advanced weather models can only give a very rough description of clouds, such as the 25% cloud cover we just mentioned. The reason is that clouds tend to be small, and weather models cannot take into account every small area of the sky. And if you take every area of the sky into account, even the fastest supercomputers can't do the calculations needed for weather forecasting. Even the dramatic increase in computing power in recent years is not enough to solve this problem.

"There's a big difference between one huge cloud and many very small clouds," says Divine, a mathematician at the University of Bath, "and the two scenarios will cause the weather to show a huge difference, but the weather forecasting models currently in use don't take that into account."

However, with a little change of direction, we can see hope. Instead of trying to speed up calculations, perhaps we can harness the power of computers to accomplish complex tasks by learning from large amounts of existing data. This is machine learning, which is a form of artificial intelligence. From online shopping to healthcare, AI is currently entering all areas of life. If this idea also applies to meteorology, weather forecasts will become more accurate and require less computing power than current weather models.

Traditional weather forecasting model

Weather is the result of a combination of the movement of the Earth's atmosphere and oceans, the movement of moisture through the atmosphere, and changes in air pressure and temperature. The atmosphere and oceans are gases and liquids, respectively, and they are both fluids, and in meteorology, there is exactly a set of equations that describe the motion of fluids: the Navier-Stokes equation.

The principle behind weather forecasting is relatively simple. Factors that describe the current weather are measured first, such as temperature, air pressure and density, wind speed, and humidity of the air. This data is then fed into a mathematical model built on the Navier-Stokes equation, so that changes in the weather can be calculated in time on a computer.

In practice, however, there are several things that can make weather forecasting tricky. First, you can't measure temperature, pressure, humidity, etc. at every point on Earth. Second, you can't measure them with unlimited accuracy. The famous butterfly effect means that as calculations progress, the inevitable little error can become very large, resulting in a very biased prediction. Third, due to the complexity of the Navier-Stokes equations, applying them to weather models requires a lot of computing power.

Earth Imageization

To be able to make predictions, weather modelers divide the Earth and its atmosphere into a grid, much like a television or computer screen splits an image into pixels. Just as each pixel is assigned a color, each grid box is assigned only one value for pressure, humidity, temperature, etc. — a value that is accurately measured by a single grid box, making calculations easy. We can then use techniques such as ensemble prediction to mitigate the effects of the butterfly effect.

Weather models divide the Earth and its atmosphere into a grid. Image: National Oceanic and Atmospheric Administration.

In the most advanced weather models currently available, the grid is about 1.5 kilometers square horizontally and about 300 meters high vertically: even the fastest supercomputers can't handle higher resolutions. Clouds can certainly be much smaller than that, they can do all sorts of wonderful things within a grid box, and many other processes can happen at scales smaller than the grid box.

To take these processes into account, weather models use mathematical formulas that roughly describe the physical properties of these processes. This estimation is called parameterization.

"Parameterisation is a step in modelling that calculates the physical properties of what is happening inside a grid box and then correlates it with the grid scale," explains Chris Bard, mathematician and expert in weather forecasting and machine learning at the University of Bath. The proportion of the sky covered by clouds in a single grid box is one quantity that is thus parameterized. "In addition to clouds, there are parameters such as radiation from the sun, fluctuations caused by gravity in the atmosphere, and friction experienced by the wind as it blows across the Earth's surface," Bard said.

What can AI do?

Machine learning is when computer algorithms learn how to discover patterns in data and then make full use of those patterns for practical applications. A classic example here is a computer learning to tell a picture of a cat from a picture of a dog. To teach a machine learning algorithm to do this, first feed it a large number of pictures of cats and dogs and tell it the correct answer for each image — whether it's a cat or a dog.

In a seemingly magical but efficient mathematical process, the algorithm carefully analyzes the picture and adjusts the internal parameters until it obtains a very high accuracy rate in the training set. Then you can give it new pictures of cats and dogs, and he will be able to distinguish the animals in the pictures with a high degree of accuracy.

When it comes to weather forecasting, we hope that machine learning algorithms can learn how to determine some details of what is happening inside the grid box from the numbers associated with the grid box by looking at a large number of real-life weather. If possible, these algorithms could be incorporated into weather models, replacing existing parametric algorithms and allowing models to include more detailed information about sub-mesh processes — including more details about the cloud's behavior and organization.

Try AI

Both Bard and Devine are part of a research group called "Mathematics in Deep Learning," which explores a range of potential applications of machine learning and the mathematics behind it. They mentored graduate student Coward on a project with the Met Office to test whether machine learning could provide more information about cloud cover.

The total surface area of these cirrus clouds is greater than the surface area of the same volume of cloud spheres. Image: Fa Martin

For such a test, the first thing we need to do is determine what we want the machine algorithm to learn about the cloud. Coward's answer based on geometric results is that in the case of the same cloud cover, the surface area of the entire cloud when the clouds are all gathered together tends to be smaller than when it is divided into many small clouds.

Therefore, the surface area of the entire cloud, also known as the cloud perimeter, is a good indicator of what kind of cloud is present in the grid box – Cumulus macro or slender cirrus. It is also a useful parameter for improving other parameterization processes and algorithms, such as those that predict the transmission of radiation through the clouds.

The question is whether machine learning algorithms can estimate the cloud perimeter within a single grid box based on the numbers assigned to the entire grid box. "This is the goal of the Cowardd project: machine learning on the estimation of the cloud perimeter based on a range of environmental factors." Devine said.

To train the algorithm, Coward used a dataset of clouds recorded in Oklahoma, USA. "They set up a bunch of cameras in the space," Devine explains, "and the cameras can read if there are clouds on a one-meter-sized grid scale." Over a three-year period, cloud layers were recorded every 20 seconds, and using that data, machine learning algorithms produced what Coward calls "completely unique insights into the cloud lifecycle."

Coward used this data to train two machine learning algorithms. After training them, he compared the cloud perimeter predicted by the algorithm with the cloud perimeter recorded by the camera.

The better of the two algorithms has an error of 16%. It's not zero, but it's not very big either. In fact, the best methods for parameterizing cloud perimeters without using machine learning also have an error of close to 24%. So, in this case, machine learning is more than a third more accurate than non-machine learning.

Proof of concept

Coward's project is one of a series of initial attempts to test whether machine learning can be used for weather forecasting. "Machine learning is a very new approach for people in the field," Devine said, "and we're in its infancy right now, and most of the content is experimental, and people are trying different things, trying to come up with new technologies and see how they perform." ”

The hope is that machine learning will eventually calculate not only cloud cover, but also other phenomena in weather models. If this method succeeds, artificial intelligence is finally applied to the weather forecasting app, and then you will know the good news.