Over the past few years, DeepMind must have been the most compelling AI team. Not only has it developed artificial intelligence like AlphaGo, which has made headlines, but it has also helped scientists push the frontiers of science in many scientific fields. The various types of artificial intelligence developed by the DeepMind team have made major breakthroughs in the "protein folding problem", providing the chemical community with tools to predict the distribution of electrons within molecules, as well as machine learning to identify mathematical structures and patterns. (See "A Fundamental Big Problem in Biology Ushers in a Major Breakthrough!") "The Most Powerful Weapon for chemistry", "Exploring Novel Ways of Pure Mathematics". ）
Recently, the DeepMind team brought another surprise. In collaboration with the Swiss Center for Plasma (SPC), it has developed a new approach to magnetic control of plasma based on deep reinforcement learning and, for the first time, applied to real-world plasma control, a transformative technology that opens up new avenues for advancing nuclear fusion research. The study was recently published in the journal Nature.
"Artificial star" tokamak
To address the global energy crisis, researchers have long sought an unlimited source of clean energy. Nuclear fusion is a strong candidate. By crushing and fusing hydrogen, this process is able to release enormous amounts of energy. This reaction is "ubiquitous" in the universe, powering stars, including the Sun.
On Earth, scientists are also trying to "build the sun." One way to reproduce such extreme conditions is to use a device called a tokamak.
The tokamak is a donut-shaped vacuum device surrounded by magnetic coils that uses a powerful magnetic field to confine the plasma to extreme heat of hundreds of millions of degrees Celsius, which is even hotter than the core of the sun, allowing nuclear fusion reactions between hydrogen atoms. This method is widely used, and there are currently about dozens of tokamaks in use around the world, including China's famous "Oriental Supercycle" (Advanced Experimental Superconducting Tokamak Experimental Device).
In the new study, a tokamak device located at the SPC allows the presence of various plasma loci, hence the name conjuvation tokamak (TCV). The position of a plasma is related to its shape and position in the device, that is, scientists can use it to investigate new ways to limit and control plasma.
TCV photos and 3D models. | Image credit: DeepMind & SPC/EPFL
But the plasma in the device is inherently unstable, and the processes required to sustain nuclear fusion become a complex challenge. For example, a control system needs to coordinate the many magnetic coils of the tokamak and adjust their voltages at a frequency of thousands of times per second, ensuring that the plasma does not touch the walls of the container and pose problems.
Leverage AI to help solve problems
In the new study, the research team describes how a controller was built and run on the TCV by an AI algorithm that successfully controlled the fusion plasma. Using a learning architecture that combines deep RL and a simulated environment, the team built a control system that both keeps the plasma stable and precisely shapes the plasma into different shapes.
The first problem the team faced was the lack of data. TCV can only maintain the plasma for about three seconds at most in one experiment, after which it takes 15 min to cool and reset before the next attempt can be made. Not only that, but multiple research teams often use tokamak together, which further limits the time available for experiments.
Given this obstacle, they first turned to the simulator to help advance the research. SPC has built a powerful set of simulation tools capable of modeling the dynamics of the tokamak. These tools allow deep reinforcement learning systems to learn to control TCVs in simulations and then validate these results on real TCVs.
While this is a cheaper and more convenient way to train controllers, there are still many hurdles. For example, the plasma simulator runs so slowly that it takes hours of a computer to simulate a one-second real-world experiment. In addition, the condition of TCV changes every day, which requires algorithm improvements in physics and simulation to adapt to the actual situation of the hardware.
The core issue remains complexity. Existing plasma control systems are complex, with each of TCV's 19 magnetic coils requiring a separate controller, and each controller uses algorithms to estimate the characteristics of the plasma in real time and adjust the voltage of the magnet accordingly.
The architecture of the new study revolutionizes this by using a single neural network to control all the coils at once and automatically learn what voltage conditions are best for achieving a certain plasma profile.
The team first demonstrated the ability of a single controller to manipulate many aspects of the plasma. The controller first shapes the plasma according to the desired shape, then moves the plasma down and separates it from the wall, suspending it in the middle of the container with two "legs". The plasma is stabilized, which is also necessary to measure the properties of the plasma. Finally, the plasma is directed back to the top of the vessel and safely destroyed.
A controller trained with deep reinforcement learning guides the plasma through multiple stages of the experiment. The figure on the left is an internal view of the tokamak during the experiment. The image on the right is the reconstructed plasma shape. | Image credit: DeepMind & SPC/EPFL
Subsequently, the team successfully created a series of plasma shapes. For example, they made a "snowflake" shape with many "legs" (2 right in the figure below) that can help reduce the cost of cooling by dispersing the exhaust energy to different contact points on the vessel wall.
Deep learning controllers create a series of plasmas of different shapes. | Image credit: DeepMind & SPC/EPFL
The team also showed the shape of a proposal for a next-generation Tokamak ITER (International Thermonuclear Fusion Experimental Reactor) under construction (middle of the figure above) to help predict the behavior of ITER's medium ions. They even tried some shapes that had never been tried before in TCVs, called "droplets" (1 left in the image above), when there were two plasmas inside the container at the same time.
It can be said that this system can find controllers for all these different conditions. By changing the set goal, the algorithm can find a suitable controller on its own.
The future of nuclear fusion
The implementation of this "plasma shaping" shows that the system has successfully controlled the plasma and, more importantly, has allowed scientists to study how plasma reacts under different conditions, improving our understanding of fusion reactors.
This study is another powerful example of how machine learning can help scientists tackle major challenges and accelerate scientific discoveries. A successful demonstration of tokamak control shows the power of artificial intelligence in accelerating and assisting the science of nuclear fusion.
The DeepMind team believes that the use of artificial intelligence will become more and more complex in the future. This ability to create controllers autonomously can even be used to design new types of tokamaks and their controllers. They also predict that in the coming years, reinforcement learning may be expected to become a transformative technology for industrial and scientific control applications.
#创作团队:
Compile: Takeko
Typography: Wenwen
#参考来源:
https://deepmind.com/blog/article/Accelerating-fusion-science-through-learned-plasma-control
https://actu.epfl.ch/news/epfl-and-deepmind-use-ai-to-control-plasmas-for-nu/
#图片来源:
Cover image: Curdin Wüthrich /SPC/EPFL
Top image: DeepMind & SPC/EPFL