Text | Academic headlines, author | Cooper, Editor, | Kou Jianchao
Many potential applications of artificial intelligence (AI) involve making more optimized real-time decisions when interacting with humans, and competitive or game games are the best stage for display.
Today, a cover article published in the journal Nature reports that AI has defeated world champion human players in the racing battle game Gran Turismo (GT Racing). The AI program, called "Gran Turismo (GT)Sophy," is a neural network driver that exhibits exceptional speed, maneuverability, and driving strategy while adhering to the rules of racing.
The core team that completed the development of this AI program came from Sony AI, the GT Racing series of games was developed by Polyphony Digital in Japan, faithfully reproducing the non-linear control challenges of real racing cars, encapsulating complex multi-agent interactions, and the game was released on game console platforms such as Sony PlayStation and PSP, making it a popular racing game with a very realistic manipulation experience.
With the blessing of this AI program, human players will probably no longer be able to run the enhanced version of the stand-alone program, right?
Screenshot of the game | (Source: GT Racing)
Researchers believe that this result may make racing games more interesting and provide high-level racing for training professional racers and discovering new racing techniques. This approach is also likely to be applied to real-world systems such as robots, drones, and self-driving cars.
Speed and excitement on the track
Driving a racing car requires great skill. Modern Formula One racing has to do with astonishing engineering precision, however, the sport's popularity has less to do with the car's performance PK as with the skill and courage of top drivers when pushing their car's performance to the limit. For more than a century, success on the track has been fraught with speed and passion.
Figure | Formula 1 race (Source: GNEWS)
The goal of racing is simple: if you finish the track in less time than your competitors, you win. Achieving this, however, requires extremely complex physics warfare, and racing the track requires careful use of friction between the tires and the road, which is limited.
In order to win the race, the driver must choose to keep the car on track within the ever-changing limits of friction. Brake too early when cornering, and your car will slow down and waste time. Brake too late, and when you're close to the tightest part of the turn, you won't have enough cornering force to keep the route you want. Braking too hard may cause the car to rotate.
As a result, professional racers are very good at discovering and maintaining the limits of the car through lap after lap throughout the race.
Although the maneuvering limits of racing cars are complex, they can be well described in physics, so it is only natural that they can be calculated or learned.
In recent years, deep reinforcement learning (DRL) has become a key component of AI research milestones in areas such as Atari, StarCraft, and Dota. In order for AI to have an impact on robotics and automation, researchers must demonstrate the ability to successfully control complex physical systems, and in addition, many potential applications of AI technology require interactions in close proximity to humans, while respecting imprecise human norms, and car racing is a typical area full of these challenges.
Figure | game match data comparison (Source: Nature)
In recent years, research into autonomous racing has accelerated with the use of full-scale, large-scale and simulated vehicles. A common approach is to precompute trajectories and use model predictive controls to perform those trajectories. However, when driving at the absolute limits of friction, small modeling errors can be catastrophic.
Racing with other riders places higher demands on AI modeling accuracy and introduces complex aerodynamic interactions, further prompting engineers to improve control schemes to continuously predict and adapt to the optimal trajectory of the track, and one day, driverless cars will compete with human riders off the track.
The refinement of the "AI racer"
During the development of GT Sophy, the researchers explored a variety of ways to use machine learning to avoid modeling complexity, including using supervised learning to model vehicle dynamics, and using imitation learning, evolutionary methods, or reinforcement learning to learn driving strategies.
In order to succeed, racers must possess a high degree of skill in four areas: (1) car control, (2) racing tactics, (3) racing etiquette, and (4) racing strategy.
In order to control the car, the drivers have a detailed understanding of their vehicle dynamics and the characteristics of the track. On this basis, the driver builds the required tactical skills to perform precise maneuvers by defending the opponent. At the same time, drivers must adhere to highly refined but imprecise rules of sportsmanship, and finally, drivers apply strategic thinking when simulating their opponents and deciding when and how to attempt to overtake.
The success of GT Sophy in an environment that requires real-time, continuous control in a highly realistic, complex physical environment shows for the first time that in a range of car and track types, it is possible to train a better AI agent than top human racers.
This result can be seen as another important step in the continued development of competitive missions such as chess, Go, adventure, playing cards, and StarCraft.
Figure | GT Sophy training (Source: Nature)
Notably, the GT Sophy learned to take a detour in just a few hours and surpassed 95% of the human players in the dataset, training for another nine days, accumulating more than 45,000 hours of driving time and reducing lap time by a tenth of a second until lap time stopped improving.
Progress rewards alone aren't enough to incentivize AI programs to win the game. If the human opponent is fast enough, the AI program will learn to follow and try to accumulate more rewards without risking a potentially catastrophic collision to achieve overtaking.
To evaluate gt sophy, the researchers pitted GT Sophy against top GT drivers in two races, and the GT Sophy achieved superhuman timing performance on all three tracks tested, capable of performing several types of turns, effectively using drift, disrupting vehicles behind, intercepting opponents and performing other emergency maneuvers.
While GT Sophy has demonstrated sufficient tactical skills, there are still many areas to be improved, especially when it comes to strategic decision-making. For example, GT Sophy sometimes leaves enough room on the same track for opponents to take advantage of.
Figure| AI driver surpasses human player (Source: Nature)
Competitive gaming is more worth paying attention to
Regarding e-sports, game games, AI can defeat humans is not a rare thing, and it is certain that AI will become stronger and stronger, even the top human players can only bow to the wind, but can win electronic games without much suspense and significance, the key is to see how these ai programs that surpass humans can effectively overcome industrial bottlenecks and truly benefit human life.
On February 10, 1996, the supercomputer Deep Blue challenged chess world champion Kasparov for the first time to lose 2:4. In May 1997, the Deep Blue defeated Kasparov 3.5:2.5, becoming the first computer system to beat the world chess champion within the standard match time frame.
But Deep Blue's flaw is that it has no intuition, does not have a real "intelligent soul", and can only rely on super computing power to make up for the shortcomings of analytical thinking, and Deep Blue, which won the game, soon retired.
In March 2016, Google AI's AlphaGo beat Go world champion Lee Sedol in four games, considered a milestone in the true sense of AI, AlphaGo used a combination of Monte Carlo tree search and two deep neural networks, in this design, the computer can spontaneously learn to analyze and train like the human brain, and constantly learn to improve chess power.
Since then, a variety of AI program rookies have emerged, and on December 10, 2018, DeepMind's artificial intelligence AlphaStar, developed for the real-time strategy game StarCraft, was able to abuse 99.8% of the world's human professionals.
Undoubtedly, the current GT Sophy is a continuation of ai victory.
J. Christian Gerdes, a professor in the Department of Mechanical Engineering at Stanford University, believes that the impact of the GT Sophy study may extend far beyond video games, and as many companies work to perfect fully autonomous vehicles that transport goods or passengers, it is worth exploring further as much of the software should use neural networks and how much should be based solely on physics.
Overall, neural networks are the undisputed champions when it comes to perceiving and recognizing objects in their surroundings. Trajectory planning is still the field of physics and optimization, however, and GT Sophy's success on the gaming track suggests that neural networks may one day play a bigger role in software for automated vehicles than they do today.
Perhaps more challenging is the change per lap. In reality, the condition of a car's tyres changes from lap to lap, and human drivers have to adapt to this change throughout the race. Can GT Sophy do the same with more data? Where does this data come from? This will give artificial intelligence more room for evolution.
Resources: