Fish and sheep Bowen (This article is reproduced from the public account "qubits")
Unexpectedly, Sony's AAA game masterpiece actually appeared on the cover of Nature:
Publicize the game to the cover of top academic journals, I serve.
However, this article is not an additional money to advertise, but a real technical paper -
An AI from Sony called GT Sophy.
Look at the AI race driver's race screen, first a witty change of lanes to bypass the opponent's card slot:
Then, when glued in the corner, daxiu operated, and at a speed of 190km/h, he overtook the opponent in one fell swoop, leaving the opponent behind.
Don't think that AI defeats humans in racing games is nothing special, you know, such a race is played in GT racing cars -
This is a highly realistic racing game, which means that the players are almost driving in real life.
Pictured on the right is GT world champion Takuma Miyazono
GT Sophy is also the world's first AI to beat the human champion in such a racing game that requires continuous judgment and high-speed reaction.
Professor Chris Gerdes, co-director of the Stanford Automotive Research Center, said:
Gt Sophy is so adept at outpacing human drivers in the race that this is a milestone advance for AI.
It is worth mentioning that the latest generation of GT series games Gran Turismo 7 will officially land on the PS4/PS5 platform on March 4.
While players are unlikely to play against GT Sophy in the first place, Sony has said they are working to integrate this powerful AI into the GT race car.
It's not simple to race AI
The victory of AI over humans is not new enough to many people.
But compared to the AI that used to play in board games such as chess and Go, or action games such as StarCraft and Dota, the simple goal of racing that seems to be "first to the finish and then win" is actually much more complicated:
Brakes, for example, are a kind of "physical warfare" that constantly explores the friction between car tires and roads.
Brake too early when cornering, you may not be able to cross the corner smoothly, and you will lose speed;
Braking too late, it is difficult to maintain the desired route trajectory, and even cause the car body to rotate.
Throughout the race, racers must always maintain judgment of their surroundings and be able to react in a timely manner based on road conditions.
At the same time, driving skills, tactical strategies for different opponents, even racing spirit and etiquette that follow the rules... All are the qualities necessary for a racing AI.
To achieve these goals, Sony set up ai's training ground in its own GT racing series of games.
It's a PS platform exclusive game, full name Gran Turismo, a true term in the automotive industry, meaning "a high-performance car capable of driving long distances."
The main theme of this game is the real driving sense, and even through a set of steering wheel peripherals with force feedback, you can experience a variety of delicate road sense feedback such as lateral g-value, front wheel slip rate, vehicle weight, and the tuning characteristics of the vehicle.
In GT games, study this KIND of GPU, which typically uses 10 to 20 PS game consoles, the same number of computing instances, and an asynchronously updated neural network, and train this AI called GT Sophy from scratch.
Training is focused on two dimensions of continuous values: steering and speed.
At the same time, in order to pull GT Sophy and humans to the same running line, the researchers also slowed down its operation speed:
The frequency of interaction with the game is 10 Hz (60 Hz by human standards), and the reaction time is 100 milliseconds, 200 milliseconds, and 250 milliseconds, respectively.
In a complete map, the researchers quantified the tracks run by the agents into "static map segments" in 6 seconds, and encoded the left and right edges and centerlines of each track into 60 equidistant three-dimensional points:
The training ground is ready, and now it's time for GT Sophy to truly be a cornering, thinking racing driver.
Actor-Evaluator Mechanism
Sony used a new deep learning algorithm, also known as the actor-evaluator (QR-SAC) method of quantitative regression.
THE AI agent is thrown into an environment without instructions and selects actions based on the evaluation of the evaluator (i.e., the value function) and is rewarded or punished based on the results of achieving the goal.
For example, an agent is rewarded for keeping at high speeds, but is also punished for going out of bounds, hitting a wall, or losing traction.
This reward and punishment mechanism allows agents to quickly get positive feedback and slowly begin to familiarize themselves with their behavior on the track.
In fact, it took gt sophistication to learn to drive on the track in just a few hours, and soon surpassed 17,700 players on the Maggiore track:
But this sheer speed, or the reward for the progress of the track completion, is not enough to motivate the agent to win the race.
Because if the opponent is fast enough, the agent can accumulate a large number of rewards by following and learning the opponent's "laziness", and eventually complete the goal.
As a result, the researchers added an "overtaking bonus," and the change in distance relative to the opponent was also proportional to the bonus points.
At the same time, there is also a reward and punishment mechanism on the collision of the car, and the researchers have set a more conservative reward:
While ensuring that the intelligent body can win with a strong enough driving style, it will not rudely drive other cars off the road and be punished by the real racing rules.
In the end, GT Sophy overcame many of the world's top racing drivers in GT Racing, including the 2021 TGR GT Cup winner, the 2020 Nations Cup World Champion and the 2021 Runner-up, the 2018 Nations Cup Asia/Oceania Champion, and so on.
In the race, GT Sophy finished first on all three tracks.
And in the race, it is able to perform several types of turns, effectively utilizing drift, disrupting vehicles behind, intercepting opponents and performing other emergency maneuvers.
However, the researchers also mentioned that GT Sophy still has some room for improvement in strategic decision-making: sometimes it leaves enough space on the same runway for opponents to take advantage of.
Sony is going to use it to develop autonomous driving
Seeing this, what do you think of GT Sophy's capabilities?
Anyway, several world-class players are convinced.
Takuma Miyazono, who won three crowns at GT Racing's world-class events, said:
Sophy was very fast, with lap times beating previous expectations for the best drivers.
In one of the races, GT Sophy Rouge defeated second-place Tomoaki Yamanaka (2021 TGR GT Cup winner) by 5.8 seconds, with a fastest lap time of 1:54.373, more than 2 seconds faster than Yamanaka.
And GT Sophy's ability is obviously not only able to kill all sides in the game.
Professor Chris Gerdes, co-director of the Stanford Automotive Research Center, pointed out that the AI technology behind GT Sophy can help develop self-driving cars:
GT Sophy's success on the track suggests that one day, neural networks could play a bigger role in self-driving car software than they do now.
At present, neural network algorithms are mainly responsible for tracking road markings and perceiving obstacles in self-driving car software, while control software still relies on artificially set rules.
Avenash Balachandran, senior manager at Toyota Research Institute, also said that "applying machine learning and autonomous control in racing is exciting".
There are also many netizens who believe that this AI may be related to Sony's newly established electric vehicle company, and perhaps in a few years, this AI can be seen in its automatic driving technology.
Of course, for Sony players, the most anticipated thing is: can the GT7, which will be launched next month, join this strongest AI?
The good news is that the GT7 will still land on the previous generation of consoles, the PS4, and players who did not buy the PS5 can also try it.
It is reproduced from the public account "Qubit" with authorization, and the content does not represent the views of this public account.
bibliography:
［2］https://sonyai.github.io/gt_sophy_public/