Sony’s AI can employ clever strategies to beat the best humans at Gran Turismo

Sony Corp. announced today that it has created an artificial intelligence that can get the better of humans when playing the simulation game Gran Turismo, which could have implications in the future for self-driving technology.

AI has already mastered such games as Go and chess, and according to Sony, it took just two days of training for the technology Gran Turismo Sophy (GT Sophy) to leave human players in the dust. At the two-day mark it was beating 95% of the best human players, and in the following days kept shaving time from its previous results.

Sony said the AI mastered a number of tracks, not only by figuring out when to slow down or accelerate but using tactics such as when the time is right to get in behind a car and use the slipstream. When it became obvious that the car wasn’t using a good racing line, it would change to another line. On top of that, it soon learned that it could be handed race penalties for various infringements on the track, as happens in real racing.

“Gran Turismo Sophy learns through novel deep reinforcement learning techniques to race at the highest level,” said Sony. “It is trained using the cloud gaming infrastructure managed by SIE for massive scale training.”

What’s interesting in this case is the AI’s ability to understand strategy, in an environment where everything matters. An example given was that it would seem rational to move as close to the car in front as possible, but the AI soon figured out that because the car in front may brake at any time, it is sometimes best to leave a gap and only go for the overtake when the time is right.

The question now is, if an actual racing car was controlled purely by such an AI, could it do as well? So far, self-driving car technology is mostly about identifying objects rather than figuring out what is the best line to take in busy traffic. This AI, on the other hand, is more about using strategies to get people to where they want to go in the least amount of time.

“What really stands out is GT Sophy’s performance against human drivers in a head-to-head competition,” said Christian Gerdes, co-director of the Center for Automotive Research at Stanford University, in Nature. “Far from using a lap-time advantage to outlast opponents, GT Sophy simply outraces them.”

Sony said that the AI has room to improve, such as not allowing its car to be used by other drivers when they get in its slipstream. The company also said that the car became “notably more aggressive” when it played with the absence of collision penalties.

Image: Gran Turismo