Anyone who has played a driving game for any significant amount of time has at one time or another felt like the console was somehow cheating to win. And he definitely has known as the so-called Elastic Effect: When in front, computer controlled cars stretch the gap until they seem out of reach, but when in back they stick to the player and it’s hard to shake them off. Precisely as if there were a rubber band that conditions its movements.
It’s a gimmick that programmers often use to keep up the sense of challenge in virtual races, but we’re not talking about that here. Here we are talking about “a skill incredible, far beyond human limits”.
Science and science fiction 40 years after the Terminator, James Cameron and the fear of AI: “I warned you, you didn’t listen to me” by Emanuele Capone 31 July 2023
“To beat her you must never get distracted”
The words are of Valerius Galloundoubtedly the best Italian player of Grand Touring It is probably among the best sim-drivers in the worldand they refer to GT Sophy, which is the artificial intelligence that Sony AI and Polyphony Digital are developing for the hugely popular PlayStation video game.
Made accessible to the public on a limited number of circuits inside Gran Turismo 7 between February and March lastSophy owes its name to the Greek word sophia, which means wisdom: try to be a wiser AI than the other AIs seen so far in video games, which (simplifying) know what are the optimal travel speeds in the various sectors of the various tracks and they aim to keep them for as long as possible following precise trajectories.
Sophie doesn’t do that: “I’ve seen them take trajectories that are unthinkable for a person”, Gallo told us again, whom we met in Amsterdam on the occasion of the screening of the film Grand Touring (video). Unthinkable twice, in the sense that “I didn’t think they were feasible” and also “I had never thought of trying them”. Precisely because a human pilot would not think them practicable. Again: “On GT7 tracks that I know by heart and where I think I drive very well, he has all the braking points further forward than mine”, because he has evidently learned that he can delay braking by a few meters to enter corners more quickly. With a smile, Gallo told us Sophy ‘is illegally fast’, a hyperbole to imply that we are on the verge of what up to now was understood as cheating. But it’s not that Sophy cheats, it’s that she learns: “she’s become very good, she adapts well to situations, even to the tussle typical of the first laps of a race, it is aggressive the right, almost on a human level”. This is an important detail, because (again) anyone with a bit of experience with driving games knows that, no matter how good, computer opponents are rarely as aggressive and cunning as people. They rarely drive dirty, as they say.
However, the real strength of Sophy is not (only) this, but it is a characteristic common to more or less all AIs: it is the constancy of performance. They are machines, artificial brains, computers, which they never get tired, they don’t get distracted, they don’t feel the pressure and they always perform at their best: “To beat her you have to be constantly at the limit and you can never get distracted – Gallo told us – Otherwise you’ll get screwed”.
youtube: Valerio Gallo vs. GT Sophy
How GT Sophy was trained
Sophy, whose also the magazine Nature has dedicated a rich and interesting analysis (This)was trained with the so-called reinforcement learning or reinforced learning: in English it is called reinforcement learning, it is a form of machine learning and It’s basically about learning from your mistakesso as not to commit them again.
It’s a process that has lasted for Sophy a couple of years, even if the actual training part was much, much shorter: about twenty months are the ones needed to establish the training program, decide how and on which data and information to make it learn, but a couple of weeks. About 45,000 hours of continuous testing, and a couple of days after taking to the track for the first time, Sophy was already faster than 95% of the human pilots taken as a reference. Arriving at superhuman abilities also noticed by Gallo, in particular on 3 circuits of Gran Turismo 7: for those who know the game, they are those of Lake Maggiore, Dragon Trail and the Circuit de la Sarthe (the one on which the 24 Hours of Le Mans is raced in the real world).
In detail, the training program started in April 2020, with over 10,000 hours of solo playand came alive in the second half of 2021: between July and October of that year, Sophy was challenged in a series of 3 repeated matches by 4 of the best players in the world of Gran Turismo Sport. Each had a different combination of circuit and car to use, and points were awarded at the end based on how well the various cars finished, some run by Sophy and some by the players: during the first race, the human team reached 86, while the AI stopped at 70. Already from the second race, the sides had reversed: 104 points for the Sophy team and just 52 for the gamers.
From there it was a crescendo, also fueled by innate advantages which Sophy, being an artificial intelligence, has compared to humans: she knows the track map perfectly, with the exact coordinates of the borders and limits not to be exceeded, and has in real time all the “precise information on the load of each tire and on sealing limits of each rubberas well as other data on the state of the vehicle it drives,” as the developers themselves admitted.
Which in fact they are looking for to limit it, to slow it down and somehow make it more human. In short, less illegal. They do this by acting on two parameters in particular: the refresh rate of its actions (initially blocked at 10 Hz, against a theoretical maximum of 60) and above all the reaction times. According to the findings, Sophy is able to react to what is happening around her speed measured between 23 and 30 milliseconds, against 200-250 milliseconds for a human pilot. Of a good human pilot, of course. To make it more manageable, its creators have added an artificial delay, locking it between 100 and 250 milliseconds. Which hasn’t stopped her from beating more or less everyone who has faced her, and also from turning this situation to her advantage.
Videogame An all-new Ferrari arrives, but you can only drive it in a videogame by Vincenzo Borgomeo 28 November 2022
twitter: Sophy learned to drift
Zero Shot, the things Sophy taught herself
Just in Amsterdam, Sony explained to us that “while we were trying to make it slower and less perfect” a strange thing happened. It just so happened that Sophy learned to drift. And of course she’s doing it better now than anyone who’s ever done it before.
Drift is a sort of controlled skid, and knowing how to master it is essential both for managing any oversteer and for increase speed out of a corner. In addition to making a show, as can be seen from the video above, shown for the first time on stage at the Theater Amsterdam.
There is no scientific explanation here on how Sophy learned this skill: like other AIs, it shows what are called emergent behaviors, that is, precisely skills that the developers did not think it had and above all they weren’t going to teach her (it’s called Zero Shot learning). What is known is that here too, again, he does things that humans didn’t believe they could do, didn’t believe it was useful to do and probably wouldn’t even be able to do in the same way: “He learned that passing suddenly from D a R (from forward to reverse, ed.) manages to stabilize the car during a skid, and manages to do it with a speed that is unthinkable for us”. An important point to understand is that everything he does is feasible and is “based on the physics of the game”: we could do it too, if only we were able.
It’s still not clear when GT Sophy will finally be accessible to everyone the players of Gran Turismo 7, after the tests at the beginning of the year: what is certain is that “it does things that 5-6 years ago were unimaginable for an AI of this type”, as explained by Sony. Or even, if she told us to paraphrase the Nexus 6’s Blade Runner“I’ve done things that you humans you didn’t think they could be done.”
@capoema