EA IS TRAINING AI PLAYERS IN BATTLEFIELD 1
The term "AI" has been used in video games since their inception, but it rarely means true artificial intelligence. It is rather seen as a generic term to describe a preprogrammed opponent or character that feigns intelligence but is really just following a narrow set of instructions. This is slowly changing with the help from people who build video games.
At GDC, EA announced that it's been training AI agents since 2016's WWI shooter Battlefield 1. The company says it's the first time this sort of work has been done in a high budget AAA title (which is disputable), but more importantly, it says the methods it is developing will help improve future video games: providing tougher, more realistic enemies for human players and giving developers new ways to debug their software.
The EA AI agents – which, unlike bots, are expected to learn how to play instead of following a set of instructions – are being trained using a combination of two standard methods, one is imitation learning and the other reinforcement learning. The first part involves the agent watching human players and then attempting to mimic them. That constitutes roughly 2% of their knowledge and sets them up on the right path.
After the learning is done, agents have to figure out the rest of the game themselves, with rewards for completing tasks (like killing the enemies) and helping them through a process of trial and error. That is part of the reinforcement learning. EA's agents play hundreds and hundreds of Battlefield games at an accelerated pace and thus improve over time. It is similar to the methods that DeepMind used to train its Go-playing AI.
You can see the AI-agents in action below:
During their training, the bots pick up all sorts of skills. They learned how to adjust their aim, to gun recoil and proved to be surprisingly good at dodging bullets. "They jump from side to side in order to not get hit," says EA’s Search for Extraordinary Experiences Division (SEED), Magnus Nordin.
Although some of their actions show how far there is to go before AI agents can play video games as naturally as humans, Nodin explains that these bots developed a particular "scanning" behaviour, where they spin around looking for something to interact with. This is due to the way they were trained using reinforcement learning, which rewards them if their scores go up or they pick up spare health and ammo. The latter reward was added because EA's researchers thought it would encourage agents to explore the maps ( as well as help them survive throughout). It does indeed come with a side effect of limiting their ambition. "When they don’t see any enemies they just start scanning for something to do," says Nordin.
Arthur Julian, who is a senior machine learning engineer at Unity, said EA's work was similar to earlier research in this domain, but solid nonetheless. "Combining these approaches and demonstrating that they can be used to train agents which can be deployed in real games is an exciting prospect," said Julian.
Nodin, acknowledges the prior AI work done in games like Chess and Go. In board games, you have a complete overview of the world and a finite number of possible outcomes, which a computer can model exhaustively learn and then select the best option. But in a shooting game, a player has to press multiple keys at once in order to walk, run, crouch, fire and so on, and the combination of all those possible actions together with trying to predict the actions of others makes a vastly more complex task.
For EA games, the aim is not to further the general field of AI, but rather to find out how this technology can benefit game developers. Nodin says he does not think that we will see AI players taking on humans for a while. "They won’t be in the next Battlefield because that’s not very far away, but probably the one after that – as a hybrid of classical AI and neural networks," he adds.
So before AI players start beating your ass at your favourite game, they will at least help us make them, which is not a terrible deal.