Google DeepMind thinks the best way to teach machines how to learn about the world is to place them in a virtual one, namely in video games. Since the entire experience is virtual, it’s easy to reproduce exact scenarios and get a nearly unlimited amount of data from a single game. In the past, DeepMind has developed algorithms to learn from (and beat) 2D Atari games like Breakout and Pac-Man.
In a recent paper, DeepMind has taken on the third dimension, and built a program that can navigate a 3D, Doom-like maze. The algorithm isn’t reading the code of the game, it’s literally looking at what any human player would see, and making decisions and judgments about where to go.
The algorithm was developed to explore a 3D maze-generator called Labyrinth, where the goal is to find rewards in a randomly generated maze. They also tested three other platforms, an Atari 2600 platform, a physics modelling platform, and a platform that resembled a car-racing game. Unlike Doom, there’s no shooting involved.
In the maze explorer game, points were awarded for finding apples and portals. If the algorithm found an apple, it got one point. If it found a portal, it was given 10 points and transported to another random location in the maze. After 60 seconds, the game ended and another fresh game began. With no training, the computer got about 2 points per game, but after 200 million steps over 3 days, it averaged around 50 points. DeepMind concluded that this score means that the computer found a “reasonable strategy” for exploring and finding the portals. However, the team didn’t supply a human benchmark for this game.
This is cool for both AI researchers and gamers. For researchers, this is could mean better tactics for computer vision and spatial recognition, which is big in self-driving cars and robots. If we can deploy these algorithms in extremely detailed, yet virtual scenarios, it might make it exponentially easier to train the brains of robots that exist in our real world. We’re also getting to the point where we can analyze behavior of artificial intelligence. In games that learning algorithms have already mastered like chess or Go, those well-acquainted with the game can tell what an aggressive or passive move would look like, but it’s much easier to tell in a 3D environment.
And this also means in the future, gamers could play against a literal A.I. that learns and strategizes, rather than pre-coded bots with difficulty settings. Different characters could have different personalities based on specific training data, and they would react differently from the experiences they have with you in-game.