OpenAI: Development of SkyNet for gaming doesn’t stop

If you play Dota 2, you must have heard about OpenAI — a non-profit organization that builds sophisticated bots, capable of beating humans in Go and Dota 2.

 The goal of this organization is building Artificial General Intelligence (AGI) that can perform a broad variety of tasks within a single general system. Programming bots for competitive gaming is a natural way to cultivate AGI, as it gives a set of benchmarks defining performance. The overall success can be measured by setting up a match against real humans.

 OpenAI has produced a team of bots (OpenAI Five) that competed vs professional players at TI with a limited set of heroes. According to journalist Tony Peng, AI agents will be able to compete with the full list of heroes by the end of this year or in 2019.

What makes OpenAI bots so strong?

 In early days of development of their Dota 2 agent,  OpenAI ditched any artificial rule-based restrictions and replaced them with incentive-based reinforcement learning. The multiple instances of bots self-play on 128,000 CPU cores and 256 GPUs and each day accumulate training with an equivalent of up to 180 years of game time.

 RL caused its own set of setbacks, such as endless training loops, in which bots’ skill literally plateaus. Open AI team deals with this problem by setting the bots to play vs their earlier versions 20% of the time.

 Roshan turned out to be a major obstacle for RL — the bots flatly refused to risk attacking the creep. Open AI team solved the problem by randomizing Roshan’s health and encouraging the bots to try and get Aegis when the beast was vulnerable.

This year at TI the official Open  AI Five squad lost to professional Dota 2 players. It was a little bit surprising to the community as before TI the agents of OpenAI had crushed every opponent they faced. Nevertheless, researchers delivered very competitive bots and weren’t upset by the result.

 Last week OpenAI tweeted, that current version of their AI agent achieved 80% win rate versus the model that played on-stage vs paiN Gaming.