Environmental curriculum learning for efficiently achieving superhuman play in games

reinforcement learning;curriculum learning;sample efficiency;StarCraft II;Pommerman;Monte Carlo tree search

Sun, Ray ; Peng ; Jian

关键词: reinforcement learning;    curriculum learning;    sample efficiency;    StarCraft II;    Pommerman;    Monte Carlo tree search;   

Others  :  https://www.ideals.illinois.edu/bitstream/handle/2142/108014/SUN-THESIS-2020.pdf?sequence=1&isAllowed=y

美国|英语

来源: The Illinois Digital Environment for Access to Learning and Scholarship

【 摘 要 】

Reinforcement learning has made large strides in training agents to play games, including complex ones such as arcade game Pommerman and real-time strategy game StarCraft II. To allow agents to grasp the many concepts in these games, curriculum learning has been used to teach agents multiple skills over time. We present Environmental Curriculum Learning, a new technique for creating a curriculum of environment versions for an agent to learn in sequence. By adding helpful features to the state and action spaces, and then removing these helpers over the course of training, agents can focus on the fundamentals of a game one at a time. Our experiments in Pommerman illustrate the design principles of ECL, and our experiments in StarCraft II show that ECL produces agents with far better final performance than without it, when using the same training algorithm. Our StarCraft II ECL agent exceeds previous score records in a StarCraft II minigame, including human records, while taking far less training time to do so than previous approaches.