11-02-2020

A research paper “Model-Based Reinforcement Learning for Atari” has been chosen for a spotlight presentation during the upcoming International Conference on Learning Representation. Only 5% of the papers submitted receive this distinction. The paper is a result of collaboration of MIMUW, Institute of Mathematics of the Polish Academy of Sciences, University of Illinois at Urbana–Champaign, Google Brain and deepsense.ai.

The high-level goal described in the research is how to enable computers to learn useful behaviors through interaction with an environment. This is akin, and inspired, by human learning. For example a newborn baby learns motoric skills via a rather lengthy process. It starts from seemingly uncoordinated movements, which gradually get refined into more and more coherent deliberate motions.

Acting deliberately requires imagination. The role of imagination is hard to overestimate, it enables planning and avoiding potentially harmful actions, without really executing them. On the formal level, imagination is a predictive module. In the easiest form, given the state of an environment and a desired action it outputs the next state after a short amount of time. Clearly, this can be looped to produce  longer term predictions.

“In our work we endowed a learning system with a predictive model and measured how much it improves learning. We discovered that it requires less data to learn skills, compared to standard reinforcement learning algorithms without imagination” – says Błażej Osiński, a PhD student at MIMUW, worker in deepsense.ai and one of the leading authors of the publication.


"Spotlight" is a short oral presentation for exceptionally good papers, only 5% of submitted papers were distinguished this way. This comes amidst our work gaining critical attention in the AI research community. The preprint of the article has been cited more than 50 times in just 10 months.

This year’s ICLR conference, will be held in Addis Ababa, Ethiopia, and is hoped to encourage researchers from Africa and the Middle East to take part. The conference is among the most important scientific events related to deep learning in the world.