Bayesian decomposition of multi-modal dynamical systems for reinforcement learning

NEUROCOMPUTING

卷:416

Article

Kaiser, Markus^1,2  Otte, Clemens¹  Runkler, Thomas A.^1,2  Ek, Carl Henrik³ 

[1] Siemens AG, Otto Hahn Ring 6, D-81739 Munich, Germany

[2] Tech Univ Munich, Boltzmannstrae 3, D-85748 Garching, Germany

[3] Univ Bristol, MVB, Woodland Rd, Clifton BS8 1UB, England

关键词: Bayesian machine learning;    Gaussian processes;    Hierarchical gaussian processes;    Reinforcement learning;    Model-based reinforcement learning;    Stochastic policy search;    Data-efficiency;   

DOI  :  10.1016/j.neucom.2019.12.132

来源: Elsevier

【 摘 要 】

In this paper, we present a model-based reinforcement learning system where the transition model is treated in a Bayesian manner. The approach naturally lends itself to exploit expert knowledge by introducing priors to impose structure on the underlying learning task. The additional information introduced to the system means that we can learn from small amounts of data, recover an interpretable model and, importantly, provide predictions with an associated uncertainty. To show the benefits of the approach, we use a challenging data set where the dynamics of the underlying system exhibit both operational phase shifts and heteroscedastic noise. Comparing our model to NFQ and BNN+LV, we show how our approach yields human-interpretable insight about the underlying dynamics while also increasing data-efficiency. (C) 2020 Published by Elsevier B.V.

【 授权许可】

Free