Future Internet | |
Latent Structure Matching for Knowledge Transfer in Reinforcement Learning | |
Yi Zhou1  Fenglei Yang1  | |
[1] School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China; | |
关键词: latent structure matching; reinforcement learning; transfer learning; action advice; policy improvement; mountain car; | |
DOI : 10.3390/fi12020036 | |
来源: DOAJ |
【 摘 要 】
Reinforcement learning algorithms usually require a large number of empirical samples and give rise to a slow convergence in practical applications. One solution is to introduce transfer learning: Knowledge from well-learned source tasks can be reused to reduce sample request and accelerate the learning of target tasks. However, if an unmatched source task is selected, it will slow down or even disrupt the learning procedure. Therefore, it is very important for knowledge transfer to select appropriate source tasks that have a high degree of matching with target tasks. In this paper, a novel task matching algorithm is proposed to derive the latent structures of value functions of tasks, and align the structures for similarity estimation. Through the latent structure matching, the highly-matched source tasks are selected effectively, from which knowledge is then transferred to give action advice, and improve exploration strategies of the target tasks. Experiments are conducted on the simulated navigation environment and the mountain car environment. The results illustrate the significant performance gain of the improved exploration strategy, compared with traditional
【 授权许可】
Unknown