Neurorobotic reinforcement learning for domains with parametrical uncertainty

Frontiers in Neurorobotics

Neuroscience

Camilo Amaya¹  Axel von Arnim² 

[1] Department of Neuromorphic Computing, Fortiss-Research Institute, Munich, Bavaria, Germany;null;

关键词: domain randomization;    neuromorphic computing;    neurorobotics;    reinforcement learning;    robot control;    spiking neural networks;   

DOI  :  10.3389/fnbot.2023.1239581

 received in 2023-06-13, accepted in 2023-09-26,  发布年份 2023

来源: Frontiers

【 摘 要 】

Neuromorphic hardware paired with brain-inspired learning strategies have enormous potential for robot control. Explicitly, these advantages include low energy consumption, low latency, and adaptability. Therefore, developing and improving learning strategies, algorithms, and neuromorphic hardware integration in simulation is a key to moving the state-of-the-art forward. In this study, we used the neurorobotics platform (NRP) simulation framework to implement spiking reinforcement learning control for a robotic arm. We implemented a force-torque feedback-based classic object insertion task (“peg-in-hole”) and controlled the robot for the first time with neuromorphic hardware in the loop. We therefore provide a solution for training the system in uncertain environmental domains by using randomized simulation parameters. This leads to policies that are robust to real-world parameter variations in the target domain, filling the sim-to-real gap.To the best of our knowledge, it is the first neuromorphic implementation of the peg-in-hole task in simulation with the neuromorphic Loihi chip in the loop, and with scripted accelerated interactive training in the Neurorobotics Platform, including randomized domains.

【 授权许可】

Unknown   
Copyright © 2023 Amaya and von Arnim.

【 图 表 】

FMOLB_fmolb-2023-1252178_wc_tfx27.tif