Modelling choice and reaction time during arbitrary visuomotor learning through the coordination of adaptive working memory andreinforcement learning

Frontiers in Behavioral Neuroscience

Mehdi eKhamassi¹  Guillaume eViejo¹  Benoît eGirard¹  Andrea eBrovelli² 

[1] CNRS, UMR 7222, ISIR, F-75005;Institut des Neurosciences de la Timone (INT), UMR 7289, CNRS -Aix Marseille Université;Sorbonne Université, UPMC, Univ Paris 06, UMR 7222, ISIR, F-75005;

关键词: Behavior;    decision-making;    reinforcement learning;    working memory;    reaction times;    instrumental learning;   

DOI  :  10.3389/fnbeh.2015.00225

来源: DOAJ

【 摘 要 】

Current learning theory provides a comprehensive description of how we and other animalslearn, and places behavioral flexibility and automaticity at heart of adaptive behaviors. However, the computations supporting the interactions between goal-directed and habitual decision-making systems are still poorly understood. Previous functional magnetic resonance imaging(fMRI) results suggest that the brain hosts complementary computations that may differentiallysupport goal-directed and habitual processes in the form of a dynamical interplay rather than aserial recruitment of strategies. To better elucidate the computations underlying flexible behavior, we develop a dual-system computational model which can predict both performance (i.e.,participants’ choices) and modulations in reaction times during learning of a stimulus-response association task. The habitual system is modelled with a simple Q-learning algorithm (QL).For the goal-directed system, we propose a new Bayesian Working Memory (BWM) modelwhich searches for information in the history of previous trials in order to minimize Shannonentropy. We propose a model for QL and BWM coordination such that the expensive memorymanipulation is under control of, among others, the level of convergence of the habitual learning.We test the ability of QL or BWM alone to explain human behavior, and compare them with theperformance of model combinations, to highlight the need for such combinations to explainbehavior. Two of the tested combination models are derived from the literature, the latter beingour new proposal. In conclusion, all subjects were better explained by model combinations, andthe majority of them by our new coordination proposal.

【 授权许可】

Unknown