‘Proactive’ use of cue-context congruence for building reinforcement learning’s reward function

BMC Neuroscience

Research Article

Judit Zsuga¹  Klara Biro¹  Csaba Papp¹  Gabor Tajti¹  Bela Juhasz²  Rudolf Gesztelyi³  Magdolna Emma Szilasi³ 

[1] Department of Health Systems Management and Quality Management for Health Care, Faculty of Public Health, University of Debrecen, Nagyerdei krt. 98, 4032, Debrecen, Hungary;Department of Pharmacology and Pharmacotherapy, Faculty of Medicine, University of Debrecen, Nagyerdei krt. 98, 4032, Debrecen, Hungary;Department of Pharmacology, Faculty of Pharmacy, University of Debrecen, Nagyerdei krt. 98, 4032, Debrecen, Hungary;

关键词: Model-based reinforcement learning;    Proactive brain;    Bellman equation;    Reward function;    Policy function;    Cue-context congruence;   

DOI  :  10.1186/s12868-016-0302-7

 received in 2015-11-12, accepted in 2016-10-14,  发布年份 2016

来源: Springer

【 摘 要 】

BackgroundReinforcement learning is a fundamental form of learning that may be formalized using the Bellman equation. Accordingly an agent determines the state value as the sum of immediate reward and of the discounted value of future states. Thus the value of state is determined by agent related attributes (action set, policy, discount factor) and the agent’s knowledge of the environment embodied by the reward function and hidden environmental factors given by the transition probability. The central objective of reinforcement learning is to solve these two functions outside the agent’s control either using, or not using a model.ResultsIn the present paper, using the proactive model of reinforcement learning we offer insight on how the brain creates simplified representations of the environment, and how these representations are organized to support the identification of relevant stimuli and action. Furthermore, we identify neurobiological correlates of our model by suggesting that the reward and policy functions, attributes of the Bellman equitation, are built by the orbitofrontal cortex (OFC) and the anterior cingulate cortex (ACC), respectively.ConclusionsBased on this we propose that the OFC assesses cue-context congruence to activate the most context frame. Furthermore given the bidirectional neuroanatomical link between the OFC and model-free structures, we suggest that model-based input is incorporated into the reward prediction error (RPE) signal, and conversely RPE signal may be used to update the reward-related information of context frames and the policy underlying action selection in the OFC and ACC, respectively. Furthermore clinical implications for cognitive behavioral interventions are discussed.

【 授权许可】

CC BY   
© The Author(s) 2016