A distributional code for value in dopamine-based reinforcement learning | |
Article | |
关键词: REWARD; GRADIENTS; CIRCUITRY; RESPONSES; NEURONS; SITES; D-1; | |
DOI : 10.1038/s41586-019-1924-6 | |
来源: SCIE |
【 摘 要 】
Since its introduction, the reward prediction error theory of dopamine has explained a wealth of empirical phenomena, providing a unifying framework for understanding the representation of reward and value in the brain(1-3). According to the now canonical theory, reward predictions are represented as a single scalar quantity, which supports learning about the expectation, or mean, of stochastic outcomes. Here we propose an account of dopamine-based reinforcement learning inspired by recent artificial intelligence research on distributional reinforcement learning(4-6). We hypothesized that the brain represents possible future rewards not as a single mean, but instead as a probability distribution, effectively representing multiple future outcomes simultaneously and in parallel. This idea implies a set of empirical predictions, which we tested using single-unit recordings from mouse ventral tegmental area. Our findings provide strong evidence for a neural realization of distributional reinforcement learning.
【 授权许可】
Free