Discovering Implied Serial Order Through Model-Free and Model-Based Learning

Frontiers in Neuroscience

Greg Jensen¹  Herbert S. Terrace²  Vincent P. Ferrera² 

[1] Department of Neuroscience, Columbia University, New York, NY, United States;Department of Psychiatry, Columbia University, New York, NY, United States;Department of Psychology, Columbia University, New York, NY, United States;

关键词: reinforcement learning;    model-free learning;    model-based learning;    cognitive maps;    transitive inference;   

DOI  :  10.3389/fnins.2019.00878

来源: DOAJ

【 摘 要 】

Humans and animals can learn to order a list of items without relying on explicit spatial or temporal cues. To do so, they appear to make use of transitivity, a property of all ordered sets. Here, we summarize relevant research on the transitive inference (TI) paradigm and its relationship to learning the underlying order of an arbitrary set of items. We compare six computational models of TI performance, three of which are model-free (Q-learning, Value Transfer, and REMERGE) and three of which are model-based (RL-Elo, Sequential Monte Carlo, and Betasort). Our goal is to assess the ability of these models to produce empirically observed features of TI behavior. Model-based approaches perform better under a wider range of scenarios, but no single model explains the full scope of behaviors reported in the TI literature.

【 授权许可】

Unknown