Computational mechanisms of curiosity and goal-directed exploration

eLife

Johannes Passecker¹  Karl J Friston²  Tobias U Hauser³  Thomas HB FitzGerald⁴  Martin Kronbichler⁵  Philipp Schwartenbeck⁶ 

[1] Centre for Cognitive Neuroscience, University of Salzburg, Salzburg, Austria;Mortimer B. Zuckerman Mind Brain and Behavior Institute, New York, United States;Neuroscience Institute, Christian-Doppler-Klinik, Paracelsus Medical University Salzburg, Salzburg, Austria;Oxford Centre for Functional MRI of the Brain, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom;Department for Cognitive Neurobiology, Center for Brain Research, Medical University Vienna, Vienna, Austria;Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom;

关键词: exploration;    exploitation;    active learning;    active inference;    curiosity;    intrinsic motivation;   

DOI  :  10.7554/eLife.41703

来源: DOAJ

【 摘 要 】

Successful behaviour depends on the right balance between maximising reward and soliciting information about the world. Here, we show how different types of information-gain emerge when casting behaviour as surprise minimisation. We present two distinct mechanisms for goal-directed exploration that express separable profiles of active sampling to reduce uncertainty. ‘Hidden state’ exploration motivates agents to sample unambiguous observations to accurately infer the (hidden) state of the world. Conversely, ‘model parameter’ exploration, compels agents to sample outcomes associated with high uncertainty, if they are informative for their representation of the task structure. We illustrate the emergence of these types of information-gain, termed active inference and active learning, and show how these forms of exploration induce distinct patterns of ‘Bayes-optimal’ behaviour. Our findings provide a computational framework for understanding how distinct levels of uncertainty systematically affect the exploration-exploitation trade-off in decision-making.

【 授权许可】

Unknown