| Paper: | PS-1A.32 | ||
| Session: | Poster Session 1A | ||
| Location: | H Lichthof | ||
| Session Time: | Saturday, September 14, 16:30 - 19:30 | ||
| Presentation Time: | Saturday, September 14, 16:30 - 19:30 | ||
| Presentation: | Poster | ||
| Publication: | 2019 Conference on Cognitive Computational Neuroscience, 13-16 September 2019, Berlin, Germany | ||
| Paper Title: | Computational advantages of dopaminergic states for decision making | ||
| Manuscript: | Click here to view manuscript | ||
| License: |  This work is licensed under a Creative Commons Attribution 3.0 Unported License. |
||
| DOI: | https://doi.org/10.32470/CCN.2019.1390-0 | ||
| Authors: | Alana Jaskir, Michael Frank, Brown University, United States | ||
| Abstract: | Dopamine's (DA) role in the striatal direct (D1) and indirect (D2) pathways suggests a more complex system than that captured by standard reinforcement learning (RL) models. The Opponent Actor Learning (OpAL) model (Collins & Frank, 2014) presented a more biologically plausible and interactive account, incorporating interactive incentive motivation and learning effects of dopamine in one dual-actor framework. In OpAL, DA modulates not only learning but the influence of each actor on decision making, where the two actors specialize in encoding the benefits and costs of actions (D1 and D2 pathways, respectively). While OpAL accounts for a wide range of DA effects on learning and choice, formal analysis of the normative advantage of allowing the motivational state (the level of dopamine at choice) to be optimized is still needed. We present simulations which suggest a computational benefit to high motivational states in "rich" environments where all action options have high probability of reward; conversely, lower motivational states have computational benefit in "lean" environments. We show how online modulation of motivational states according to the environment value or the inference about the appropriate latent state of the environment confers a benefit beyond that afforded by classic RL in learning and risk paradigms. These simulations offer a clue as to the normative function of the biology of RL that differs from the standard model-free RL algorithms in computer science. | ||