2019 Conference on Cognitive Computational Neuroscience

13-16 September 2019    Berlin, Germany

Technical Program

Paper Detail

Paper: PS-2A.2
Session: Poster Session 2A
Location: H Lichthof
Session Time: Sunday, September 15, 17:15 - 20:15
Presentation Time:Sunday, September 15, 17:15 - 20:15
Presentation: Poster
Publication: 2019 Conference on Cognitive Computational Neuroscience, 13-16 September 2019, Berlin, Germany
Paper Title: Deep neural networks can predict human behavior in arcade games
Manuscript:  Click here to view manuscript
License: Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
DOI: https://doi.org/10.32470/CCN.2019.1043-0
Authors: Holger Mohr, Technische Universität Dresden, Germany; Radoslaw M. Cichy, Freie Universität Berlin, Germany; Hannes Ruge, Technische Universität Dresden, Germany
Abstract: In a standard experimental paradigm typically used in cognitive neuroscience, time is discretized into distinct events and the presented stimulus material is sampled from a low number of categories. While this approach allows conducting highly controlled experiments, its ecological validity is limited, as in real life the human brain has to operate on a continuous time scale and also has to process complex stimuli that typically cannot be assigned to a low-dimensional stimulus space. The encoding model approach has been introduced to address these issues by using high-dimensional sets of stimulus features as a means to analyze neuroimaging data from complex and time-continuous tasks. Recently, activations from deep neural networks (DNNs) were proposed to serve as features in the encoding model approach. However, it has been argued that such DNN-based features might be uninformative for human neuroimaging data, as the behavior of a trained DNN does not necessarily have to resemble human behavior on a given task. Here, we present preliminary evidence (N = 1) that DNN activations from the top network layer can predict human behavior with high fidelity in three different Atari 2600 arcade games based on a linear model. These findings clear the way for extending this type of analysis to neuroimaging data, testing whether DNN activations extracted from hidden layers explain variance in the fMRI signal of task-related brain regions.