2019 Conference on Cognitive Computational Neuroscience

13-16 September 2019    Berlin, Germany

Technical Program

Paper Detail

Paper: PS-2A.57
Session: Poster Session 2A
Location: H Lichthof
Session Time: Sunday, September 15, 17:15 - 20:15
Presentation Time:Sunday, September 15, 17:15 - 20:15
Presentation: Poster
Publication: 2019 Conference on Cognitive Computational Neuroscience, 13-16 September 2019, Berlin, Germany
Paper Title: Enabling naturalistic neuroscience through behavior mining: Analysis of long-term human brain and video recordings
Manuscript:  Click here to view manuscript
License: Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
DOI: https://doi.org/10.32470/CCN.2019.1104-0
Authors: Satpreet H. Singh, Steven M. Peterson, Rajesh P. N. Rao, Bingni W. Brunton, University of Washington, United States
Abstract: Much of our understanding in human neuroscience has been informed by data collected in pre-designed and well-controlled experimental tasks, where timings of cues, stimuli, and behavioral responses are known precisely. Recent technological advances have enabled us to study longer and increasingly naturalistic brain recordings, giving rise to a new paradigm named "naturalistic neuroscience" where neural computations associated with spontaneous behaviors are studied. Analyzing such unstructured, long-term, and multi-modal data with no a priori experimental design remains very challenging. Here we present an automated approach for analysing naturalistic datasets using behavior mining. Our analysis pipeline robustly uncovers and annotates instances of human upper-limb movements in long-term naturalistic behavior data (≈18 million video frames per patient) using algorithms from computer-vision, time-series segmentation, and string pattern-matching. We analyze simultaneously recorded human electrocorticography (ECoG) brain recordings to uncover neural correlates associated with these naturalistic events, and show that they corroborate prior results from traditional controlled experiments. We also demonstrate the efficacy of our approach as a source of training data for brain-computer interface decoders.