Encoding functional brain interactions from computational visual features

Encoding and decoding in functional magnetic resonance imaging (fMRI) are receiving increasing interest recently. In this paper, we propose an fMRI encoding model to predict the human brain's responses to free viewing of video clips. The novelty of our study is that we represent the stimuli using a variety of representative visual features, which can describe the global color distribution, local shape and spatial information contained in videos and they have been proven to be effective by computer vision studies. Our experimental results demonstrate that brain network responses during free viewing of videos can be robustly and accurately predicted by using visual features across subjects. Our study suggests the feasibility to exploring cognitive neuroscience studies by computational image/video analysis and provides a novel concept of using the brain encoding as a test-bed for evaluating visual feature extraction.