Semantic-based saccadic scanpath prediction for autism spectrum disorder

Saccadic scanpath prediction has become an active area of research within visual attention modeling in recent years. However, most existing methods focus on predicting scanpaths for typically developing individuals, with a limited investigation into more complex applications, such as attention modeling for atypical populations. In this study, we address this gap by focusing on a representative atypical population, namely individuals with autism spectrum disorder (ASD). The complex and unknown semantic biases in scanpaths within such populations pose significant challenges for accurate scanpath prediction. To tackle these challenges, we propose a novel scanpath prediction framework that captures potential semantic biases effectively. Our method employs a two-branch structure for fixation semantics encoding, which extracts global spatial descriptors to represent spatial cues and semantic maps for ASD-specific semantic biases, respectively. Based on the historical semantic information, we dynamically update the difference of semantics (DoS) map to predict the sequence of fixations. Given the limited availability of ASD-specific saliency maps, we leverage the n-reference transfer learning technique during the pre-training stage. In the experiment, we constructed two datasets for ASD scanpath prediction. Extensive experimental evaluations demonstrate that our method consistently outperforms state-of-the-art approaches across multiple metrics and datasets. This framework not only advances the understanding of common characteristics in ASD but also holds significant potential for research into attention-related specific diseases.