Enhancing weakly supervised video anomaly detection via prototype-driven pseudo-labeling

Video anomaly detection is a critical task in video analysis, focusing on identifying unexpected events that deviate from regular patterns. This paper addresses weakly supervised video anomaly detection (WS-VAD), which leverages video-level annotations to detect abnormal frames within a sequence. The commonly used approach in WS-VAD is multiple instance learning (MIL) with ranking loss, which selects high-scoring segments as targets; however, noisy initial predictions can mislead the optimization and degrade performance. To overcome this, we propose a prototype-driven approach. Assuming that normal segments share a common distribution, we first construct a normal prototype from clean normal videos and then extract abnormal segments from noisy abnormal videos to build an abnormal prototype. By designing a loss function to enlarge the distance between those prototypes, we enhance the model’s ability to distinguish between normal and abnormal instances. Additionally, we introduce a pseudo-label generation strategy, where segment-level pseudo-labels are assigned based on their similarity to normal and abnormal prototypes, improving the accuracy of segment selection. Furthermore, an adaptive local-context temporal modeling approach is proposed to capture anomalies with diverse temporal characteristics. Experiments show that our method outperforms existing approaches, achieving 86.48% AUC on UCF-Crime and 82.48% AP on XD-Violence. The source code is available at https://github.com/EvansLR/PL-VAD.