Breakthrough in fine-grained video anomaly detection on highway: New benchmark and model

Automatic detection of anomalies is critical for surveillance video analysis, especially for highway videos. In contrast to traditional coarse-grained methods, which only focus on identifying abnormal video clips, fine-grained detection more effectively meets practical demands. Because it can further classify the specific anomaly types which helps monitoring systems issue targeted alerts. Nevertheless, existing research often overlooks the semantic information in video descriptions, which is essential for capturing the fine contextual relationship between anomalies. Therefore, it is extremely necessary to explore how to utilize the description information to enhance fine-grained video anomaly detection. To achieve this, we construct the first Highway Anomaly Dataset (HAD) containing video descriptions and propose a novel multi-modal training paradigm called Dual-Classification with Dual-Text (DCDT). It integrates coarse-grained binary classification and fine-grained multi-class classification tasks, involving both description text and label text. In DCDT, we leverage video-description alignment to increase the intra-class similarity for normal events and inter-class differences among anomalies, while adopting video-label alignment to distinguish between normal and various types of anomalies. Experimental results demonstrate that DCDT achieves superior performance on our proposed HAD benchmark, as well as on the widely-used UCF-Crime and XD-Violence datasets. Furthermore, the HAD dataset provides an innovative platform for research in video analysis and multi-modal learning.