Multi-fractional spatial-spectral constraint for detecting sub-mosaic objects in snapshot spectral imaging

This paper presents a sub-mosaic object detection solution using multi-fractional spatial-spectral constraint (MFSC) for mosaic spectral videos. Snapshot spectral imaging technology, equipped with spectral filter array (SFA), captures mosaic spectral videos of dynamic scenes in real time and has attracted increasing attention for object detection and tracking tasks using spectral videos. However, the spatial-spectral tradeoff sampling in this imaging technology will cause vague spatial structure and incomplete spectral signatures for sub-mosaic objects smaller than the SFA's basic pattern size, making it difficult to detect and track such objects. Existing spectral detection and tracking methods fall short in delivering satisfactory results for sub-mosaic objects due to their disregard for sub-mosaic object characteristics or disruption of spatial-spectral aliasing correlations. Considering that sub-mosaic objects occur within the scope of the basic pattern of the SFA, the proposed MFSC treats the sub-mosaic object as an anomaly in the scene to detect it directly from the mosaic spectral image. MFSC extracts discriminative spatial-spectral fractional features to obtain the preliminary detection results through the mosaic multi-fractional collaborative representation (MMCR) module. For further background purification, the multi-distance spatial-spectral constraint (MDSC) module is designed to adaptively capture the spatial-spectral difference between the test pixel and its surrounding pixels in multi-distance neighborhoods, which serves as a constraint for MMCR to adjust the detection results. Experiment results on our fully-annotated sub-mosaic object video dataset demonstrate the effectiveness and robustness of the proposed method in sub-mosaic object detection and tracking tasks.