SDBF-Net: Semantic and disparity bidirectional fusion network for 3D semantic detection on incidental satellite images

In this paper, we propose a conceptually simple, flexible, and general framework for the semantic stereo task on incidental satellite images. Our method efficiently detects the objects in an incidental satellite image for generating a high-quality segmentation map, and more accurately match the left-right incidental satellite images for obtaining a more accurate disparity map at the same time. The method, called semantic and disparity bidirectional fusion network (SDBF-Net), consists of three main modules: the Semantic Segmentation Module (SSM), the Stereo Matching Module (SMM), and the Fusion Module (FM). The semantic segmentation module takes advantage of the capacity of global context information by extending the receptive field to produce the initial segmentation map. The stereo matching module applies the 3D convolutional operation to regularize the feature map of left-right images to generate the initial disparity map. The fusion module fuses the initial segmentation and disparity map to obtain the refined segmentation and disparity map. Extensive quantitative and qualitative evaluations on the US3D dataset demonstrate the superiority of our proposed SDBF-Net approach, which outperforms state-of-the-art semantic stereo approaches significantly.