Marrying polarization to stereo: Real-time stereo matching via polarimetric cues

Stereo matching is crucial for binocular 3D surface reconstruction. Existing stereo matching algorithms mainly rely on surface texture features to measure the correlation between target pixels and candidate pixels, which poses challenges on highly reflective and weakly textured surfaces. Previous research has demonstrated that polarization imaging can eliminate highlights and enhance surface texture features. To achieve highly synergistic collaboration between polarization fusion networks and stereo matching networks, fully leveraging polarization information in stereo matching tasks, this study constructs an end-to-end polarization fusion-based stereo matching network, PF-IGEV: 1) First, we establish the real-time stereo matching network CA-IGEV, innovatively introducing a context-aware grouped correlation computation module (CAGWC). By incorporating an explicit context-aware mechanism during correlation volume computation, the module significantly improves robustness in occluded and weakly textured regions. 2) A synthetic polarized binocular image dataset with depth ground truth is constructed using a style transfer network, effectively alleviating real-world polarized binocular data scarcity, and leveraged for supervised training of the cascaded PF-IGEV network. Experimental results demonstrate that compared to other real-time stereo matching algorithms, CA-IGEV achieves leading accuracy on the KITTI benchmarks. Notably, on the KITTI 2012 benchmark, it achieves 3-noc and 3-all error rates of 1.01% and 1.34%, respectively, with an average runtime of only 55ms. In experiments on real-world polarization binocular images, PF-IGEV demonstrates superior matching visual quality and fewer erroneous matches compared to CA-IGEV, confirming that incorporating polarization information effectively enhances the stereo matching accuracy. The source code is publicly available at https://github.com/FiredTable/Polar3D.