Enhance Image-to-Point-Cloud Registration with Beltrami Flow

Image-to-point-cloud (I2P) registration is a fundamental yet challenging problem in computer vision. Despite significant advances in deep learning, I2P registration struggles with correspondence accuracy when training samples are limited. To address this challenge, we propose a Beltrami flow based I2P registration method termed Flow-I2P. From the perspective of information geometry, I2P registration can be reframed as a manifold alignment problem. Our in-depth analysis shows that Beltrami flow enhances I2P registration by improving manifold alignment quality. Building on this analysis, we introduce a Beltrami flow based cross-modality feature interaction layer, B-flow, to progressively refine manifold alignment. To reduce memory and computation demands, B-flow is then optimized into C-flow through the incorporation of feature covariance-based attention. We further enhance I2P registration performance by developing Flow-I2P, which incorporates normal features, stacked C-flow layers, and a two-stage training strategy. To evaluate the registration performance of Flow-I2P, we conduct extensive experiments on five indoor and outdoor datasets, including RGB-D V2, 7-Scenes, ScanNet, KITTI, and a self-collected dataset. Our results indicate that Flow-I2P achieves higher inlier ratio (IR) and registration recall (RR) compared to state-of-the-art methods. We conclude that Flow-I2P significantly enhances I2P registration with superior capabilities.