A Coarse-to-Fine Reconstruction Framework for Non-Lambertian Photometric Stereo

Photometric stereo aims to regress object surface normal from a set of images observed under varying illuminations. Although existing methods have achieved promising results, the irregular high-frequency detail is ignored, especially in complex and tiny surface folds. To address this problem, a coarse-to-fine reconstruction framework is proposed for non-Lambertian photometric stereo. Specifically, a coarse network is designed to roughly predict object surface normal, which learns the mapping from observed images to coarse surface normal. Then, to deal with the high-frequency information loss, we introduce a fine network to extract high-frequency information by leveraging both coarse surface normal and observation images. Meanwhile, to provide more supervision, we design a reconstruction module to reconstruct observed images from predicted surface normal and illuminations. Extensive experiments have demonstrated that the proposed method outperforms existing works and restores high-frequency detail effectively. In addition, the proposed method promotes the robustness under sparse illuminations.