Modeling optical imaging pipeline and learning contrastive-based representation for hybrid-corrupted image restoration

Addressing optical corruptions presents a significant challenge due to the inherent variability introduced during both manufacturing and imaging processes. Existing approaches often struggle to handle hybrid corruptions that appear in real-world situations. To address this issue, we propose a versatile computational imaging system aimed at approximating and correcting real-world corruptions. Our method meticulously considers external camera perturbations and internal optical deviations, improving the realism of our system and post-processing pipeline. We generate synthetic data pairs through a well-crafted pipeline, creating a dedicated training dataset without the need for extensive real paired data. For optical correction, we introduce Opticformer, a Transformer-based model adept at adaptively focusing on critical regions and addressing spatially varying corruptions. To enhance its capabilities, we incorporate a novel contrastive-based representation learning method, enabling Opticformer to generate multi-scale refined representations without prior knowledge. Extensive experiments validate our pipeline’s ability to simulate corruptions, closely approximating real-world images. Our proposed correction method successfully mitigates spatially variant corruptions, surpassing peer restoration methods and achieving state-of-the-art performance.