FGBVD-KD: Frequency-Guided bias-variance decomposition knowledge distillation for fracture detection

Fracture detection in medical imaging is a critical diagnostic task for identifying bone injuries, where deep learning achieves high accuracy using large-scale models. However, their computational cost hinders clinical deployment. Knowledge distillation (KD) addresses this challenge by compressing the high-performance teacher to the lightweight student. Nevertheless, current feature distillation methods prioritize semantic feature imitation, neglecting critical structural details like bone morphology abnormalities and biomechanical disruptions specific to bone injuries. This study identifies a fundamental limitation: prevailing distillation paradigms compel students to fit teachers’ feature means ( bias learning ) while neglecting variance alignment, which is essential for structural details. We propose FGBVD-KD, a Frequency-Guided Bias-Variance Decomposition framework that decouples distillation loss into bias (mean-fitting) and variance (distribution-alignment) terms. Leveraging the finding that semantic features concentrate in low-frequency component while structural features reside in high-frequency domains, we introduce two novel modules: 1) Feature Frequency Expectation Estimation (FFEE), a lightweight module estimating prediction expectations to bridge bias-variance computation; 2) Frequency-domain Information Decomposition (FID), splitting teacher features into low-frequency and high-frequency components. FGBVD-KD guides students to utilize low-frequency component for semantic bias learning and high-frequency component for structural variance alignment. Evaluations on two fracture detection benchmarks demonstrate state-of-the-art performance across diverse teacher-student pairs. Importantly, our framework achieves this superior accuracy while introducing negligible parameters and computational overhead during inference, maintaining identical model size and FLOPs as the student. This makes it particularly suitable for resource-constrained clinical environments. Ablation studies confirm the effectiveness of both FFEE and FID modules. Our code is available at: https://github.com/1123026073/FGBVD-KD .