SRPKD-IGhostNet: A Highly Precise and Lightweight Fault Diagnosis Framework Based on Sample-Related Progressive Knowledge Distillation and Multiscale Fusion Convolution

Deep learning-based fault diagnosis methods have been successfully deployed in a range of engineering applications. However, with the increasing complexity of network structures, it is difficult to perform deep learning training on systems with limited computational resources. To address this challenge, this study proposes a highly precise and lightweight fault diagnosis framework based on sample-related progressive knowledge distillation (SRPKD) and multiscale fusion convolution, called SRPKD-IGhostNet. The SRPKD is proposed to address the limitation that the teacher model is unable to adequately transfer the knowledge in traditional knowledge distillation (KD). The method enables the student model to acquire more accurate and useful knowledge by transferring the attention-based sample-related knowledge in the middle layer of the teacher model and gradually reducing the erroneous output of the distillation process. Additionally, IGhostNet is designed as the student model in the SRPKD, employing a multiscale fusion convolution block to replace traditional convolution, thereby reducing computational cost while maintaining high diagnostic accuracy. Extensive experiments are conducted on mechanical fault simulation platforms, such as MCDS, with limited computational resources. The results demonstrate that SRPKD-IGhostNet outperforms existing state-of-the-art lightweight fault diagnosis frameworks in accuracy and computational complexity, highlighting its significant potential for industrial applications.