Cross-modal zero-sample diagnosis framework utilizing non-contact sensing data fusion

Gearboxes, fundamental components in the domains of manufacturing, transportation, and aerospace apparatus, are highly susceptible to impairments. The emerging technique of non-contact sensing measurement holds promise as a valuable tool for real-time monitoring and diagnosis of gearboxes, especially in challenging, dynamic environments. However, historical non-contact diagnostic techniques often face difficulties due to load-variation-induced interference, which leads to inconsistent diagnostic results. Moreover, acquiring sufficient non-contact sensing data for specific rare or severe gearbox faults can be exceedingly challenging and, at times, unattainable. This data scarcity limits the application of existing supervised and semi-supervised non-contact diagnostic methods. To address these limitations, this paper develops a novel diagnostic model, called Non-Contact Sensing Data Fusion Driven Cross-Modal Zero-sample Diagnostic Network. Specifically, we first develop a novel central fusion module to facilitate feature-level fusion at both global and local scales utilizing infrared thermal and acoustic data, thereby establishing a rich and comprehensive fault representation. Following that, we construct an end-to-end cross-modal zero-sample diagnostic architecture to mine and fuse complementary fault information. This approach enhances zero-sample diagnostic performance through efficient information utilization and cross-modal feature fusion. Afterward, we adopt a cross-modal self-enhancing fusion strategy during the training phase, which improves the information fusion performance of intra-class features from various modalities. This strategy effectively reduces misclassification risks and augments the robustness of the proposed zero-sample diagnostic network against interference from load variations. Comprehensive experimental results confirm that our proposed NCFZD sets the new state-of-the-art in multiple non-contact, zero-sample diagnostic scenarios, reaching HM scores of 86.06 %, 81.53 %, and 62.56 % respectively. By incorporating the information fusion theory, this model advances gearbox diagnostics in non-contact sensing and contributes to the broader field of information fusion theory by demonstrating its practical application in real-world problem-solving scenarios.