PGSC: A gradient sparsification communication optimization criterion for nonequilibrium thermodynamics

Gradient compression can reduce communication overhead. However, current static sparsity techniques may disturb gradient dynamics, resulting in unstable model convergence and reduced feature discriminative ability, whereas transmitting the complete gradient leads to high costs. To address this issue, inspired by nonequilibrium thermodynamics, this paper proposes a Physics-guided Gradient Sparsification Criterion (PGSC). Specifically, we formulate a continuous field equation based on the gradient magnitude distribution, deriving an adaptive decay rule for the sparsification threshold during the training phase. We then dynamically adjust the sparsification threshold according to this rule, effectively addressing the complexity of multimodal features and ensuring consistent information transmission. Our method achieves adaptive co-optimization of gradient compression and model accuracy by establishing a dynamic equilibrium mechanism between gradient dissipation and information entropy. This approach ensures stable convergence rates while preserving the gradient structure of multi-scale features. Extensive experiments on public datasets, including CIFAR-10, MNIST, and FLIR_ADAS_v2, demonstrate significant advantages over competitors such as TopK and quantization compression, while also reducing communication costs.