Federated learning based on partial label masking weighted distillation

Federated learning, an emerging paradigm in distributed machine learning, has garnered significant attention for its ability to collaboratively train a global model without compromising local data privacy. However, the inherent challenge of data heterogeneity in FL leads to severe forgetting issues, resulting in substantial degradation of model performance. Addressing this issue is crucial for the advancement of FL. To mitigate the risk of catastrophic forgetting, we propose a federated learning algorithm based on partial label masking weighted distillation (FedPLD). This algorithm tackles the forgetting challenge posed by data heterogeneity by distilling knowledge from a few classes outside the local data distribution and enhancing teacher model generation through global model weighting. Experimental results demonstrate that our approach achieves superior model performance under various data heterogeneity conditions across multiple datasets without incurring additional communication overhead.