FedKDC: Toward Efficient Federated Learning via Knowledge Distillation and Data Compression for Heterogeneous Devices

Federated Learning (FL) faces critical challenges in heterogeneous and resource-constrained environments, including device diversity, high communication overhead, and training delays. Therefore, we propose FedKDC, a federated learning framework that integrates knowledge distillation with data compression to jointly optimize server bandwidth, client computation resources, and compression ratios, thereby minimizing training latency. In particular, FedKDC employs a Generative Adversarial Network (GAN)-based generator to produce synthetic data for knowledge transfer across heterogeneous models without sharing raw data, mitigating privacy risks. Then, FedKDC uses a loss-driven adaptive compression mechanism to adjust the minimum compression threshold based on training stability, reducing communication volume while maintaining accuracy. In addition, we further discuss the problem of resource allocation under system constraints, and uses Particle Swarm Optimization (PSO) algorithm to solve it. Based on the three real world datasets (i.e., Fashion-MNIST, CIFAR-10, and CIFAR-100), the experimental results demonstrate that FedKDC reduces communication cost by up to 17% and training time by 8%. This shows that FedKDC is effective for large-scale heterogeneous FL deployment while maintaining the accuracy of the model.