Hypernetworks-Based Hierarchical Federated Learning on Hybrid Non-IID Datasets for Digital Twin in Industrial IoT

Digital twin technology deployed in the industrial Internet of Things (IoT) will be unavailable because of data heterogeneity and data islands. As a machine learning approach for distributed architectures, federated learning generates a global model for edge devices by only transferring model parameters, and it ensures that the private data of each device is not leaked. This is naturally suitable for solving this problem. Because devices' data cannot be exchanged with other devices, one major challenge is learning an effective model among devices with heterogeneous data. Motivated by this issue, we propose hierarchical federated learning using a hypernetwork (HN) algorithm to alleviate the influence of non-IID data among the devices, including label distribution skew, quantity skew, and feature distribution skew. The lower layer of our method uses hypernetworks to generate the parameters of the local model for the devices, while its upper layer updates hypernetworks by aggregating their model parameters. It decouples the number of parameters transmitted by the upper and lower layers, and it achieves good accuracy while improving communication efficiency and reducing computation costs.