Mitigating bias in heterogeneous federated learning via stratified client selection

Client selection strategies have become a widely adopted approach in recent years within the studies on Federated Learning (FL). This strategy aims to handle the communication efficiency problem between the server and clients. However, due to certain differences in data distribution among clients, the update variance among the selected subset of clients can be relatively large, significantly affecting the convergence speed and final accuracy of the FL task. To solve this problem, we propose a stratified client selection algorithm named FedHD to mitigate the impact of system heterogeneity on FL and simultaneously reduce the long-term bias caused by client non-independent and identically distributed (non-IID) data. First, a stratified sampling method is proposed to coordinate the selection of clients in the same round to mitigate system heterogeneity, and multi-armed bandits (MAB) are used to eliminate the bias caused by stratified sampling. Secondly, a novel client selection scheme is proposed to explore combinations between clients, which aims to approximate the overall datadistribution of sampled clients to the global data distribution while reducing the variance of client selections, thereby mitigating long-term bias. In a heterogeneous environment, we demonstrated the effectiveness of the client selection scheme. Experimental results confirm that our method stabilizes the update of the global model, encourages a more uniform (i.e., fair) performance among clients, and is compatible with popular FL algorithms.