Benchmarking Client Selection Methods in Non-IID Federated Learning Scenarios

Abstract

Federated learning (FL) is a machine learning technique aimed at collectively acquiring a shared prediction model. When applying FL, the diversity of client data distributions and hardware configurations is a challenge that must be handled. Consequently, randomly sampling clients for training a FL model may not take advantage of the local updates, leading to poor performance of the global model. In this paper, we present a benchmark study in which we evaluate different strategies for worker selection in the context of FL. These strategies can be applied in challenging cases where data exhibits Non-Independent and Identically Distributed (N-IID) characteristics across clients. We select a set of well-known strategies introduced in the literature and performed extensive evaluations according to various scenarios. The obtained results show that there is no fit for all methods for client selection, and the client selection must be selected based on specific criteria.