A Survey of Clustering Federated Learning in Heterogeneous Data Scenarios

Abstract

Federated learning, as a collaborative training paradigm that preserves raw data privacy, offers an effective solution for data protection concerns. However, its practical implementation faces significant challenges due to data heterogeneity. This heterogeneity manifests as non-independent and identically distributed (non-IID) data across participating entities, resulting in degraded model performance, slower convergence rates, and training instability. While conventional federated learning approaches—including parameter averaging, knowledge distillation, and personalization techniques—offer certain advantages, their efficacy remains limited in severely heterogeneous environments. This survey systematically examines research advancements in clustered federated learning for addressing data heterogeneity challenges, encompassing fundamental principles, model architecture development, and algorithmic implementations. We provide a detailed analysis of innovative algorithms ranging from IFCA to FedGroup, and from FCL-GNN to FedAC, highlighting their technical contributions and applicable scenarios. Furthermore, we explore emerging research directions including clustering interpretability, multi-source heterogeneous information fusion, dynamic clustering mechanisms, and resource-aware optimization. Clustered federated learning effectively enhances model performance and convergence efficiency while maintaining privacy by grouping participants with similar data distributions into clusters and training specialized models for each cluster. With ongoing technological progress, clustered federated learning shows promise for achieving an optimal balance between privacy preservation and learning efficiency in critical domains such as healthcare and finance, thereby contributing to the sustainable development of artificial intelligence technologies.