Comparative Performance Evaluation of Six Federated Learning Frameworks Under Locked FedAvg: Native SDKs and a Shared Reference Harness for Edge-Oriented 6G Applications
Vasileios D. Batsios, Constantinos T. AngelisFederated learning (FL) enables privacy-preserving collaborative training at the network edge, a core capability envisioned for sixth-generation (6G) wireless systems. While surveys and scale-oriented benchmarks advance FL methodology, documented, head-to-head comparisons of mainstream Python frameworks under identical FedAvg settings remain scarce. We benchmark six frameworks—Flower, TensorFlow Federated (TFF), FedML, NVIDIA FLARE, OpenFL, and PySyft—distinguishing two native SDK integrations (Flower, TFF) from four runs of a shared PyTorch FedAvg reference harness (FedML, NVIDIA FLARE, OpenFL, PySyft) in a controlled two-phase study on a Proxmox virtualized testbed with containerized runners, formalize the FedAvg objective and communication-cost model, and position our contribution against prior surveys, scale benchmarks, and single-framework documentation. Each framework–dataset pair is repeated over five IID partitions (random seeds 42–46); we report round-10 mean ± standard deviation for accuracy, wall time, and simulated communication volume. Phase 1 (MNIST) confirms protocol fairness (99.22±0.07–99.29±0.06% accuracy) with moderate wall-time spread; Phase 2 (CIFAR-10) exposes stack-dependent accuracy gaps (TFF 71.16±0.23% vs. ≈68% for PyTorch runners). We report per-round accuracy and loss curves with variability bands, wall-time comparisons, and simulated parameter traffic for all six frameworks across nine figures. The experimental protocol, model topology, and hyperparameters are specified in full; per-round JSON metrics and global model checkpoints are published. The study provides a documented baseline for 6G edge framework selection and for follow-on network-constrained and security experiments.