Evaluating the Performance of Federated Learning Models in Non-Independent and Identically Distributed Data Scenarios for Mobile Applications
Keywords:
Federated Learning, Non-IID Data, Mobile Applications, Distributed Learning, Statistical Heterogeneity, Data PrivacyAbstract
Federated Learning (FL) offers a privacy-preserving approach to training machine learning models across distributed devices without centralized data aggregation. However, its performance is significantly challenged in real-world mobile environments, where data distributions are often non-independent and identically distributed (non-IID). This paper evaluates the robustness, accuracy, and convergence behavior of state-of-the-art federated learning algorithms under non-IID data scenarios typical in mobile applications. We investigate several benchmark algorithms on simulated and real-world mobile datasets, exploring their sensitivity to varying degrees of statistical heterogeneity. The findings provide insight into current algorithmic limitations and point toward future research directions that could improve FL resilience in mobile ecosystems.
References
Kairouz, Peter, H. Brendan McMahan, et al. “Advances and Open Problems in Federated Learning.” arXiv preprint arXiv:1912.04977, 2019.
McMahan, H. Brendan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. “Communication-Efficient Learning of Deep Networks from Decentralized Data.” Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, 2017, pp. 1273–1282.
Li, Tian, Anit Kumar Sahu, Virginia Smith, and Ameet Talwalkar. “Federated Optimization in Heterogeneous Networks.” arXiv preprint arXiv:1812.06127, 2018.
Karimireddy, Sai Praneeth, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. “Scaffold: Stochastic Controlled Averaging for Federated Learning.” International Conference on Machine Learning, 2020, pp. 5132–5143.
Zhao, Yue, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. “Federated Learning with Non-IID Data.” arXiv preprint arXiv:1806.00582, 2018.
Bonawitz, Keith, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. “Practical Secure Aggregation for Privacy-Preserving Machine Learning.” Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017, pp. 1175–1191.
Smith, Virginia, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S. Talwalkar. “Federated Multi-Task Learning.” Advances in Neural Information Processing Systems, vol. 30, 2017.
Hard, Andrew, Kanishka Rao, Rajiv Mathews, Swaroop Ramaswamy, Françoise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage. “Federated Learning for Mobile Keyboard Prediction.” arXiv preprint arXiv:1811.03604, 2018.
Fallah, Alireza, Aryan Mokhtari, and Asuman Ozdaglar. “Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach.” Advances in Neural Information Processing Systems, vol. 33, 2020.
Li, Xiang, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. “On the Convergence of FedAvg on Non-IID Data.” International Conference on Learning Representations, 2020.
Wang, Jialei, and Mikhail Belkin. “Beyond Distributional Assumptions: An Information-Theoretic Approach to Non-IID Data in Federated Learning.” arXiv preprint arXiv:1906.06629, 2019.
Yurochkin, Mikhail, Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Tamara Khosla, and Yasaman Khazaeni. “Bayesian Nonparametric Federated Learning of Neural Networks.” International Conference on Machine Learning, 2019, pp. 7252–7261.
Caldas, Sebastian, Peter Wu, Tian Li, Jakub Konečný, H. Brendan McMahan, Virginia Smith, and Ameet Talwalkar. “LEAF: A Benchmark for Federated Settings.” arXiv preprint arXiv:1812.01097, 2018.
Luo, Minghong, Jialiang Zhang, and Qiang Yang. “Cost-Efficient Federated Learning in Mobile Edge Computing.” IEEE Internet of Things Journal, vol. 7, no. 10, 2020, pp. 9519–9532.
Lin, Tian, Lingjing Kong, Sebastian U. Stich, and Martin Jaggi. “Ensemble Distillation for Robust Model Fusion in Federated Learning.” Advances in Neural Information Processing Systems, vol. 33, 2020.
Downloads
Published
Issue
Section
License
Copyright (c) 2022 Qiang Yang (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
