Adaptive Sparse Ternary Compression with Dynamic Gradient Thresholding for Communication-Efficient Federated Learning

Nithyaniranjana Murthy Chittaiah; S. H. Manjula

doi:10.48084/etasr.18548

Authors

Nithyaniranjana Murthy Chittaiah Department of Computer Science and Engineering, University Visvesvaraya College of Engineering (UVCE), Bangalore University, Bangalore, India
S. H. Manjula Department of Computer Science and Engineering, University Visvesvaraya College of Engineering (UVCE), Bangalore University, Bangalore, India

Volume: 16 | Issue: 3 | Pages: 35281-35286 | June 2026 | https://doi.org/10.48084/etasr.18548

Received: 5 March 2026 | Revised: 26 March 2026 and 30 March 2026 | Accepted: 1 April 2026 | Online: 6 June 2026

Corresponding author: Nithyaniranjana Murthy Chittaiah

Abstract

Federated Learning (FL) supports collaborative model training while not exchanging raw data, but suffers from scalability issues due to a significant communication overhead when updating models too frequently. Sparse Ternary Compression (STC) addresses this by both gradient sparsification and ternarization; however, fixed sparsification thresholds do not adapt to gradually changing gradient distributions during training. This study introduces a dynamic gradient sparsification method, called Adaptive-STC, which learns optimized sparsification thresholds for each communication round by using statistics from local gradients. The proposed strategy provides an adaptive mechanism to control sparsity during training and results in better communication efficiency while maintaining the model's accuracy. Experimental results on both CIFAR-10 and MedMNIST using a lightweight version of VGG11 show that Adaptive-STC reduces communication cost by up to 18% compared to the dense FedAvg and fixed-threshold STC, while incurring less than 0.3% performance degradation. These findings demonstrate the value of adaptive thresholding in data-efficient FL.

Keywords:

federated learning, communication efficiency, gradient compression, sparse ternary compression, adaptive thresholding

References

Q. Yang, Y. Liu, T. Chen, and Y. Tong, "Federated Machine Learning: Concept and Applications," ACM Transactions on Intelligent Systems and Technology, vol. 10, no. 2, pp. 1–19, Mar. 2019.

S. Zhang et al., "Federated Learning in Intelligent Transportation Systems: Recent Applications and Open Problems," IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 5, pp. 3259–3285, May 2024.

S. Han et al., "Practical and Robust Federated Learning With Highly Scalable Regression Training," IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 10, pp. 13801–13815, Oct. 2024.

A. Alsarhan, M. Barhoush, B. Khassawneh, M. Al-Essa, M. Aljaidi, and Q. Al-Na’amneh, "Deep Learning Utilization for DDoS Attack Detection with Federated Learning: A Case Study on the CICDDoS2019 Dataset," Engineering, Technology & Applied Science Research, vol. 16, no. 1, pp. 31203–31208, Feb. 2026.

H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, "Communication-Efficient Learning of Deep Networks from Decentralized Data." arXiv, Jan. 26, 2023.

M. R. A. Berkani et al., "Advances in Federated Learning: Applications and Challenges in Smart Building Environments and Beyond," Computers, vol. 14, no. 4, Mar. 2025, Art. no. 124.

D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. Vojnovic, "QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding." arXiv, Oct. 7, 2016.

C. Park and N. Lee, "S3 GD-MV: Sparse-SignSGD with Majority Vote for Communication-Efficient Distributed Learning," in 2023 IEEE International Symposium on Information Theory (ISIT), June 2023, pp. 2266–2271.

J. Fei, C. Y. Ho, A. N. Sahu, M. Canini, and A. Sapio, "Efficient sparse collective communication and its application to accelerate distributed deep learning," in Proceedings of the 2021 ACM SIGCOMM 2021 Conference, Aug. 2021, pp. 676–691.

A. F. Aji and K. Heafield, "Sparse Communication for Distributed Gradient Descent," in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017, pp. 440–445.

H. Sun et al., "Sparse Gradient Compression for Distributed SGD," in Database Systems for Advanced Applications, vol. 11447, G. Li, J. Yang, J. Gama, J. Natwichai, and Y. Tong, Eds. Springer International Publishing, 2019, pp. 139–155.

J. Wangni, J. Wang, J. Liu, and T. Zhang, "Gradient Sparsification for Communication-Efficient Distributed Optimization." arXiv, Oct. 26, 2017.

J. Xu, W. Du, Y. Jin, W. He, and R. Cheng, "Ternary Compression for Communication-Efficient Federated Learning," IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 3, pp. 1162–1176, Mar. 2022.

T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, "Federated Optimization in Heterogeneous Networks." arXiv, Apr. 21, 2020.

Y. Zheng, S. Lai, Y. Liu, X. Yuan, X. Yi, and C. Wang, "Aggregation Service for Federated Learning: An Efficient, Secure, and More Resilient Realization," IEEE Transactions on Dependable and Secure Computing, vol. 20, no. 2, pp. 988–1001, Mar. 2023.

S. Wang et al., "Adaptive Federated Learning in Resource Constrained Edge Computing Systems," IEEE Journal on Selected Areas in Communications, vol. 37, no. 6, pp. 1205–1221, June 2019.

M. Lan, Q. Ling, S. Xiao, and W. Zhang, "Quantization Bits Allocation for Wireless Federated Learning," IEEE Transactions on Wireless Communications, vol. 22, no. 11, pp. 8336–8351, Nov. 2023.

D. Yin, Y. Chen, K. Ramchandran, and P. Bartlett, "Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates." arXiv, Mar. 5, 2018.

A. Koloskova, S. U. Stich, and M. Jaggi, "Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication." arXiv, Feb. 1, 2019.

T. Vogels, S. P. Karimireddy, and M. Jaggi, "PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization," ArXiv, May 31, 2019.

S. P. Karimireddy, Q. Rebjock, S. U. Stich, and M. Jaggi, "Error Feedback Fixes SignSGD and other Gradient Compression Schemes." arXiv, Jan. 28, 2019.

A. Krizhevsky, "Learning Multiple Layers of Features from Tiny Images," University of Toronto, Canada, 2009.

"CIFAR-10 and CIFAR-100 datasets." [Online]. Available: https://www.cs.toronto.edu/~kriz/cifar.html.

J. Yang et al., "MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classification," Scientific Data, vol. 10, no. 1, Jan. 2023, Art. no. 41.

J. Yang et al., “[MedMNIST+] 18x Standardized Datasets for 2D and 3D Biomedical Image Classification with Multiple Size Options: 28 (MNIST-Like), 64, 128, and 224.” Zenodo, Jan. 16, 2024.