Efficient Load Balancing for Future Dense Networks using Radio over Fiber Infrastructure and applying Different Learning Rates
Received: 3 January 2025 | Revised: 3 February 2025 | Accepted: 11 February 2025 | Online: 3 April 2025
Corresponding author: Mahfida Amjad Dipa
Abstract
Reinforcement Learning (RL) can lead to effective Load-Balancing (LB) mechanisms, as traditional methods cannot always provide an optimal solution in cellular networks. This study proposes an RL-based LB scheme for a dense network that uses radio over fiber infrastructure. The proposed technique is based on LB constraints in the action space that maintain zero violation during the learning process. In this technique, a Deep Q-Network agent was chosen to search for an optimal policy to maximize the expected cumulative long-term reward to satisfy the constraints. This study uses the number of user entities per base station in the dense network as constraints to maintain average throughput based on the Signal-to-Noise Ratio (SNR) generated from the radio frequency signals of the network. The proposed method outperformed at an SNR of 38 dB with a throughput of 32 Mbps for a 20 MHz channel bandwidth for macro- and microcells in the dense network. Furthermore, this study examined the effect of different learning rates as hyperparameters in the system. The proposed approach shows that when the agent was trained with a learning rate of 1e-3, the network performed well by obtaining a higher CDF compared to a learning rate of 1e-5. In addition, the system achieved higher rewards for a learning rate of 1e-3 with or without LB constraints, confirming the efficiency of the proposed scheme. The simulation results showed that CDF was 4% higher when using constraints compared to without constraints.
Keywords:
RoF, dense network, radio over fiber, load balancing, reinforcement learning, learning rateDownloads
References
Z. He and J. Wang, "Non-cooperative differential game based Load Balancing Algorithm in Radio-over-Fibre system," China Communications, vol. 11, no. 14, pp. 79–85, 2014.
A. Awada, I. Viering, B. Wegmann, and A. Klein, "Application of Game Theory for Load Balancing in Long Term Evolution Networks," Frequenz, vol. 64, no. 9–10, pp. 180–184, Oct. 2010.
H. H. Lu et al., "A combined fibre/free-space-optical communication system for long-haul wireline/wireless transmission at millimetre-wave/sub-THz frequencies," Communications Engineering, vol. 2, no. 1, pp. 1–8, May 2023.
G. Pandey, A. Choudhary, and A. Dixit, "Wavelength Division Multiplexed Radio Over Fiber Links for 5G Fronthaul Networks," IEEE Journal on Selected Areas in Communications, vol. 39, no. 9, pp. 2789–2803, Sep. 2021.
K. Singh, A. Dixit, and V. K. Jain, "Hybrid Medium Access Control Protocol for Radio-Over-Fiber Networks," IEEE Access, vol. 9, pp. 110889–110903, 2021.
B. G. Kim, S. H. Bae, H. Kim, and Y. C. Chung, "Feasibility of RoF-based optical fronthaul network for next-generation mobile communications," in 2017 Opto-Electronics and Communications Conference (OECC) and Photonics Global Conference (PGC), Singapore, Jul. 2017, pp. 1–2.
A. Udalcovs et al., "An Insight into the Total Cost of Ownership of 5G Fronthauling," in 2018 20th International Conference on Transparent Optical Networks (ICTON), Bucharest, Romania, Jul. 2018, pp. 1–5.
B. G. Kim, S. H. Bae, H. Kim, and Y. C. Chung, "Optical fronthaul technologies for next-generation mobile communications," in 2016 18th International Conference on Transparent Optical Networks (ICTON), Trento, Italy, Jul. 2016, pp. 1–3.
S. Saibharath, S. Mishra, and C. Hota, "Swap-Based Load Balancing for Fairness in Radio Access Networks," IEEE Wireless Communications Letters, vol. 10, no. 11, pp. 2412–2416, Aug. 2021.
W. K. Lai, Y. J. Yu, P. L. Tsai, and M. H. Shen, "A Multidepth Load-Balance Scheme for Clusters of Congested Cells in Ultradense Cellular Networks," Wireless Communications and Mobile Computing, vol. 2020, no. 1, 2020, Art. no. 6975890.
K. Suresh et al., "Enhanced Metaheuristic Algorithm-Based Load Balancing in a 5G Cloud Radio Access Network," Electronics, vol. 11, no. 21, Jan. 2022, Art. no. 3611.
D. Wu et al., "Reinforcement learning for communication load balancing: approaches and challenges," Frontiers in Computer Science, vol. 5, May 2023.
B. Lim and M. Vu, "Distributed Multi-Agent Deep Q-Learning for Load Balancing User Association in Dense Networks," IEEE Wireless Communications Letters, vol. 12, no. 7, pp. 1120–1124, Jul. 2023.
R. Ahmad, M. D. Soltani, M. Safari, and A. Srivastava, "Reinforcement Learning-Based Near-Optimal Load Balancing for Heterogeneous LiFi WiFi Network," IEEE Systems Journal, vol. 16, no. 2, pp. 3084–3095, Jun. 2022.
A. Lobinger, S. Stefanski, T. Jansen, and I. Balan, "Load Balancing in Downlink LTE Self-Optimizing Networks," in 2010 IEEE 71st Vehicular Technology Conference, Taipei, Taiwan, 2010, pp. 1–5.
H. M. Kavana, V. B. Kavya, B. Madhura, and N. Kamat, "Load Balancing using SDN Methodology," International Journal of Engineering Research & Technology, vol. 7, no. 5, pp. 206–208, May 2008.
H. Wang, J. Ren, and T. Li, "Resource Allocation with Load Balancing for Cognitive Radio Networks," in 2010 IEEE Global Telecommunications Conference GLOBECOM 2010, Miami, FL, USA, Dec. 2010, pp. 1–5.
T. Oyama and T. Seyama, "Load-Balancing Techniques for 5G Millimeter-Wave Distributed Antenna System," in 2021 IEEE 93rd Vehicular Technology Conference (VTC2021-Spring), Helsinki, Finland, Apr. 2021, pp. 1–5.
"5G; NR; Physical layer procedures for data," ETSI, TS 138 214-V16.2.0-Release 16, 2020.
D. Jain and B. Iyer, "Design and analysis of high‐speed four‐channel WDM Radio over Fiber system for Millimeter-wave applications," International Journal of System Assurance Engineering and Management, vol. 14, no. 3, pp. 746–758, Jul. 2023.
B. Tamrakar, K. Singh, and P. Kumar, "Performance Measurement of Radio over Fiber System at 20 GHz and 30 GHz by Employing with and without Optical Carrier Suppression," in 2019 6th International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, India, Mar. 2019, pp. 167–172.
R. A. Shafik, Md. S. Rahman, and A. R. Islam, "On the Extended Relationships Among EVM, BER and SNR as Performance Metrics," in 2006 International Conference on Electrical and Computer Engineering, Dhaka, Bangladesh, Dec. 2006, pp. 408–411.
T. T. H. Ly, H. S. Nguyen, T. S. Nguyen, V. V. Huynh, T. L. Nguyen, and M. Voznak, "Outage Probability Analysis in Relaying Cooperative Systems with NOMA Considering Power Splitting," Symmetry, vol. 11, no. 1, Jan. 2019, Art. no. 72.
M. Alhabo, L. Zhang, and O. Oguejiofor, "Inbound handover interference-based margin for load balancing in heterogeneous networks," in 2017 International Symposium on Wireless Communication Systems (ISWCS), Bologna, Aug. 2017, pp. 146–151.
Y. Xu, W. Xu, Z. Wang, J. Lin, and S. Cui, "Load Balancing for Ultradense Networks: A Deep Reinforcement Learning-Based Approach," IEEE Internet of Things Journal, vol. 6, no. 6, pp. 9399–9412, Sep. 2019.
G. Dalal, K. Dvijotham, M. Vecerik, T. Hester, C. Paduraru, and Y. Tassa, "Safe Exploration in Continuous Action Spaces." arXiv, Jan. 26, 2018.
J. Brownlee, "Understand the Impact of Learning Rate on Neural Network Performance," MachineLearningMastery.com, Jan. 24, 2019. https://www.machinelearningmastery.com/understand-the-dynamics-of-learning-rate-on-deep-learning-neural-networks/.
P. E. I. Rivera and M. Erol-Kantarci, "Competitive Multi-Agent Load Balancing with Adaptive Policies in Wireless Networks." arXiv, Oct. 13, 2021.
"Keras documentation: Adam." https://keras.io/api/optimizers/adam/.
M. Q. Abdulhasan, M. I. Salman, C. K. Ng, N. K. Noordin, S. J. Hashim, and F. Hashim, "An Adaptive Threshold Feedback Compression Scheme Based on Channel Quality Indicator (CQI) in Long Term Evolution (LTE) System," Wireless Personal Communications, vol. 82, no. 4, pp. 2323–2349, Jun. 2015, https://doi.org/10.1007/s11277-015-2350-1.
"NORM.DIST Function," Corporate Finance Institute. https://corporatefinanceinstitute.com/resources/excel/norm-dist-function/.
"NORMDIST function - Microsoft Support." https://support.microsoft.com/en-us/office/normdist-function-126db625-c53e-4591-9a22-c9ff422d6d58.
A. Y. Al-Zahrani, "On the Statistical Distribution of Packets Service Time in Cellular Access Networks," Engineering, Technology & Applied Science Research, vol. 9, no. 6, pp. 4996–5000, Dec. 2019.
A. S. Afolabi, S. Ahmed, and O. A. Akinola, "A Reinforcement Learning Approach for Interference Management in Heterogeneous Wireless Networks," International Journal of Interactive Mobile Technologies (iJIM), vol. 15, no. 12, Jun. 2021, Art. no. 65.
Downloads
How to Cite
License
Copyright (c) 2025 Mahfida Amjad Dipa, Syamsuri Yaakob, Fadlee Rasid, Faisul Ahmad, Azwan Mahmud
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
