A Hybrid Imbalanced DDoS Detection Framework Utilizing CNN, LSTM, and K-Means SMOTE
Received: 12 December 2025 | Revised: 15 January 2026 | Accepted: 23 January 2026 | Online: 4 April 2026
Corresponding author: Rissal Efendi
Abstract
Cyberattacks remain a highly disruptive threat to modern networks. However, the imbalanced nature of real-world network traffic, where attack data constitute only a small fraction, poses significant challenges for accurate detection. This study proposes a hybrid deep learning framework that combines Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) models with a K-means Synthetic Minority Oversampling Technique (SMOTE) to address class imbalance in penetration testing data. A total of 1,532,029 network flow records were collected during penetration testing, comprising 1,230,487 benign flows (80.4%) and 301,542 malicious flows (19.6%), which represented Distributed Denial of Service (DDoS) attacks, including SYN floods, UDP floods, and ICMP floods. The CNN component extracts spatial features from network flows, while the LSTM captures their temporal dependencies. K-means SMOTE enhances detection by generating realistic synthetic samples for minority attack classes. The experimental results show that the CNN-LSTM model with K-means SMOTE achieves a DDoS detection recall of 94.59% and an F1-score of 89.45%, significantly outperforming the imbalanced baseline, with a recall of 64.35% and an F1-score of 73.05%, as well as other classifiers such as Support Vector Machine (SVM) and Random Forest (RF). These findings demonstrate the model's robustness and practicality in detecting minority-class attacks under real-world conditions.
Keywords:
cyberattack detection, DDoS, CNN-LSTM, K-means SMOTE, imbalanced dataDownloads
References
D. M. A. A. Afraji, J. Lloret, and L. Peñalver, "Deep Learning-Driven Defense Strategies for Mitigating DDoS Attacks in Cloud Computing Environments," Cyber Security and Applications, vol. 3, Dec. 2025, Art. no. 100085. DOI: https://doi.org/10.1016/j.csa.2025.100085
A. Abdelkhalek and M. Mashaly, "Addressing the Class Imbalance Problem in Network Intrusion Detection Systems Using Data Resampling and Deep Learning," The Journal of Supercomputing, vol. 79, no. 10, pp. 10611–10644, Jul. 2023. DOI: https://doi.org/10.1007/s11227-023-05073-x
M. A. Talukder et al., "Machine Learning-Based Network Intrusion Detection for Big and Imbalanced Data Using Oversampling, Stacking Feature Embedding and Feature Extraction," Journal of Big Data, vol. 11, no. 1, Feb. 2024, Art. no. 33. DOI: https://doi.org/10.1186/s40537-024-00886-w
S. Aktar and A. Yasin Nur, "Towards DDoS Attack Detection Using Deep Learning Approach," Computers & Security, vol. 129, Jun. 2023, Art. no. 103251. DOI: https://doi.org/10.1016/j.cose.2023.103251
N. Mandela and F. Etyang, "Comparative Analysis of Deep Learning Models for Effective Denial of Service (DoS) Attack Detection in Network Security," Journal of Electrical Systems and Information Technology, vol. 12, no. 1, Sep. 2025, Art. no. 73. DOI: https://doi.org/10.1186/s43067-025-00267-0
M. Mbow, H. Koide, and K. Sakurai, "Handling Class Imbalance Problem in Intrusion Detection System Based on Deep Learning," International Journal of Networking and Computing, vol. 12, no. 2, pp. 467–492, 2022. DOI: https://doi.org/10.15803/ijnc.12.2_467
M. S. Milosevic and V. M. Ciric, "Extreme Minority Class Detection in Imbalanced Data for Network Intrusion," Computers & Security, vol. 123, Dec. 2022, Art. no. 102940. DOI: https://doi.org/10.1016/j.cose.2022.102940
R. Barona and E. Baburaj, "An Efficient DDoS Attack Detection and Categorization Using Adolescent Identity Search-Based Weighted SVM Model," Peer-to-Peer Networking and Applications, vol. 16, no. 2, pp. 1227–1241, Mar. 2023. DOI: https://doi.org/10.1007/s12083-023-01460-6
T. Wu, H. Fan, H. Zhu, C. You, H. Zhou, and X. Huang, "Intrusion Detection System Combined Enhanced Random Forest with SMOTE Algorithm," EURASIP Journal on Advances in Signal Processing, vol. 2022, no. 1, Dec. 2022, Art. no. 39. DOI: https://doi.org/10.1186/s13634-022-00871-6
M. S. Christo, J. J. Menandas, M. George, and S. V. Nuna, "DDoS Detection using Multilayer Perceptron," in 2023 4th International Conference on Electronics and Sustainable Communication Systems (ICESC), Coimbatore, India, Jul. 2023, pp. 688–693. DOI: https://doi.org/10.1109/ICESC57686.2023.10193406
J. Kim, J. Kim, H. Kim, M. Shim, and E. Choi, "CNN-Based Network Intrusion Detection against Denial-of-Service Attacks," Electronics, vol. 9, no. 6, Jun. 2020, Art. no. 916. DOI: https://doi.org/10.3390/electronics9060916
Y. Yang, S. Tu, R. H. Ali, H. Alasmary, M. Waqas, and M. N. Amjad, "Intrusion Detection Based on Bidirectional Long Short-Term Memory with Attention Mechanism," Computers, Materials & Continua, vol. 74, no. 1, pp. 801–815, 2023. DOI: https://doi.org/10.32604/cmc.2023.031907
V. Hnamte and J. Hussain, "DCNNBiLSTM: An Efficient Hybrid Deep Learning-Based Intrusion Detection System," Telematics and Informatics Reports, vol. 10, Jun. 2023, Art. no. 100053. DOI: https://doi.org/10.1016/j.teler.2023.100053
Z. S. Dhahir, "A Hybrid Approach for Efficient DDoS Detection in Network Traffic Using CBLOF-Based Feature Engineering and XGBoost," Journal of Future Artificial Intelligence and Technologies, vol. 1, no. 2, pp. 174–190, Sep. 2024. DOI: https://doi.org/10.62411/faith.2024-33
S. S. Bamber, A. V. R. Katkuri, S. Sharma, and M. Angurala, "A Hybrid CNN-LSTM Approach for Intelligent Cyber Intrusion Detection System," Computers & Security, vol. 148, Jan. 2025, Art. no. 104146. DOI: https://doi.org/10.1016/j.cose.2024.104146
A. A. Najar and S. Manohar Naik, "DDoS Attack Detection Using CNN-BiLSTM with Attention Mechanism," Telematics and Informatics Reports, vol. 18, Jun. 2025, Art. no. 100211. DOI: https://doi.org/10.1016/j.teler.2025.100211
R. Efendi, "Optimizing Neural Network Architecture for Detecting DDOS Attacks Using ANN and XGBoost in Imbalanced Networks," Engineering, Technology & Applied Science Research, vol. 15, no. 3, pp. 22518–22526, Jun. 2025. DOI: https://doi.org/10.48084/etasr.9909
Y. Xue, C. Kang, and H. Yu, "A Network Intrusion Detection System Utilizing a Novel Autoencoder and a Hybrid Enhanced LSTM-CNN-Based Residual Network," Computers & Security, vol. 151, Apr. 2025, Art. no. 104328. DOI: https://doi.org/10.1016/j.cose.2025.104328
H. C. Altunay and Z. Albayrak, "A Hybrid CNN+LSTM-Based Intrusion Detection System for Industrial IoT Networks," Engineering Science and Technology, an International Journal, vol. 38, Feb. 2023, Art. no. 101322. DOI: https://doi.org/10.1016/j.jestch.2022.101322
M. Abdallah, N. An Le Khac, H. Jahromi, and A. Delia Jurcut, "A Hybrid CNN-LSTM Based Approach for Anomaly Detection Systems in SDNs," in Proceedings of the 16th International Conference on Availability, Reliability and Security, Vienna, Austria, Aug. 2021, pp. 1–7. DOI: https://doi.org/10.1145/3465481.3469190
A. O. Widodo, B. Setiawan, and R. Indraswari, "Machine Learning-Based Intrusion Detection on Multi-Class Imbalanced Dataset Using SMOTE," Procedia Computer Science, vol. 234, pp. 578–583, 2024. DOI: https://doi.org/10.1016/j.procs.2024.03.042
A. Hozouri, A. Mirzaei, and M. Effatparvar, "A Comprehensive Survey on Intrusion Detection Systems with Advances in Machine Learning, Deep Learning and Emerging Cybersecurity Challenges," Discover Artificial Intelligence, vol. 5, no. 1, Nov. 2025, Art. no. 314. DOI: https://doi.org/10.1007/s44163-025-00578-1
J. C. Mondragon, P. Branco, G.-V. Jourdan, A. E. Gutierrez-Rodriguez, and R. R. Biswal, "Advanced IDS: A Comparative Study of Datasets and Machine Learning Algorithms for Network Flow-Based Intrusion Detection Systems," Applied Intelligence, vol. 55, no. 7, May 2025, Art. no. 608. DOI: https://doi.org/10.1007/s10489-025-06422-4
Y. Yang, H. Akbarzadeh Khorshidi, and U. Aickelin, "A Diversity-Based Synthetic Oversampling Using Clustering for Handling Extreme Imbalance," SN Computer Science, vol. 4, no. 6, Nov. 2023, Art. no. 848. DOI: https://doi.org/10.1007/s42979-023-02249-3
D. Akgun, S. Hizal, and U. Cavusoglu, "A New DDoS Attacks Intrusion Detection Model Based on Deep Learning for Cybersecurity," Computers & Security, vol. 118, Jul. 2022, Art. no. 102748. DOI: https://doi.org/10.1016/j.cose.2022.102748
Downloads
How to Cite
License
Copyright (c) 2026 Rissal Efendi, Indrastanti Ratna Widiasari, Erwien Christianto
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
