Personalized Evaluation of Online Predictive Systems through Learner Profiles
Received: 7 October 2025 | Revised: 21 November 2025 and 8 December 2025 | Accepted: 19 December 2025 | Online: 4 April 2026
Corresponding author: Fatna Ennibras
Abstract
Distance education institutions continue to face high rates of failure and dropout each year, mainly due to the autonomy of the learner and the absence of continuous monitoring. Numerous predictive systems have been developed to identify students at risk; however, their evaluation often relies on global performance metrics that overlook the heterogeneity of learner profiles. This paper introduces a personalized evaluation methodology for predictive models, applied to real data from Loghate, an online multilingual learning platform implemented at Hassan II University of Casablanca to promote language acquisition among students. After thorough preprocessing, engagement and performance indicators were extracted, and learners were clustered into homogeneous profiles using the K-means algorithm. A stacking ensemble model, combining several supervised learning techniques, was then trained and assessed both globally and within each profile. The model achieved strong global performance (82% accuracy), yet clear disparities appeared across learner profiles, with one group reaching 88% accuracy while another dropped to 75%. These variations show that global metrics can mask subgroup-specific biases and reduce the reliability of early warning systems. This approach reinforces the need for differentiated evaluation to build fairer and more reliable predictive systems that adapt interventions to learners' needs. In the long term, it could support the development of adaptive and explainable dashboards for more personalized and equitable learning environments.
Keywords:
online learning, at-risk students, prediction, personalized evaluation, stacking, clustering, learner profilesDownloads
References
Y. H. Hu, C. L. Lo, and S. P. Shih, "Developing early warning systems to predict students’ online learning performance," Computers in Human Behavior, vol. 36, pp. 469–478, July 2014. DOI: https://doi.org/10.1016/j.chb.2014.04.002
S. Rizvi, B. Rienties, and S. A. Khoja, "The role of demographics in online learning; A decision tree based approach," Computers & Education, vol. 137, pp. 32–47, Aug. 2019. DOI: https://doi.org/10.1016/j.compedu.2019.04.001
R. Al-Shabandar, A. J. Hussain, P. Liatsis, and R. Keight, "Detecting At-Risk Students With Early Interventions Using Machine Learning Techniques," IEEE Access, vol. 7, pp. 149464–149478, 2019. DOI: https://doi.org/10.1109/ACCESS.2019.2943351
H. Karalar, C. Kapucu, and H. Gürüler, "Predicting students at risk of academic failure using ensemble model during pandemic in a distance learning system," International Journal of Educational Technology in Higher Education, vol. 18, no. 1, Dec. 2021, Art. no. 63. DOI: https://doi.org/10.1186/s41239-021-00300-y
J. L. Hung, M. C. Wang, S. Wang, M. Abdelrasoul, Y. Li, and W. He, "Identifying At-Risk Students for Early Interventions—A Time-Series Clustering Approach," IEEE Transactions on Emerging Topics in Computing, vol. 5, no. 1, pp. 45–55, Jan. 2017. DOI: https://doi.org/10.1109/TETC.2015.2504239
R. Alshabandar, A. Hussain, R. Keight, A. Laws, and T. Baker, "The Application of Gaussian Mixture Models for the Identification of At-Risk Learners in Massive Open Online Courses," in 2018 IEEE Congress on Evolutionary Computation (CEC), Rio de Janeiro, Brazi, July 2018, pp. 1–8. DOI: https://doi.org/10.1109/CEC.2018.8477770
B. Bouihi, A. Bousselham, E. Aoula, F. Ennibras, and A. Deraoui, "Prediction of Higher Education Student Dropout based on Regularized Regression Models," Engineering, Technology & Applied Science Research, vol. 14, no. 6, pp. 17811–17815, Dec. 2024. DOI: https://doi.org/10.48084/etasr.8644
D. Bañeres, M. E. Rodríguez, A. E. Guerrero-Roldán, and A. Karadeniz, "An Early Warning System to Detect At-Risk Students in Online Higher Education," Applied Sciences, vol. 10, no. 13, June 2020, Art. no. 4427. DOI: https://doi.org/10.3390/app10134427
M. Yağcı, "Educational data mining: prediction of students’ academic performance using machine learning algorithms," Smart Learning Environments, vol. 9, no. 1, Dec. 2022, Art. no. 11. DOI: https://doi.org/10.1186/s40561-022-00192-z
J. B. Diekuu, M. S. Mekala, U. S. Abonie, J. Isaacs, and E. Elyan, "Predicting student next-term performance in degree programs using AI-based approach: a case study from Ghana," Cogent Education, vol. 12, no. 1, Dec. 2025, Art. no. 2481000. DOI: https://doi.org/10.1080/2331186X.2025.2481000
W. Holmes et al., "Ethics of AI in Education: Towards a Community-Wide Framework," International Journal of Artificial Intelligence in Education, vol. 32, no. 3, pp. 504–526, Sept. 2022. DOI: https://doi.org/10.1007/s40593-021-00239-1
S. Slade and P. Prinsloo, "Learning Analytics: Ethical Issues and Dilemmas," American Behavioral Scientist, vol. 57, no. 10, pp. 1510–1529, Oct. 2013. DOI: https://doi.org/10.1177/0002764213479366
A. Ben Soussia, C. Treuillier, A. Roussanaly, and A. Boyer, "Learning Profiles to Assess Educational Prediction Systems," in Artificial Intelligence in Education, vol. 13355, M. M. Rodrigo, N. Matsuda, A. I. Cristea, and V. Dimitrova, Eds. Springer International Publishing, 2022, pp. 41–52. DOI: https://doi.org/10.1007/978-3-031-11644-5_4
K. R. Premlatha, B. Dharani, and T. V. Geetha, "Dynamic learner profiling and automatic learner classification for adaptive e-learning environment," Interactive Learning Environments, vol. 24, no. 6, pp. 1054–1075, Aug. 2016. DOI: https://doi.org/10.1080/10494820.2014.948459
N. Zhang, G. Biswas, and Y. Dong, "Characterizing Students’ Learning Behaviors Using Unsupervised Learning Methods," in Artificial Intelligence in Education, vol. 10331, E. André, R. Baker, X. Hu, Ma. M. T. Rodrigo, and B. Du Boulay, Eds. Springer International Publishing, 2017, pp. 430–441. DOI: https://doi.org/10.1007/978-3-319-61425-0_36
M. Goyal, "k-mean clustering algorithm and its real use case in the security domain," Medium, Aug. 24, 2021. https://monilgoyal.medium.com/k-mean-clustering-algorithm-and-its-real-use-case-in-the-security-domain-fb97f23113a5.
R. Katuwal, P. N. Suganthan, and L. Zhang, "Heterogeneous oblique random forest," Pattern Recognition, vol. 99, Mar. 2020, Art. no. 107078. DOI: https://doi.org/10.1016/j.patcog.2019.107078
T. Lan, H. Hu, C. Jiang, G. Yang, and Z. Zhao, "A comparative study of decision tree, random forest, and convolutional neural network for spread-F identification," Advances in Space Research, vol. 65, no. 8, pp. 2052–2061, Apr. 2020. DOI: https://doi.org/10.1016/j.asr.2020.01.036
I. Rish, "An empirical study of the naive Bayes classifier," in Proceedings of the IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, 2001, pp. 41–46.
N. Šarlija and M. Stanić, "Logistic regression modelling: procedures and pitfalls in developing and interpreting prediction models," Croatian Operational Research Review, vol. 8, no. 2, pp. 631–652, Dec. 2017. DOI: https://doi.org/10.17535/crorr.2017.0041
J. Hamidzadeh and S. Moslemnejad, "Identification of uncertainty and decision boundary for SVM classification training using belief function," Applied Intelligence, vol. 49, no. 6, pp. 2030–2045, June 2019. DOI: https://doi.org/10.1007/s10489-018-1374-0
M. J. Somers and J. C. Casal, "Using Artificial Neural Networks to Model Nonlinearity: The Case of the Job Satisfaction—Job Performance Relationship," Organizational Research Methods, vol. 12, no. 3, pp. 403–417, July 2009. DOI: https://doi.org/10.1177/1094428107309326
X. Chang, X. Jia, and C. Fan, "A probabilistic prediction model based on stacking ensemble learning for completion time in flexible shop-floor," Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture, vol. 239, no. 10, pp. 1458–1471, Aug. 2025. DOI: https://doi.org/10.1177/09544054241277581
Ç. Demirel, "RocketStack: Level-aware deep recursive ensemble learning framework with adaptive feature fusion and model pruning dynamics." arXiv, 2025.
R. Z. Pek, S. T. Ozyer, T. Elhage, T. Ozyer, and R. Alhajj, "The Role of Machine Learning in Identifying Students At-Risk and Minimizing Failure," IEEE Access, vol. 11, pp. 1224–1243, 2023. DOI: https://doi.org/10.1109/ACCESS.2022.3232984
L. A and S. M, "An Efficient Android Malware Detection Framework with Stacking Ensemble Model," International Journal of Engineering Trends and Technology, vol. 70, no. 4, pp. 294–302, Apr. 2022. DOI: https://doi.org/10.14445/22315381/IJETT-V70I4P226
G. Raftopoulos, G. Davrazos, and S. Kotsiantis, "Evaluating Fairness Strategies in Educational Data Mining: A Comparative Study of Bias Mitigation Techniques," Electronics, vol. 14, no. 9, May 2025, Art. no. 1856. DOI: https://doi.org/10.3390/electronics14091856
Downloads
How to Cite
License
Copyright (c) 2026 Fatna Ennibras, Bouchra Bouihi, Rachid El Aasri, Es-Saadia Aoula, Abdelmajid Bousselham
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
