Cross-Patient Evaluation of CNN-Based Facial Expression Recognition for Intubated ICU Patients Using Leave-One-Patient-Out Validation
Received: 13 December 2025 | Revised: 9 January 2026 | Accepted: 17 January 2026 | Online: 4 April 2026
Corresponding author: Emy Setyaningsih
Abstract
Intubated Intensive Care Unit (ICU) patients experience severe communication limitations due to medical device occlusions and altered physiological conditions, which make facial expression interpretation challenging for healthcare professionals. Facial Expression Recognition (FER) based on Convolutional Neural Network (CNN) offers a promising solution; however, its reliability in real ICU environments critically depends on the model's ability to generalize across patients with heterogeneous clinical characteristics. This study evaluates the cross-patient performance of a CNN-based FER system using a two-stage transfer learning approach and a rigorous patient-independent validation protocol, Leave-One-Patient-Out Cross-Validation (LOPOCV). Four ImageNet backbones, ResNet50, DenseNet121, MobileNetV2, and EfficientNetB0, were trained on 33 videos from 10 intubated ICU patients mapped into three FER classes. Conventional frame-level training yielded high accuracies, with DenseNet121 achieving 100% under non-patient-independent evaluation. An ablation study showed that partially unfreezing the final backbone layers produced the most stable configuration for small-scale clinical datasets. However, LOPOCV revealed a marked performance decrease (mean accuracy ≈ 45%), highlighting identity leakage inherent in frame-level evaluation and limited cross-patient generalization under high occlusion and inter-patient variability. These findings establish a more realistic, patient-independent performance baseline for FER in intubated ICU patient settings and underscore the need for temporal architectures and multimodal strategies to improve robustness and clinical reliability.
Keywords:
convolutional neural network, facial expression recognition, cross-patient evaluation, intubated ICU patients, leave-one-patient-out cross-validation, transfer learningDownloads
References
C. Çelebi and K. Öztepe Yeşilyurt, "Ensuring Effective Communication with Patients Receiving Mechanical Ventilation Support in Intensive Care Units: Current Communication Materials," Cyprus Journal of Medical Sciences, Aug. 2025. DOI: https://doi.org/10.4274/cjms.2025.2025-28
M. Danielis, A. Povoli, E. Mattiussi, and A. Palese, "Understanding patients’ experiences of being mechanically ventilated in the Intensive Care Unit: Findings from a meta‐synthesis and meta‐summary," Journal of Clinical Nursing, vol. 29, no. 13–14, pp. 2107–2124, July 2020. DOI: https://doi.org/10.1111/jocn.15259
S. Fathonah, E. Ernawati, and M. S. Hakim, "Communication Experiences of Conscious Indonesian Patients Who Underwent Mechanical Breathing in an Intensive Care Unit: A Phenomenological Approach," Jordan Medical Journal, vol. 58, no. 4, Nov. 2024. DOI: https://doi.org/10.35516/jmj.v58i4.1601
S. Nerella, K. Khezeli, A. Davidson, P. Tighe, A. Bihorac, and P. Rashidi, "End-to-End Machine Learning Framework for Facial AU Detection in Intensive Care Units." arXiv, 2022.
C. L. Wu et al., "Deep Learning-Based Pain Classifier Based on the Facial Expression in Critically Ill Patients," Frontiers in Medicine, vol. 9, Mar. 2022, Art. no. 851690. DOI: https://doi.org/10.3389/fmed.2022.851690
S. Gkikas and M. Tsiknakis, "Automatic assessment of pain based on deep learning methods: A systematic review," Computer Methods and Programs in Biomedicine, vol. 231, Apr. 2023, Art. no. 107365. DOI: https://doi.org/10.1016/j.cmpb.2023.107365
S. Alphonse, S. Abinaya, and N. Kumar, "Pain assessment from facial expression images utilizing Statistical Frei-Chen Mask (SFCM)-based features and DenseNet," Journal of Cloud Computing, vol. 13, no. 1, Sept. 2024, Art. no. 142. DOI: https://doi.org/10.1186/s13677-024-00706-9
D. Bhagat, A. Vakil, R. K. Gupta, and A. Kumar, "Facial Emotion Recognition (FER) using Convolutional Neural Network (CNN)," Procedia Computer Science, vol. 235, pp. 2079–2089, 2024. DOI: https://doi.org/10.1016/j.procs.2024.04.197
A. Liu and H. Yue, "Facial Expression Recognition Based on CNN-LSTM," in Proceedings of the 2023 7th International Conference on Electronic Information Technology and Computer Engineering, Oct. 2023, pp. 486–491. DOI: https://doi.org/10.1145/3650400.3650480
Q. Zhu, H. Zhuang, M. Zhao, S. Xu, and R. Meng, "A study on expression recognition based on improved mobilenetV2 network," Scientific Reports, vol. 14, no. 1, Apr. 2024, Art. no. 8121. DOI: https://doi.org/10.1038/s41598-024-58736-x
B. Li and D. Lima, "Facial expression recognition via ResNet-50," International Journal of Cognitive Computing in Engineering, vol. 2, pp. 57–64, June 2021. DOI: https://doi.org/10.1016/j.ijcce.2021.02.002
R. A. Elsheikh, M. A. Mohamed, A. M. Abou-Taleb, and M. M. Ata, "Improved facial emotion recognition model based on a novel deep convolutional structure," Scientific Reports, vol. 14, no. 1, Nov. 2024, Art. no. 29050. DOI: https://doi.org/10.1038/s41598-024-79167-8
A. Talukder and S. Ghosh, "Facial Image expression recognition and prediction system," Scientific Reports, vol. 14, no. 1, Nov. 2024, Art. no. 27760. DOI: https://doi.org/10.1038/s41598-024-79146-z
C. J. Meryl, K. Dharshini, D. Sujitha Juliet, J. Akila Rosy, and S. S. Jacob, "Deep Learning based Facial Expression Recognition for Psychological Health Analysis," in 2020 International Conference on Communication and Signal Processing (ICCSP), July 2020, pp. 1155–1158. DOI: https://doi.org/10.1109/ICCSP48568.2020.9182094
R. Angeline and A. A. Nithya, "Deep Human Facial Emotion Recognition: A Transfer Learning Approach Using EfficientNetB0 Model," Journal of Theoretical and Applied Information Technology, vol. 102, no. 8, pp. 3664–3670.
M. Bie, H. Xu, Q. Liu, Y. Gao, K. Song, and X. Che, "DA-FER: Domain Adaptive Facial Expression Recognition," Applied Sciences, vol. 13, no. 10, May 2023, Art. no. 6314. DOI: https://doi.org/10.3390/app13106314
R. Grover and S. Bansal, "Efficient Facial Expression Recognition Through Lightweight CNN Technique on Public Datasets," SN Computer Science, vol. 6, no. 1, Dec. 2024, Art. no. 15. DOI: https://doi.org/10.1007/s42979-024-03557-y
T. Alrimy, A. Alloqmani, A. Alotaibi, N. Aljohani, and S. Kammoun, "Facial Expression Recognition Based on Well-Known ConvNet Architectures," Journal of King Abdulaziz University: Computing and Information Technology Sciences, vol. 12, no. 1, July 2023. DOI: https://doi.org/10.4197/Comp.12-1.5
W. Zhe et al., "A Research on Two-Stage Facial Occlusion Recognition Algorithm based on CNN," Engineering, Technology & Applied Science Research, vol. 14, no. 6, pp. 18205–18212, Dec. 2024. DOI: https://doi.org/10.48084/etasr.8736
S. Nerella, J. Cupka, M. Ruppert, P. Tighe, A. Bihorac, and P. Rashidi, "Pain Action Unit Detection in Critically Ill Patients," in 2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC), July 2021, pp. 645–651. DOI: https://doi.org/10.1109/COMPSAC51774.2021.00094
T. J. Bradshaw, Z. Huemann, J. Hu, and A. Rahmim, "A Guide to Cross-Validation for Artificial Intelligence in Medical Imaging," Radiology: Artificial Intelligence, vol. 5, no. 4, July 2023, Art. no. e220232. DOI: https://doi.org/10.1148/ryai.220232
"Medical Emergency Detection - v2." Roboflow, [Online]. Available: https://universe.roboflow.com/sangeevan-workspace/medical-emergency-detection/dataset/2.
T. Winger, C. Ozdemir, S. L. Narasimhan, and J. Srivastava, "Time-Adaptive Machine Learning Models for Predicting the Severity of Heart Failure with Reduced Ejection Fraction," Diagnostics, vol. 15, no. 6, Mar. 2025, Art. no. 715. DOI: https://doi.org/10.3390/diagnostics15060715
Downloads
How to Cite
License
Copyright (c) 2026 Septiana Fathonah, Emy Setyaningsih, Erma Susanti, Taukhit
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
