Hybrid 3D U-Net and Attention Mechanisms for Whole Heart Segmentation from CT Images
Received: 2 January 2025 | Revised: 27 January 2025 and 4 February 2025 | Accepted: 6 February 2025 | Online: 3 March 2025
Corresponding author: Anusha Kotte
Abstract
Accurate delineation of heart structures from multimodal images is crucial for the treatment and investigation of different cardiovascular diseases. Automated whole-heart segmentation remains a challenging task due to its complex structure and imbalances in sample data. Convolutional Neural Networks (CNNs) are popular due to their efficiency in segmenting medical images. However, they often struggle to capture long-range dependencies and lack the precision needed for complex anatomic structures such as the heart. To overcome these limitations, this study presents a hybrid 3D U-Net framework that utilizes residual connections with attention mechanisms to improve feature learning and localization of cardiac structures. Residual connections stabilize training in deeper networks and attention blocks focus on relevant regions, refining segmentation quality. This network focuses on relevant regions and uses attention blocks to enhance quality. The proposed architecture was evaluated on 40 volumetric CT images of the Multi-Modality Whole Heart Segmentation (MM-WHS) dataset, achieving an average dice score of 85%. These results demonstrate the effectiveness and high accuracy of the proposed method for delineating cardiac substructures, offering potential clinical utility for automated cardiac analysis.
Keywords:
deep learning, cardiac CT, attention mechanisms, whole-heart segmentationDownloads
References
"Cardiovascular diseases - Level 2 cause | Institute for Health Metrics and Evaluation." https://www.healthdata.org/research-analysis/diseases-injuries-risks/factsheets/2021-cardiovascular-diseases-level-2-disease.
S. Park and M. Chung, "Cardiac segmentation on CT Images through shape-aware contour attentions," Computers in Biology and Medicine, vol. 147, Aug. 2022, Art. no. 105782.
O. Ronneberger, P. Fischer, and T. Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation," in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, 2015, pp. 234–241.
F. Milletari, N. Navab, and S. A. Ahmadi, "V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation," in 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA, Oct. 2016, pp. 565–571.
M. J. J. Ghrabat et al., "Utilizing Machine Learning for the Early Detection of Coronary Heart Disease," Engineering, Technology & Applied Science Research, vol. 14, no. 5, pp. 17363–17375, Oct. 2024.
E. Gibson et al., "Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks," IEEE Transactions on Medical Imaging, vol. 37, no. 8, pp. 1822–1834, Dec. 2018.
O. Oktay et al., "Attention U-Net: Learning Where to Look for the Pancreas." arXiv, May 20, 2018.
M. Chung, J. Lee, S. Park, C. E. Lee, J. Lee, and Y. G. Shin, "Liver segmentation in abdominal CT images via auto-context neural network and self-supervised contour attention," Artificial Intelligence in Medicine, vol. 113, Mar. 2021, Art. no. 102023.
F. Liu, K. Wang, D. Liu, X. Yang, and J. Tian, "Deep pyramid local attention neural network for cardiac structure segmentation in two-dimensional echocardiography," Medical Image Analysis, vol. 67, Jan. 2021, Art. no. 101873.
F. Liu, K. Wang, D. Liu, X. Yang, and J. Tian, "Deep pyramid local attention neural network for cardiac structure segmentation in two-dimensional echocardiography," Medical Image Analysis, vol. 67, Jan. 2021, Art. no. 101873.
X. Sun, P. Garg, S. Plein, and R. J. van der Geest, "SAUN: Stack attention U-Net for left ventricle segmentation from cardiac cine magnetic resonance imaging," Medical Physics, vol. 48, no. 4, pp. 1750–1763, 2021.
Y. Zeng et al., "MAEF-Net: Multi-attention efficient feature fusion network for left ventricular segmentation and quantitative analysis in two-dimensional echocardiography," Ultrasonics, vol. 127, Jan. 2023, Art. no. 106855.
W. Zhang, F. Lu, W. Zhao, Y. Hu, H. Su, and M. Yuan, "ACCPG-Net: A skin lesion segmentation network with Adaptive Channel-Context-Aware Pyramid Attention and Global Feature Fusion," Computers in Biology and Medicine, vol. 154, Mar. 2023, Art. no. 106580.
Q. Tong et al., "RIANet: Recurrent interleaved attention network for cardiac MRI segmentation," Computers in Biology and Medicine, vol. 109, pp. 290–302, Jun. 2019.
L. Guo et al., "Dual attention enhancement feature fusion network for segmentation and quantitative analysis of paediatric echocardiography," Medical Image Analysis, vol. 71, Jul. 2021, Art. no. 102042.
K. N. Wang et al., "AWSnet: An auto-weighted supervision attention network for myocardial scar and edema segmentation in multi-sequence cardiac magnetic resonance images," Medical Image Analysis, vol. 77, Apr. 2022, Art. no. 102362.
X. Ding, Y. Peng, C. Shen, and T. Zeng, "CAB U-Net: An end-to-end category attention boosting algorithm for segmentation," Computerized Medical Imaging and Graphics, vol. 84, Sep. 2020, Art. no. 101764.
S. Gao, H. Zhou, Y. Gao, and X. Zhuang, "BayeSeg: Bayesian modeling for medical image segmentation with interpretable generalizability," Medical Image Analysis, vol. 89, Oct. 2023, Art. no. 102889.
X. Zhuang, "Multivariate Mixture Model for Myocardial Segmentation Combining Multi-Source Images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 12, pp. 2933–2946, Sep. 2019.
X. Luo and X. Zhuang, "X-Metric: An N-Dimensional Information-Theoretic Framework for Groupwise Registration and Deep Combined Computing," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 7, pp. 9206–9224, Jul. 2023.
F. Wu and X. Zhuang, "Minimizing Estimated Risks on Unlabeled Data: A New Formulation for Semi-Supervised Medical Image Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 5, pp. 6021–6036, Feb. 2023.
T. Liu, Y. Tian, S. Zhao, X. Huang, and Q. Wang, "Automatic Whole Heart Segmentation Using a Two-Stage U-Net Framework and an Adaptive Threshold Window," IEEE Access, vol. 7, pp. 83628–83636, 2019.
L. R. Dice, "Measures of the Amount of Ecologic Association Between Species," Ecology, vol. 26, no. 3, pp. 297–302, 1945.
Downloads
How to Cite
License
Copyright (c) 2025 Anusha Kotte, V. Kamakshi Prasad
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
