Sandeep Angara*, Nishith Reddy Mannuru, Aashrith Mannuru and Sharath Thirunagaru

University of North Texas, USA

*Corresponding Author: Sandeep Angara, University of North Texas, USA.

Received: December 15, 2023; Published: December 20, 2023

Abstract

Pneumonia remains a leading cause of morbidity and mortality worldwide. Chest X-ray (CXR) imaging is a fundamental diagnostic tool, but traditional analysis relies on time-intensive expert evaluation. Recently, deep learning has shown immense potential for automating pneumonia detection from CXRs. This paper explores applying neural networks to improve CXR-based pneumonia diagnosis. We developed a novel model fusing Convolution Neural networks (CNN) and Vision Transformer networks via model-level ensembling. Our fusion architecture combines a ResNet34 variant and a Multi-Axis Vision Transformer small model. Both base models are initialized with ImageNet pre-trained weights. The output layers are removed, and features are combined using a flattening layer before final classification. Experiments used the Kaggle pediatric pneumonia dataset containing 1,341 normal and 3,875 pneumonia CXR images. We compared our model against standalone ResNet34, Vision Transformer, and Swin Transformer Tiny baseline models using identical training procedures. Extensive data augmentation, Adam optimization, learning rate warmup, and decay were employed. The fusion model achieved a state-of-the-art accuracy of 94.87%, surpassing the baselines. We also attained excellent sensitivity, specificity, kappa score, and positive predictive value. Confusion matrix analysis confirms fewer misclassifications. The ResNet34 and Vision Transformer combination enables jointly learning robust features from CNN’s and Transformer paradigms. This model-level ensemble technique effectively integrates their complementary strengths for enhanced pneumonia classification.

Keywords: Convolution Neural Networks; Vison Transformers; Pneumonia; Chest X-rays; Deep Learning; Ensemble Techniques

References

1. K K Yadav and S Awasthi. “Childhood Pneumonia: What’s Unchanged, and What’s New?”. Indian Journal of Pediatrics 7. (2023).
2. C L Fischer Walker., et al. “Global burden of childhood pneumonia and diarrhoea”. The Lancet 381.9875 (2013).
3. HJ Zar., et al. “Pneumonia in low and middle income countries: Progress and challenges”. Thorax11 (2013).
4. P Gang., et al. “Dimensionality reduction in deep learning for chest X-ray analysis of lung cancer”. in Proceedings - 2018 10th International Conference on Advanced Computational Intelligence, ICACI 2018, (2018).
5. N Barakat., et al. “A machine learning approach on chest X-rays for pediatric pneumonia detection”. Digit Health 9 (2023).
6. H Sharma., et al. “Feature extraction and classification of chest X-ray images using CNN to detect pneumonia”. in Proceedings of the Confluence 2020 - 10th International Conference on Cloud Computing, Data Science and Engineering, (2020).
7. J G Lee., et al. “Deep learning in medical imaging: General overview”. Korean Journal of Radiology4. (2017).
8. A Krizhevsky., et al. “ImageNet classification with deep convolutional neural networks”. Commun ACM6 (2017).
9. Y LeCun., et al. “Deep learning (2015), Y. LeCun, Y. Bengio and G. Hinton”. Nature 521 (2015).
10. A Esteva., et al. “A guide to deep learning in healthcare”. Nature Medicine 1. (2019).
11. L Oakden-Rayner., et al. “Hidden stratification causes clinically meaningful failures in machine learning for medical imaging”. in ACM CHIL 2020 - Proceedings of the 2020 ACM Conference on Health, Inference, and Learning, (2020).
12. M Roberts., et al. “Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans”. Nature Machine Intelligence3 (2021).
13. F Jiang., et al. “Artificial intelligence in healthcare: Past, present and future”. Stroke and Vascular Neurology4. (2017).
14. J R Zech., et al. “Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study”. PLoS Medicine11 (2018).
15. R Jain., et al. “Pneumonia detection in chest X-ray images using convolutional neural networks and transfer learning”. Measurement (Lond) 165 (2020).
16. Sunil L Bangare., et al. “Pneumonia Detection and Classification using CNN and VGG16”. International Journal of Advanced Research in Science, Communication and Technology (2022).
17. A Ijaz., et al. “Deep Learning for Pneumonia Diagnosis Using CXR Images”. in Proceedings - 2023 6th International Conference of Women in Data Science at Prince Sultan University, WiDS-PSU (2023).
18. S Gündel., et al. “Learning to recognize abnormalities in chest X-rays with location-aware dense networks”. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (2019).
19. AH Alharbi and H A Hosni Mahmoud. “Pneumonia Transfer Learning Deep Learning Model from Segmented X-rays”. Healthcare (Switzerland)6 (2022).
20. D Srivastav., et al. “Improved classification for pneumonia detection using transfer learning with GAN based synthetic image augmentation”. in Proceedings of the Confluence 2021: 11th International Conference on Cloud Computing, Data Science and Engineering (2021).
21. A Manickam., et al. “Automated pneumonia detection on chest X-ray images: A deep learning approach with different optimizers and transfer learning architectures”. Measurement (Lond) 184 (2021).
22. SV Militante., et al. “Pneumonia and COVID-19 Detection using Convolutional Neural Networks”. in Proceeding - 2020 3rd International Conference on Vocational Education and Electrical Engineering: Strengthening the framework of Society 5.0 through Innovations in Education, Electrical, Engineering and Informatics Engineering, ICVEE 2020 (2020).
23. Z P Jiang., et al. “An improved VGG16 model for pneumonia image classification”. Applied Sciences (Switzerland)23 (2021).
24. P Szepesi and L Szilágyi. “Detection of pneumonia using convolutional neural networks and deep learning”. Biocybernetics and Biomedical Engineering3 (2022).
25. AK Jaiswal., et al. “Identifying pneumonia in chest X-rays: A deep learning approach”. Measurement (Lond) 145 (2019).
26. D Varshni., et al. “Pneumonia Detection Using CNN based Feature Extraction”. in Proceedings of 2019 3rd IEEE International Conference on Electrical, Computer and Communication Technologies, ICECCT (2019).
27. P Rajpurkar., et al. “CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning”.
28. K Almezhghwi., et al. “Convolutional neural networks for the classification of chest X-rays in the IoT era”. Multimedia Tools and Applications19 (2021).
29. S Chen., et al. “Interpretable CNN-Multilevel Attention Transformer for Rapid Recognition of Pneumonia from Chest X-Ray Images”. IEEE J Biomed Health Inform (2023).
30. DS Kermany., et al. “Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning”. Cell 5 (2018).
31. K He., et al. “Deep residual learning for image recognition”. in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2016).
32. Z Tu., et al. “MaxViT: Multi-Axis Vision Transformer”. Apr. (2022).
33. A Paszke., et al. “PyTorch: An imperative style, high-performance deep learning library”. in Advances in Neural Information Processing Systems, (2019).
34. R Wightman., et al. “rwightman/pytorch-image-models: v0.8.10dev0 Release”. Feb (2023).
35. DP Kingma and J Ba. “Adam: A Method for Stochastic Optimization”. Dec. (2014).
36. A Dosovitskiy., et al. “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale”. (2020).
37. Z Liu., et al. “Swin Transformer: Hierarchical Vision Transformer using Shifted Windows”. Mar. (2021).

Citation

Citation: Sandeep Angara., et al. “A Novel Method to Enhance Pneumonia Detection Via a Model-Level Ensembling of CNN and Vision Transformer". Acta Scientific Computer Sciences 6.1 (2024): 21-28.

Copyright

Copyright: © 2024 Sandeep Angara., et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.