Evolutionary Computation Paradigm to Determine Deep Neural Networks Architectures

Renato Constantin Ivanescu; Smaranda Belciug; Andrei  Nascu; Mircea Sebastian Serbanescu; Dominic Gabriel Iliescu

doi:10.15837/ijccc.2022.5.4886

Authors

Renato Constantin Ivanescu University of Craiova, Department of Computers and Information Technologies, Romania
Smaranda Belciug University of Craiova, Department of Computer Science, Romania
Andrei Nascu University of Craiova, Department of Computer Science, Romania
Mircea Sebastian Serbanescu University of Craiova, Romania and University of Medicine and Pharmacy of Craiova, Romania
Dominic Gabriel Iliescu University of Craiova, Romania and University of Medicine and Pharmacy of Craiova, Romania

DOI:

https://doi.org/10.15837/ijccc.2022.5.4886

Keywords:

Deep Learning, evolutionary computation, Statistical Analysis, fetal morphology, image classification

Abstract

Image classification is usually done using deep learning algorithms. Deep learning architectures are set deterministically. The aim of this paper is to propose an evolutionary computation paradigm that optimises a deep learning neural network’s architecture. A set of chromosomes are randomly generated, after which selection, recombination, and mutation are applied. At each generation the fittest chromosomes are kept. The best chromosome from the last generation determines the deep learning architecture. We have tested our method on a second trimester fetal morphology database. The proposed model is statistically compared with DenseNet201 and ResNet50, proving its competitiveness.

References

Boyle, B., et al., (2018). Estimating Global Burden of Disears due to congenital anomaly: an analysis of European Data, Archives of Disease in Childhood - Fetal and neonatal edition, 103, F22-F28, 2018

https://doi.org/10.1136/archdischild-2016-311845

Kinsner-Ovaskainen, A., et al., (2018). European Monitoring of Congenital Anomalies: JRC EUROCAT Report on Statistical Monitoring of Congenital Anomalies (2008-2017), EUR 30158 EN, Publications Office of the European Union, Luxembourg, doi: 10.2760/65886, 2018.

Lobo, I., Zhaurova, K., (2008). Birth defects: causes and statistics, Nature Education, 1 (1), 18, 2008.

AlQaheri, H., et al., (2021). Toward an autonomous incubation system for monitoring premature infants. Studies in Informatics and Control, 30 (4), 121-131, https://doi.org/10.23846/v30i4y202111, 2021.

https://doi.org/10.24846/v30i4y202111

Tegnander, E., Eik-Nes, S.H., (2006). The examiner's ultrasound experience has a significant impact on the detection rate of congenital heart defect at the second trimester fetal examination. Ultrasound Obstet Gyncol, 28, 8-14, 2006.

https://doi.org/10.1002/uog.2804

Salomon, L., et al., (2008). A score-based method for quality control of fetal images at routine second trimester ultrasound examination. Prenat Diagn, 28 (9), 822-827, 2008.

https://doi.org/10.1002/pd.2016

Matsuoka, R., Komatsu, M., et al., (2019). A novel deep learning based system for fetal cardiac screening. Ultrasound Obstet Gyn, https://doi.org/10.1002/uog.20945, 2019

https://doi.org/10.1002/uog.20945

Komatsu, R., Matsuoka, R., et al., (2019). Novel AI-guided ultrasound screening system for fetal heart can demonstrate findings in timeline diagram. Ultrasound Obstet Gyn, http://doi.org/10.1002/uog.20796,2019.

https://doi.org/10.1002/uog.20796

Namburete, A. et al., (2018). Fully automated alignment of 3D fetal brain ultrasound to a canonical reference space using multi-task learning. Med Image Anal., 46, 1-14, 2018.

https://doi.org/10.1016/j.media.2018.02.006

Torrents-Barrena, J. et al, (2019). Assessment of radiomics and deep learning for the segmenation of fetal and maternal anatomy in magnetic resonance imaging and ultrasound. Acad. Radiol., S1076-6332(19)30575-6, 2019.

Al-Bander, B. et al., (2020). Improving fetal head countour detection by object localization with deep learning. Annual Conference on MedicalImage Understanding and Analysis, Springer, 142- 150, 2020.

https://doi.org/10.1007/978-3-030-39343-4_12

Belciug, S., (2022). Learning deep neural networks' architectures using differential evolution. Case study: medical imaging processing, Computers in Biology and Medicine, 146, 105623, 2022.

https://doi.org/10.1016/j.compbiomed.2022.105623

Lindauer, M., Hutter, F., (2019). Best Practices for Scientific Research on Neural Architecture Search, arxiv.org/abs/1909.02453, 2019.

Stanley, K.O. (2017). Neuroevolution: a Different Kind of Deep Learning, 2017.

Whitelam, S., Selin V., Park, S-W., Tamblyn, I., (2021). Correspondence between neuroevolution and gradient descent. Nat Commun, 12, 6317 https://doi.org/10.1038/s41467-021-26568-2, 2021.

https://doi.org/10.1038/s41467-021-26568-2

Khadka, S. et al., (2019). Evolutionary reinforcement learning for sample-efficient multiagent coordination. ECLR 2020, 2019.

Hajewski, J., Oliviera, S., Xing, X. (2020). Distributed evolution of deep autoencoders. arxiv.org/abs/2004.07607, 2020.

https://doi.org/10.1145/3377929.3390011

Sun, Y., Wang, B., Xue, B, Jin, Y., Yen, G.G., Zhang, M., (2020). Surrogate-assisted evolutionary deep learning using an end-to-end random forest based performance predictor. IEEE Trans Evol Comput, 24 (2), 350-364, 2020.

https://doi.org/10.1109/TEVC.2019.2924461

Xie, L., Yuille, A., (2017). Genetic CNN, Computer vision and pattern recognition, arxiv:1703.01513, 2017.

https://doi.org/10.1109/ICCV.2017.154

Al-Oudat, M., et al., (2021). An Interactive automation for human biliary tree diagnosis using computer vision. International Journal of Computers Communications & Control, 16, 5, 4275, https://doi.org/10.15837/ijccc.2021.5.4275, 2021.

https://doi.org/10.15837/ijccc.2021.5.4275

Dumitrach, I et al., (2021). Neuro-inspired framework for cognitive manufacturing control, International Journal of Computers Communications & Control, 16, 6, 4519, https://doi.org/10.15837/ijccc.2021.6.4519, 2021.

https://doi.org/10.15837/ijccc.2021.6.4519

Miikkulainen, R., Liang, J.Z. et al., (2017). Evolving Deep Neural Networks, CoRR, abs/1703.00548, 2017.

Liu, X. et al. (2021). A method based on multiple population genetic algorithm to select hyperparameters of industrial intrusion detection classifier. Studies in Informatics and Control, 30 (3), 39-49, https://doi.org/1024846/v30i3y202104, 2021.

https://doi.org/10.24846/v30i3y202104

Serban, C., Carp, D., (2021). Using genetic algorithm to solve discounted generalized transportation problem, Studies in Informatics and Control, 30 (3), 29-38, https://doi.org/10.24846/v30i3202103, 2021.

https://doi.org/10.24846/v30i3y202103

Gorunescu, F. et al., (2005). An evolutionary computation approach to probabilistic neural networks with application to hepatic cancer diagnosis, 18th IEEE Symposium on computer based medical systems, 461-466, 2005.

Eiben, A.E., Smith, J.E., (2003). Introduction to Evolutionary computing, Berlin, Springer- Verlag, 2003.

https://doi.org/10.1007/978-3-662-05094-1

Haupt, R.L., Haupt, S.E. (2004). Practical genetic algorithms, 2nd ed, UK. John Wiley & Sons, 2004.

https://doi.org/10.1002/0471671746

Puiu, A., et al., (2021). Privacy-preserving and explainable AI for cardiovascular imaging, Studies in Informatics and Control, 30 (2), 21-32, 2021, https://doi.org/10.24846/v30i2y202102, 2021.

https://doi.org/10.24846/v30i2y202102

Gorunescu, F. et al. (2010). A statistical framework for evaluating neural networks to predict recurrent events in breast cancer, J Gen Sys, 39 (5), 471-488, 2010.

https://doi.org/10.1080/03081079.2010.484282

Demsar, J. (2006). Statistical comparisons of classifiers over multiple datasets, Mach Learn Res, 7, 1-30, (2006).

Seltman, H., (2018), Experimental design and analysis. https://stat.cmu/edu/hseltman/309/Book/Book.pdf, 2018.