nach oben

Neural Computing and Applications

Erschienen in:

09.11.2023 | Original Article

Hyperparameter optimization of pre-trained convolutional neural networks using adolescent identity search algorithm

verfasst von: Ebubekir Akkuş, Ufuk Bal, Fatma Önay Koçoğlu, Selami Beyhan

Erschienen in: Neural Computing and Applications | Ausgabe 4/2024

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Convolutional neural networks (CNNs) are widely used deep learning (DL) models for image classification. The selected hyperparameters for training convolutional neural network (CNN) models have a significant effect on the performance. Therefore, hyperparameter optimization (HPO) is an important process to design optimal CNN models. In this study, Adolescent Identity Search Algorithm (AISA) and Bayesian Optimization (BO) methods were applied for HPO of pre-trained CNN models to improve their classification performance. Diabetic retinopathy (DR) classification was chosen as the application problem of the study and Kaggle Diabetic Retinopathy Detection dataset was used. We used pre-trained CNN models named AlexNet, MobileNetV2, ResNet18, and GoogLeNet. To the best of our knowledge, this study represents the first use of AISA-based HPO for DR classification. The results show that hybrid models incorporating AISA-based HPO achieve better accuracy with fewer iterations than BO-based HPO hybridized models.

Vorheriger Artikel Multidimensional relational knowledge embedding for coreference resolution

Nächster Artikel Prediction and classification of minerals using deep residual neural network

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Caballé-Cervigón N, Castillo-Sequera JL, Gómez-Pulido JA, Gómez-Pulido JM, Polo-Luque ML (2020) Machine learning applied to diagnosis of human diseases: a systematic review. Appl Sci 10(15):5135. https://doi.org/10.3390/app10155135CrossRef

Lin WY, Hu YH, Tsai CF (2011) Machine learning in financial crisis prediction: a survey. In: IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 42(4), 421–436. https://doi.org/10.1109/TSMCC.2011.2170420

Gupta VK, Shukla SK, Rawat RS (2022) Crime tracking system and people’s safety in india using machine learning approaches. Int J Modern Res 2(1):1–7

Mladenova T (2021) Machine learning in business: A short overview. In: 2021 5th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), pp. 448–451. https://doi.org/10.1109/ISMSIT52890.2021.9604744.

Frank M, Drikakis D, Charissis V (2020) Machine-learning methods for computational science and engineering. Computation 8(1):15. https://doi.org/10.3390/computation8010015CrossRef

Raji ID, Bello-Salau H, Umoh IJ, Onumanyi AJ, Adegboye MA, Salawudeen AT (2022) Simple deterministic selection-based genetic algorithm for hyperparameter tuning of machine learning models. Appl Sci 12(3):1186. https://doi.org/10.3390/app12031186CrossRef

Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J (2018) Recent advances in convolutional neural networks. Pattern Recognit 77:354–377. https://doi.org/10.1016/j.patcog.2017.10.013CrossRef

Indolia S, Goswami AK, Mishra SP, Asopa P (2018) Conceptual understanding of convolutional neural network-a deep learning approach. Proc Comput Sci 132:679–688. https://doi.org/10.1016/j.procs.2018.05.069CrossRef

Loussaief S, Abdelkrim A (2018) Convolutional neural network hyper-parameters optimization based on genetic algorithms. Int J Adv Comput Sci Appl. https://doi.org/10.14569/IJACSA.2018.091031CrossRef

10.

Ghawi R, Pfeffer J (2019) Efficient hyperparameter tuning with grid search for text categorization using knn approach with bm25 similarity. Open Comput Sci 9(1):160–180. https://doi.org/10.1515/comp-2019-0011CrossRef

11.

Shekar BH, Dagnew G (2019) Grid search-based hyperparameter tuning and classification of microarray cancer data. In: 2019 Second International Conference on Advanced Computational and Communication Paradigms (ICACCP), pp. 1–8. https://doi.org/10.1109/ICACCP.2019.8882943

12.

Probst P, Wright MN, Boulesteix AL (2019) Hyperparameters and tuning strategies for random forest. Wiley interdiscip Rev Data Min Knowl Discov. https://doi.org/10.1002/widm.1301CrossRef

13.

Bergstra J, Bardenet R, Bengio Y, Kégl B (2011) Algorithms for hyper-parameter optimization. Advances in neural information processing systems 24

14.

Mantovani RG, Rossi AL, Vanschoren J, Bischl B, De Carvalho AC (2015) Effectiveness of random search in svm hyper-parameter tuning. In: 2015 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. https://doi.org/10.1109/IJCNN.2015.7280664

15.

Chen L, Su PH, Gasic M (2015) Hyper-parameter optimisation of gaussian process reinforcement learning for statistical dialog management. In: Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and dialog, pp. 407–411. https://doi.org/10.18653/v1/W15-4653

16.

Dernoncourt F, Lee JY (2016) Optimizing neural network hyperparameters with gaussian processes for dialog act classification. In: IEEE Spoken Language Technology Workshop (SLT), pp. 406–413. https://doi.org/10.1109/SLT.2016.7846296

17.

Snoek J, Rippel O, Swersky K, Kiros R, Satish N, Sundaram N, Patwary M, Prabhat M, Adams R (2015) Scalable bayesian optimization using deep neural networks. In: International Conference on Machine Learning, pp. 2171–2180. https://doi.org/10.48550/arXiv.1502.05700

18.

Nagpal M, Kaushal M, Sharma A (2022) A feature reduced intrusion detection system with optimized svm using big bang big crunch optimization. Wirel Pers Commun 122(2):1939–1965. https://doi.org/10.1007/s11277-021-08975-2CrossRef

19.

Fowler MJ (2008) Microvascular and macrovascular complications of diabetes. Clin Diab 26(2):77–82. https://doi.org/10.2337/diaclin.26.2.77CrossRef

20.

Datta P, Das P, Kumar A (2022) Hyper parameter tuning based gradient boosting algorithm for detection of diabetic retinopathy: an analytical review. Bull Electr Eng Inf 11(2):814–824. https://doi.org/10.11591/eei.v11i2.3559CrossRef

21.

Vasireddi HK, Raja Reddy GNV (2022) Deep feed forward neural network-based screening system for diabetic retinopathy severity classification using the lion optimization algorithm. Graefe’s Arch Clin Exp Ophthalmol 260(4):1245–1263. https://doi.org/10.1007/s00417-021-05375-xCrossRef

22.

Abirami A, Kavitha R (2023) A novel automated komodo mlipir optimization-based attention bilstm for early detection of diabetic retinopathy. Signal Image Video Process. https://doi.org/10.1007/s11760-022-02407-9CrossRef

23.

Dayana AM, Emmanuel WRS (2022) An enhanced swarm optimization-based deep neural network for diabetic retinopathy classification in fundus images. Multimed Tools Appl 81(15):20611–20642. https://doi.org/10.1007/s11042-022-12492-0CrossRef

24.

Aljehane NO (2022) An intelligent moth flame optimization with inception network for diabetic retinopathy detection and grading. In: 2nd International Conference on Computing and Information Technology (ICCIT), pp. 370–373. https://doi.org/10.1109/ICCIT52419.2022.9711602

25.

Parthiban K, Kamarasan M (2022) Diabetic retinopathy detection and grading of retinal fundus images using coyote optimization algorithm with deep learning. Multimed Tools Appl. https://doi.org/10.1007/s11042-022-14234-8CrossRef

26.

Cordero-Martínez R, Sánchez D, Melin P (2022) Hierarchical genetic optimization of convolutional neural models for diabetic retinopathy classification. Int J Hybrid Intel Syst. https://doi.org/10.3233/HIS-220004CrossRef

27.

Nancy W, Celine Kavida A (2022) Optimized ensemble machine learning-based diabetic retinopathy grading using multiple region of interest analysis and bayesian approach. J Med Imag Health Inf 12(1):35–44. https://doi.org/10.1166/jmihi.2022.3923CrossRef

28.

Das S, Saha SK (2022) Diabetic retinopathy detection and classification using cnn tuned by genetic algorithm. Multimed Tools Appl 81(6):8007–8020. https://doi.org/10.1007/s11042-021-11824-wCrossRef

29.

Kukkar A, Gupta D, Beram SM, Soni M, Singh NK, Sharma A, Neware R, Shabaz M, Rizwan A (2022) Optimizing deep learning model parameters using socially implemented iomt systems for diabetic retinopathy classification problem. IEEE Trans Comput Soc Syst. https://doi.org/10.1109/TCSS.2022.3213369CrossRef

30.

Ramya J, Rajakumar MP, Maheswari BU (2022) Deep CNN with hybrid binary local search and particle swarm optimizer for exudates classification from fundus images. J Digit Imag 35(1):56–67. https://doi.org/10.1007/s10278-021-00534-2CrossRef

31.

Shankar K, Zhang Y, Liu Y, Wu L, Chen C-H (2020) Hyperparameter tuning deep learning for diabetic retinopathy fundus image classification. IEEE access 8:118164–118173. https://doi.org/10.1109/ACCESS.2020.3005152CrossRef

32.

Dayana AM, Emmanuel WRS (2022) Deep learning enabled optimized feature selection and classification for grading diabetic retinopathy severity in the fundus image. Neural Comput Appl 34(21):18663–18683. https://doi.org/10.1007/s00521-022-07471-3CrossRef

33.

Kaggle: Diabetic retinopathy detection (2015) https://www.kaggle.com/competitions/diabetic-retinopathy-detection/data Accessed 15 Nov 2022

34.

Islam MR, Hasan MAM, Sayeed A (2020) Transfer learning based diabetic retinopathy detection with a novel preprocessed layer. In: 2020 IEEE Region 10 Symposium (TENSYMP), pp. 888–891. https://doi.org/10.1109/tensymp50017.2020.9230648

35.

Park J, Yu W, Aryal P, Ciroski V (2023) Comparative study on transfer learning for object classification and detection. In: AI-enabled Technologies for Autonomous and Connected Vehicles, pp. 125–142. Springer, (2023). https://doi.org/10.1007/978-3-031-06780-8_5

36.

Ismail S, Ismail B, Siddiqi I, Akram U (2023) PCG classification through spectrogram using transfer learning. Biomed Signal Process Control 79:104075. https://doi.org/10.1016/j.bspc.2022.104075CrossRef

37.

Zhang AS, Li NF (2023) A two-stage federated transfer learning framework in medical images classification on limited data: a covid-19 case study. In: Proceedings of the Future Technologies Conference, pp. 198–216. https://doi.org/10.1007/978-3-031-18461-1_13

38.

Talukdar M, Siddika A, Abir AH, Hassan MZ, Hossain MI (2023) Medical x-ray image classification employing dcgan and cnn transfer learning techniques. In: Proceedings of Seventh International Congress on Information and Communication Technology, pp. 839–848. https://doi.org/10.1007/978-981-19-1607-6_74

39.

Ogundokun RO, Maskeliunas R, Misra S, Damaševičius R (2022) Improved cnn based on batch normalization and adam optimizer. In: Computational Science and Its applications–ICCSA 2022 Workshops: Malaga, Spain, July 4–7, 2022, Proceedings, Part V, pp. 593–604. https://doi.org/10.1007/978-3-031-10548-7_43

40.

Öktem M, Akosman SA, Moral T, Kılıç V (2021) Derin ŏğrenme tabanlı mermer yŭzeylerinin otomatik sınıflandırılması. Avrupa Bilim ve Teknoloji Dergisi (26), 73–77. https://doi.org/10.31590/ejosat.950950

41.

Wu J, Chen XY, Zhang H, Xiong LD, Lei H, Deng SH (2019) Hyperparameter optimization for machine learning models based on bayesian optimization. J Electron Sci Technol 17(1):26–40. https://doi.org/10.11989/JEST.1674-862X.80904120CrossRef

42.

Yang L, Shami A (2020) On hyperparameter optimization of machine learning algorithms: theory and practice. Neurocomputing 415:295–316. https://doi.org/10.1016/j.neucom.2020.07.061CrossRef

43.

Kim H-C, Kang M-J (2020) Comparison of hyper-parameter optimization methods for deep neural networks. J IKEEE 24(4):969–974. https://doi.org/10.7471/ikeee.2020.24.4.969CrossRef

44.

Lee S, Bae JH, Hong J, Yang D, Panagos P, Borrelli P, Yang JE, Kim J, Lim KJ (2022) Estimation of rainfall erosivity factor in Italy and Switzerland using Bayesian optimization based machine learning models. Catena 211:105957. https://doi.org/10.1016/j.catena.2021.105957CrossRef

45.

Gupta P, Saini DK, Rawat PS, Bhagat S (2022) Hybrid big bang-big crunch based resource scheduling to improve qos in cloud infrastructure. J Intell Fuzzy Systems. https://doi.org/10.3233/JIFS-219290CrossRef

46.

Bogar E, Beyhan S (2020) Adolescent identity search algorithm (AISA): a novel metaheuristic approach for solving optimization problems. Appl Soft Comput 95:106503. https://doi.org/10.1016/j.asoc.2020.106503CrossRef

47.

Injadat M, Salo F, Nassif AB, Essex A, Shami A (2018) Bayesian optimization with machine learning algorithms toward anomaly detection. In: IEEE Global Communications Conference (GLOBECOM), pp. 1–6. https://doi.org/10.1109/GLOCOM.2018.8647714

48.

Wang K, Dowling AW (2022) Bayesian optimization for chemical products and functional materials. Curr Opin Chem Eng 36:100728. https://doi.org/10.1016/j.coche.2021.100728CrossRef

49.

Yin J, Li N (2022) Ensemble learning models with a bayesian optimization algorithm for mineral prospectivity mapping. Ore geology reviews 145:104916. https://doi.org/10.1016/j.oregeorev.2022.104916CrossRef

50.

Sultana N, Hossain SMZ, Abusaad M, Alanbar N, Senan Y, Razzak SA (2022) Prediction of biodiesel production from microalgal oil using bayesian optimization algorithm-based machine learning approaches. Fuel 309:122184. https://doi.org/10.1016/j.fuel.2021.122184CrossRef

51.

Aslan MF, Sabancı K, Durdu A, Unlersen MF (2022) Covid-19 diagnosis using state-of-the-art cnn architecture features and bayesian optimization. Computers in biology and medicine, https://doi.org/10.1016/j.compbiomed.2022.105244Get rights and content

52.

Hellan SP, Lucas CG, Goddard NH (2022) Bayesian optimisation for active monitoring of air pollution. CoRR arXiv: 2202.07595

53.

Lekouaghet B, Khelifa MA, Boukabou A (2022) Adolescent identity search algorithm for parameter extraction in photovoltaic solar cells and modules. J Comput Electron. https://doi.org/10.1007/s10825-022-01881-1CrossRef

54.

Patra JC, Kot AC (2002) Nonlinear dynamic system identification using chebyshev functional link artificial neural networks. In: IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 32(4), 505–511. https://doi.org/10.1109/doi.org/TSMCB.2002.1018769

55.

Snoek J, Larochelle H, Adams RP (2012) Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems 25. https://doi.org/10.48550/arXiv.1206.2944

56.

Kramer O, Ciaurri D, Koziel S (2011) Derivative-free optimization. Comput Optim Methods Algorithms 356:61–83. https://doi.org/10.1007/978-3-642-20859-1_4MathSciNetCrossRef

57.

Victoria AH, Maragatham G (2021) Automatic tuning of hyperparameters using Bayesian optimization. Evolv Syst 12(1):217–223. https://doi.org/10.1007/s12530-020-09345-2CrossRef

58.

Yazıcı K, Boran S (2021) Short-term wind power prediction approach based on bayesian optimization and ensemble learning. Journal of intelligent systems Theory and applications 4(2), 142–154. https://doi.org/10.38016/jista.889991

59.

Frazier PI (2018) A tutorial on bayesian optimization. CoRR arXiv:1807.02811

60.

Akçay MT, Akgündoğdu A, Tiryaki H (2021) Estimation of the average speed for a railway signaling system by using gaussian process regression methods with bayesian optimization. Demiryolu mühendisliği (14), 274–286. https://doi.org/10.47072/demiryolu.942730

61.

Qu Y, Crupi G, Cai J (2022) A broadband pa design based on bayesian optimization augmented by dynamic feasible region shrinkage. IEEE microwave and wireless components letters. https://doi.org/10.1109/LMWC.2022.3173441CrossRef

62.

Lizotte DJ (2008) Practical bayesian optimization. CoRR arXiv:1206.2944

63.

Sara A, Talu MF (2021) Derin sinir ağları için hiperparametre metodlarının ve kitlerinin incelenmesi. Dicle Üniversitesi Mühendislik Fakültesi Mühendislik Dergisi 12(2), 187–199. https://doi.org/10.24012/dumf.767700

64.

Aszemi NM, Dominic P (2019) Hyperparameter optimization in convolutional neural network using genetic algorithms. Int J Adv Comput Sci Appl. https://doi.org/10.14569/IJACSA.2019.0100638CrossRef

65.

Atef S, Eltawil AB (2020) Assessment of stacked unidirectional and bidirectional long short-term memory networks for electricity load forecasting. Electr Power Syst Res 187:106489. https://doi.org/10.1016/j.epsr.2020.106489CrossRef

66.

Raponi S, Oligeri G, Ali IM (2022) Sound of guns: digital forensics of gun audio samples meets artificial intelligence. Multimed Tools Appl 81(21):30387–30412. https://doi.org/10.1007/s11042-022-12612-wCrossRef

67.

MathWorks: MATLAB R2019b (2019). https://www.mathworks.com/products/matlab.html

68.

Microsoft Corporation: Microsoft Windows 11 (Version 21H2). (2021). https://www.microsoft.com/windows/

69.

Alibrahim H, Ludwig SA (2021) Hyperparameter optimization: Comparing genetic algorithm against grid search and bayesian optimization. In: IEEE Congress on Evolutionary Computation (CEC), pp. 1551–1559. https://doi.org/10.1109/CEC45853.2021.9504761

70.

Gour N, Tanveer M, Khanna P (2022) Challenges for ocular disease identification in the era of artificial intelligence. Neural Computing and Applications, 1–23. https://doi.org/10.1007/s00521-021-06770-5

71.

Poulinakis K, Drikakis D, Kokkinakis IW, Spottswood SM (2023) Machine-learning methods on noisy and sparse data. Mathematics 11(1):236. https://doi.org/10.3390/math11010236CrossRef

Titel: Hyperparameter optimization of pre-trained convolutional neural networks using adolescent identity search algorithm
verfasst von: Ebubekir Akkuş
Ufuk Bal
Fatma Önay Koçoğlu
Selami Beyhan
Publikationsdatum: 09.11.2023
Verlag: Springer London
Erschienen in: Neural Computing and Applications / Ausgabe 4/2024
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-023-09121-8

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Weitere Artikel der Ausgabe 4/2024

Classification of microscopic peripheral blood cell images using multibranch lightweight CNN-based model

Fault distance estimation for transmission lines with dynamic regressor selection

A new framework for early diagnosis of breast cancer using mammography images

Faster and efficient tetrahedral mesh generation using generator neural networks for 2D and 3D geometries

Proposed methodology for gait recognition using generative adversarial network with different feature selectors

An intelligent machine learning-enabled cattle reclining risk mitigation technique using surveillance videos

Premium Partner