Skip to main content
SpringerLink
Log in

PSO + GWO: a hybrid particle swarm optimization and Grey Wolf optimization based Algorithm for fine-tuning hyper-parameters of convolutional neural networks for Cardiovascular Disease Detection

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Cardiovascular diseases are one of the most common health problems worldwide. In this study, a hybrid model is proposed to predict cardiovascular diseases using optimization and deep learning methods. Tuning hyper-parameters of deep learning algorithms is important in the learning process. Therefore, numerous studies on hyper-parameter optimization have been proposed in the literature to improve the performance of deep learning algorithms. In this study, particle swarm optimization (PSO), cat swarm optimization (CSO), and the proposed hybrid of PSO and grey wolf optimization (GWO) algorithms are used for the optimization of the hyper-parameters of the 1D-VGG-16 model. In the proposed study, first, the control parameters of the optimization methods used were fine-tuned. Second, the hyper-parameters of the deep learning method were decided with the help of the optimization methods. Among the optimization methods used, the best success for the 1D-VGG-16 model was achieved with the hybrid PSO + GWO optimization algorithm. To further support the proposed study, the hyper-parameter optimization of the 2D-VGG-16 architecture was carried out with the PSO, CSO, and PSO + GWO hybrid optimization algorithms using the MNIST data set.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Data Availability

The datasets generated during and/or analysed during the current study are available in the [UCI Machine Learning] repository, [https://archive.ics.uci.edu/ml/datasets/heart+disease].

References

  • Adem K, Kilicarslan S, Cömert O (2019) Classification and diagnosis of cervical cancer with softmax classification with stacked autoencoder. Expert Syst Appl 115:557–564

    Article  Google Scholar 

  • Aufa BZ, Suyanto S, Arifianto A (2020), August Hyperparameter setting of LSTM-based language model using grey wolf optimizer. In 2020 International Conference on Data Science and Its Applications (ICoDSA) (pp. 1–5). IEEE

  • Bacanin N, Bezdan T, Tuba E, Strumberger I, Tuba M (2020) Optimizing Convolutional Neural Network Hyperparameters by Enhanced Swarm Intelligence Metaheuristics. Algorithms 13(3):67

    Article  MathSciNet  Google Scholar 

  • Baldominos A, Saez Y, Isasi P (2018) Evolutionary convolutional neural networks: An application to handwriting recognition. Neurocomputing 283:38–52

    Article  Google Scholar 

  • Bengio Y (2009) Learning deep architectures for AI. Found Trends Mach Learn 2(1):1–127

    Article  MathSciNet  MATH  Google Scholar 

  • Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13(Feb):281–305

    MathSciNet  MATH  Google Scholar 

  • Bochinski E, Senst T, Sikora T (2017), September Hyper-parameter optimization for convolutional neural network committees based on evolutionary algorithms. In 2017 IEEE International Conference on Image Processing (ICIP) (pp. 3924–3928)

  • Chang X, Nie F, Wang S, Yang Y, Zhou X, Zhang C (2015) Compound rank-$ k $ projections for bilinear analysis. IEEE Trans neural networks Learn Syst 27(7):1502–1513

    Article  MathSciNet  Google Scholar 

  • Chen K, Yao L, Zhang D, Wang X, Chang X, Nie F (2019) A semisupervised recurrent convolutional attention model for human activity recognition. IEEE Trans neural networks Learn Syst 31(5):1747–1756

    Article  Google Scholar 

  • Chen W, Gou S, Wang X, Li X, Jiao L (2018) Classification of PolSAR Images Using Multilayer Autoencoders and a Self-Paced Learning Approach. Remote Sens 10(1):1–17. doi: https://doi.org/10.3390/rs10010110

    Article  Google Scholar 

  • Chu SC, Tsai PW, Pan JS (2006), August Cat swarm optimization. In Pacific Rim international conference on artificial intelligence (pp. 854–858). Springer, Berlin, Heidelberg

  • Ciresan DC, Meier U, Gambardella LM, Schmidhuber J (2010) Deep big simple neural nets for handwritten digit recognition. Neural Comput 22:3207–3220

    Article  Google Scholar 

  • da Silva GLF, Valente TLA, Silva AC, de Paiva AC, Gattass M (2018) Convolutional neural network-based PSO for lung nodule false positive reduction on CT images. Comput Methods Programs Biomed 162:109–118

    Article  Google Scholar 

  • da Silva GL, da Silva Neto OP, Silva AC, de Paiva AC, Gattass M (2017) Lung nodules diagnosis based on evolutionary convolutional neural network. Multimedia Tools and Applications 76(18):19039–19055

    Article  Google Scholar 

  • Detrano R, Janosi A, Steinbrunn W, Pfisterer M, Schmid JJ, Sandhu S, Froelicher V (1989) International application of a new probability algorithm for the diagnosis of coronary artery disease. Am J Cardiol 64(5):304–310

    Article  Google Scholar 

  • Eberhart RC, Shi Y (2000), July Comparing inertia weights and constriction factors in particle swarm optimization. In Proceedings of the 2000 congress on evolutionary computation. CEC00 (Cat. No. 00TH8512) (Vol. 1, pp. 84–88)

  • Eberhart R, Kennedy J (1995), October A new optimizer using particle swarm theory. In MHS’95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science (pp. 39–43)

  • Fujino S, Mori N, Matsumoto K (2017), June Deep convolutional networks for human sketches by means of the evolutionary deep learning. In 2017 Joint 17th World Congress of International Fuzzy Systems Association and 9th International Conference on Soft Computing and Intelligent Systems (IFSA-SCIS) (pp. 1–5)

  • Gülcü A, Kuş Z (2019) A Survey of Hyper-parameter Optimization Methods in Convolutional Neural Networks. Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım ve Teknoloji 7(2):503–522

    Article  Google Scholar 

  • Guo B, Hu J, Wu W, Peng Q, Wu F (2019) The Tabu_Genetic Algorithm: A Novel Method for Hyper-Parameter Optimization of Learning Algorithms. Electronics 8(5):579

    Article  Google Scholar 

  • Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167.

  • Kilicarslan S, Adem K, Celik M (2020) Diagnosis and classification of cancer using hybrid model based on ReliefF and convolutional neural network. Med Hypotheses 137:109577

    Article  Google Scholar 

  • Kılıçarslan S, Adem K, Cömert O (2019) Dimension Using Particle Swarm Optimization. Düzce Üniversitesi Bilim ve Teknoloji Dergisi 7(1):769–777Prostate Cancer Diagnosis With Machine Learning Methods On Microarray Data Reduced

  • Kilicarslan S, Celik M, Sahin Ş (2021) Hybrid Models based on Genetic Algorithm and Deep Learning Algorithms for Nutritional Anemia Disease Classification. Biomed Signal Process Control 63:102231. doi: https://doi.org/10.1016/j.bspc.2020.102231

    Article  Google Scholar 

  • LeCun B, Boser JS, Denker D, Henderson RE, Howard W, Hubbard, Jackel LD (1990) “Handwritten Digit Recognition with a Back-Propagation Network”, In Advances in Neural Information Processing Systems

  • LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324

  • Li L, Jamieson K, DeSalvo G, Rostamizadeh A, Talwalkar A (2017) Hyperband: A novel bandit-based approach to hyperparameter optimization. J Mach Learn Res 18(1):6765–6816

    MathSciNet  MATH  Google Scholar 

  • Li Z, Nie F, Chang X, Nie L, Zhang H, Yang Y (2018a) Rank-constrained spectral clustering with flexible embedding. IEEE Trans neural networks Learn Syst 29(12):6073–6082

    Article  MathSciNet  Google Scholar 

  • Li Z, Nie F, Chang X, Yang Y, Zhang C, Sebe N (2018b) Dynamic affinity graph construction for spectral clustering using multiple features. IEEE Trans neural networks Learn Syst 29(12):6323–6332

    Article  MathSciNet  Google Scholar 

  • Li Z, Yao L, Chang X, Zhan K, Sun J, Zhang H (2019) Zero-shot event detection via event-adaptive concept relevance mining. Pattern Recogn 88:595–603

    Article  Google Scholar 

  • Liashchynskyi P, Liashchynskyi P (2019) Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS. arXiv preprint arXiv:1912.06059

  • Lopez-Rincon A, Tonda A, Elati M, Schwander O, Piwowarski B, Gallinari P (2018) Evolutionary optimization of convolutional neural networks for cancer miRNA biomarkers classification. Appl Soft Comput 65:91–100

    Article  Google Scholar 

  • Loshchilov I, Hutter F (2016) CMA-ES for hyperparameter optimization of deep neural networks. arXiv preprint arXiv:1604.07269

  • Luo M, Chang X, Nie L, Yang Y, Hauptmann AG, Zheng Q (2017) An adaptive semisupervised feature analysis for video semantic recognition. IEEE Trans cybernetics 48(2):648–660

    Article  Google Scholar 

  • Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61

    Article  Google Scholar 

  • Mohakud R, Dash R (2021) Designing a grey wolf optimization based hyper-parameter optimized convolutional neural network classifier for skin cancer detection. Journal of King Saud University-Computer and Information Sciences

  • Nalbant S, Karan MA (2010) İç Hastalıkları Uzmanının Anemiye Yaklaşımı Rehberi. İç Hastalıkları Dergisi 17:7–15

    Google Scholar 

  • Nalepa J, Lorenzo PR (2017), November Convergence Analysis of PSO for Hyper-Parameter Selection in Deep Neural Networks. In International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (pp. 284–295). Springer, Cham

  • Pacal I, Karaboga D (2021) A robust real-time deep learning based automatic polyp detection system. Comput Biol Med 134:104519

    Article  Google Scholar 

  • Pacal I, Karaman A, Karaboga D, Akay B, Basturk A, Nalbantoglu U, Coskun S (2022) An efficient real-time colonic polyp detection with YOLO algorithms trained by using negative samples and large datasets. Comput Biol Med 141:105031

    Article  Google Scholar 

  • Pan J, Jing B, Jiao X, Wang S (2020) Analysis and application of grey wolf optimizer-long short-term memory. IEEE Access 8:121460–121468

    Article  Google Scholar 

  • Pınar M, Okumuş O, Turgut UO, Kalıpsız O, Aktaş MS (2017) Büyük veri içeren öneri sistemleri için hiperparametre optimizasyonu. Ulusal Yazılım Mühendisliği Sempozyumu, pp 22–272

  • Poli R, Kennedy J, Blackwell T (2007) Particle swarm optimization. Swarm Intell 1(1):33–57

    Article  Google Scholar 

  • Stang M, Meier C, Rau V, Sax E (2019), August An Evolutionary Approach to Hyper-Parameter Optimization of Neural Networks. In International Conference on Human Interaction and Emerging Technologies (pp. 713–718). Springer, Cham

  • Sun Y, Xue B, Zhang M, Yen GG (2018), July An experimental study on hyper-parameter optimization for stacked auto-encoders. In 2018 IEEE Congress on Evolutionary Computation (CEC) (pp. 1–8)

  • Tan TY, Zhang L, Lim CP (2019) Intelligent skin cancer diagnosis using improved particle swarm optimization and deep learning models. Appl Soft Comput 84:105725

    Article  Google Scholar 

  • Tanyıldızı E, Demirtaş F (2019), November Hiper Parametre Optimizasyonu Hyper Parameter Optimization. In 2019 1st International Informatics and Software Engineering Conference (UBMYK) (pp. 1–5)

  • Wang B, Sun Y, Xue B, Zhang M (2018a), December A hybrid differential evolution approach to designing deep convolutional neural networks for image classification. In Australasian Joint Conference on Artificial Intelligence (pp. 237–250). Springer, Cham

  • Wang B, Sun Y, Xue B, Zhang M (2018b), July Evolving deep convolutional neural networks by variable-length particle swarm optimization for image classification. In 2018b IEEE Congress on Evolutionary Computation (CEC) (pp. 1–8)

  • Wright J, Alajmi A (2016) Efficient Genetic Algorithm sets for optimizing constrained building design problem. Int J Sustainable Built Environ 5(1):123–131

    Article  Google Scholar 

  • Yamasaki T, Honma T, Aizawa K (2017), April Efficient optimization of convolutional neural networks using particle swarm optimization. In 2017 IEEE Third International Conference on Multimedia Big Data (BigMM) (pp. 70–73)

  • Yan C, Chang X, Luo M, Zheng Q, Zhang X, Li Z, Nie F (2020) Self-weighted robust LDA for multiclass classification with edge classes. ACM Trans Intell Syst Technol (TIST) 12(1):1–19

    Google Scholar 

  • Yu E, Ma J, Sun J, Chang X, Zhang H, Hauptmann AG (2022) Deep Discrete Cross-Modal Hashing with Multiple Supervision. Neurocomputing 486:215–224

    Article  Google Scholar 

  • Yuan D, Chang X, Li Z, He Z (2022) Learning adaptive spatial-temporal context-aware correlation filters for UAV tracking. ACM Trans Multimedia Comput Commun Appl (TOMM) 18(3):1–18

    Article  Google Scholar 

  • Zhang D, Yao L, Chen K, Wang S, Chang X, Liu Y (2019) Making sense of spatio-temporal preserving representations for EEG-based human intention recognition. IEEE Trans cybernetics 50(7):3033–3044

    Article  Google Scholar 

  • Zhou R, Chang X, Shi L, Shen YD, Yang Y, Nie F (2019) Person reidentification via multi-feature fusion with adaptive graph learning. IEEE Trans neural networks Learn Syst 31(5):1592–1601

    Article  Google Scholar 

  • Zhu W, Yeh W, Chen J, Chen D, Li A, Lin Y (2019), February Evolutionary Convolutional Neural Networks Using ABC. In Proceedings of the 2019 11th International Conference on Machine Learning and Computing (pp. 156–162)

Download references

Author information

Authors and Affiliations

  1. Dept. of Software Engineering, Bandırma Onyedi Eylul University, 10250, Balıkesir, Turkey

    Serhat KILIÇARSLAN

Authors
  1. Serhat KILIÇARSLAN
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Serhat KILIÇARSLAN.

Ethics declarations

Conflict of interest

We, the authors, have no conflict of interest and disclose any financial and personal relationships with other people or organizations that could inappropriately influence the presented work.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary Material 2

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

KILIÇARSLAN, S. PSO + GWO: a hybrid particle swarm optimization and Grey Wolf optimization based Algorithm for fine-tuning hyper-parameters of convolutional neural networks for Cardiovascular Disease Detection. J Ambient Intell Human Comput 14, 87–97 (2023). https://doi.org/10.1007/s12652-022-04433-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-022-04433-4

Keywords

Search

Navigation