Skip to main content
Top
Published in: Soft Computing 10/2020

30-09-2019 | Methodologies and Application

A fast parallel genetic programming framework with adaptively weighted primitives for symbolic regression

Authors: Zhixing Huang, Jinghui Zhong, Liang Feng, Yi Mei, Wentong Cai

Published in: Soft Computing | Issue 10/2020

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Genetic programming (GP) is a popular and powerful optimization algorithm that has a wide range of applications, such as time series prediction, classification, data mining, and knowledge discovery. Despite the great success it enjoyed, selecting the proper primitives from high-dimension primitive set for GP to construct solutions is still a time-consuming and challenging issue that limits the efficacy of GP in real-world applications. In this paper, we propose a multi-population GP framework with adaptively weighted primitives to address the above issues. In the proposed framework, the entire population consists of several sub-populations and each has a different vector of primitive weights to determine the probability of using the corresponding primitives in a sub-population. By adaptively adjusting the weights of the primitives and periodically sharing information between sub-populations, the proposed framework can efficiently identify important primitives to assist the search. Furthermore, based on the proposed framework and the graphics processing unit computing technique, a high-performance self-learning gene expression programming algorithm (HSL-GEP) is developed. The HSL-GEP is tested on fifteen problems, including four real-world problems. The experimental results have demonstrated that the proposed HSL-GEP outperforms several state-of-the-art GPs, in terms of both solution quality and search efficiency.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
go back to reference Ahmad F, Isa NAM, Hussain Z, Osman MK, Sulaiman SN (2015) A ga-based feature selection and parameter optimization of an ann in diagnosing breast cancer. Pattern Anal Appl 18(4):861–870MathSciNetCrossRef Ahmad F, Isa NAM, Hussain Z, Osman MK, Sulaiman SN (2015) A ga-based feature selection and parameter optimization of an ann in diagnosing breast cancer. Pattern Anal Appl 18(4):861–870MathSciNetCrossRef
go back to reference Ahmed S, Zhang M, Peng L (2013) Enhanced feature selection for biomarker discovery in LC-MS data using GP. In: IEEE congress on evolutionary computation (CEC), pp 584–591 Ahmed S, Zhang M, Peng L (2013) Enhanced feature selection for biomarker discovery in LC-MS data using GP. In: IEEE congress on evolutionary computation (CEC), pp 584–591
go back to reference Antonio LM, Coello CCA (2018) Coevolutionary multiobjective evolutionary algorithms: survey of the state-of-the-art. IEEE Trans Evol Comput 22(6):851–865CrossRef Antonio LM, Coello CCA (2018) Coevolutionary multiobjective evolutionary algorithms: survey of the state-of-the-art. IEEE Trans Evol Comput 22(6):851–865CrossRef
go back to reference Banzhaf W, Harding S, Langdon WB, Wilson G (2008) Accelerating genetic programming through graphics processing units. In: Genetic programming theory and practice VI, pp 1–19 Banzhaf W, Harding S, Langdon WB, Wilson G (2008) Accelerating genetic programming through graphics processing units. In: Genetic programming theory and practice VI, pp 1–19
go back to reference Brameier MF, Banzhaf W (2007) Linear genetic programming. Springer, BerlinMATH Brameier MF, Banzhaf W (2007) Linear genetic programming. Springer, BerlinMATH
go back to reference Cano A, Ventura S (2014) Gpu-parallel subtree interpreter for genetic programming. In: Conference on genetic and evolutionary computation, pp 887–894 Cano A, Ventura S (2014) Gpu-parallel subtree interpreter for genetic programming. In: Conference on genetic and evolutionary computation, pp 887–894
go back to reference Chen B, Chen B, Liu H, Zhang X (2015) A fast parallel genetic algorithm for graph coloring problem based on CUDA. In: International conference on cyber-enabled distributed computing and knowledge discovery, pp 145–148 Chen B, Chen B, Liu H, Zhang X (2015) A fast parallel genetic algorithm for graph coloring problem based on CUDA. In: International conference on cyber-enabled distributed computing and knowledge discovery, pp 145–148
go back to reference Chen Q, Xue B, Niu B, Zhang M (2016) Improving generalisation of genetic programming for high-dimensional symbolic regression with feature selection. In Congress on evolutionary computation (CEC), pp 3793–3800 Chen Q, Xue B, Niu B, Zhang M (2016) Improving generalisation of genetic programming for high-dimensional symbolic regression with feature selection. In Congress on evolutionary computation (CEC), pp 3793–3800
go back to reference Chen Q, Zhang M, Xue B (2017) Feature selection to improve generalization of genetic programming for high-dimensional symbolic regression. IEEE Trans Evol Comput 21(5):792–806CrossRef Chen Q, Zhang M, Xue B (2017) Feature selection to improve generalization of genetic programming for high-dimensional symbolic regression. IEEE Trans Evol Comput 21(5):792–806CrossRef
go back to reference Chitty DM (2016b) Improving the performance of gpu-based genetic programming through exploitation of on-chip memory. Soft Comput 20(2):661–680CrossRef Chitty DM (2016b) Improving the performance of gpu-based genetic programming through exploitation of on-chip memory. Soft Comput 20(2):661–680CrossRef
go back to reference Cook S (2012) CUDA programming: a developer’s guide to parallel computing with GPUs. Elsevier, London Cook S (2012) CUDA programming: a developer’s guide to parallel computing with GPUs. Elsevier, London
go back to reference Deng W, Zhao H, Zou L, Li G, Yang X, Wu D (2017) A novel collaborative optimization algorithm in solving complex optimization problems. Soft Comput 21(15):4387–4398CrossRef Deng W, Zhao H, Zou L, Li G, Yang X, Wu D (2017) A novel collaborative optimization algorithm in solving complex optimization problems. Soft Comput 21(15):4387–4398CrossRef
go back to reference Deng W, Yao R, Zhao H, Yang X, Li G (2019a) A novel intelligent diagnosis method using optimal LS-SVM with improved pso algorithm. Soft Comput 23(7):2445–2462CrossRef Deng W, Yao R, Zhao H, Yang X, Li G (2019a) A novel intelligent diagnosis method using optimal LS-SVM with improved pso algorithm. Soft Comput 23(7):2445–2462CrossRef
go back to reference Deng W, Xu J, Zhao H (2019b) An improved ant colony optimization algorithm based on hybrid strategies for scheduling problem. IEEE Access 7:20281–20292CrossRef Deng W, Xu J, Zhao H (2019b) An improved ant colony optimization algorithm based on hybrid strategies for scheduling problem. IEEE Access 7:20281–20292CrossRef
go back to reference Dick G, Rimoni AP, Whigham PA (2015) A re-examination of the use of genetic programming on the oral bioavailability problem. In: Proceedings of the genetic and evolutionary computation conference (GECCO) Dick G, Rimoni AP, Whigham PA (2015) A re-examination of the use of genetic programming on the oral bioavailability problem. In: Proceedings of the genetic and evolutionary computation conference (GECCO)
go back to reference Espejo PG, Ventura S, Herrera F (2010) A survey on the application of genetic programming to classification. IEEE Trans Syst Man Cybern C Appl Rev 40(2):121–144CrossRef Espejo PG, Ventura S, Herrera F (2010) A survey on the application of genetic programming to classification. IEEE Trans Syst Man Cybern C Appl Rev 40(2):121–144CrossRef
go back to reference Ffrancon R, Schoenauer M (2015) Memetic semantic genetic programming. In: Proceedings of the 2015 annual conference on genetic and evolutionary computation. ACM, pp 1023–1030 Ffrancon R, Schoenauer M (2015) Memetic semantic genetic programming. In: Proceedings of the 2015 annual conference on genetic and evolutionary computation. ACM, pp 1023–1030
go back to reference Gandomi AH, Sajedi S, Kiani B, Huang Q (2016) Genetic programming for experimental big data mining: a case study on concrete creep formulation. Autom Constr 70:89–97CrossRef Gandomi AH, Sajedi S, Kiani B, Huang Q (2016) Genetic programming for experimental big data mining: a case study on concrete creep formulation. Autom Constr 70:89–97CrossRef
go back to reference Gu S, Cheng R, Jin Y (2018) Feature selection for high-dimensional classification using a competitive swarm optimizer. Soft Comput 22(3):811–822CrossRef Gu S, Cheng R, Jin Y (2018) Feature selection for high-dimensional classification using a competitive swarm optimizer. Soft Comput 22(3):811–822CrossRef
go back to reference Hancer E, Xue B, Zhang M, Karaboga D, Akay B (2018) Pareto front feature selection based on artificial bee colony optimization. Inf Sci 422:462–479CrossRef Hancer E, Xue B, Zhang M, Karaboga D, Akay B (2018) Pareto front feature selection based on artificial bee colony optimization. Inf Sci 422:462–479CrossRef
go back to reference Harding S, Banzhaf W (2007) Fast genetic programming and artificial developmental systems on GPUs. In: 21st International symposium on high performance computing systems and applications, 2007. HPCS 2007, p 2 Harding S, Banzhaf W (2007) Fast genetic programming and artificial developmental systems on GPUs. In: 21st International symposium on high performance computing systems and applications, 2007. HPCS 2007, p 2
go back to reference Harding S, Banzhaf W (2007) Fast genetic programming on GPUs. In: European conference on genetic programming, pp 90–101 Harding S, Banzhaf W (2007) Fast genetic programming on GPUs. In: European conference on genetic programming, pp 90–101
go back to reference Harvey DY, Todd MD (2015) Automated feature design for numeric sequence classification by genetic programming. IEEE Trans Evol Comput 19(4):474–489CrossRef Harvey DY, Todd MD (2015) Automated feature design for numeric sequence classification by genetic programming. IEEE Trans Evol Comput 19(4):474–489CrossRef
go back to reference Keijzer M (2003) Improving symbolic regression with interval arithmetic and linear scaling. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), vol 2610. Essex, UK, pp 70–82 Keijzer M (2003) Improving symbolic regression with interval arithmetic and linear scaling. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), vol 2610. Essex, UK, pp 70–82
go back to reference Langdon WB (2010) A many threaded CUDA interpreter for genetic programming. Springer, BerlinCrossRef Langdon WB (2010) A many threaded CUDA interpreter for genetic programming. Springer, BerlinCrossRef
go back to reference Langdon WB (2011) Graphics processing units and genetic programming: an overview. Soft Comput 15(8):1657–1669CrossRef Langdon WB (2011) Graphics processing units and genetic programming: an overview. Soft Comput 15(8):1657–1669CrossRef
go back to reference McDermott J, White DR., Luke S, Manzoni L, Castelli M, Vanneschi L, Jaskowski W, Krawiec K, Harper R, De Jong K, O’Reilly U-M (2012) Genetic programming needs better benchmarks. In: Proceedings of the 14th international conference on genetic and evolutionary computation, GECCO’12, pp 791–798 McDermott J, White DR., Luke S, Manzoni L, Castelli M, Vanneschi L, Jaskowski W, Krawiec K, Harper R, De Jong K, O’Reilly U-M (2012) Genetic programming needs better benchmarks. In: Proceedings of the 14th international conference on genetic and evolutionary computation, GECCO’12, pp 791–798
go back to reference Mei Y, Omidvar MN, Li X, Yao X (2016a) A competitive divide-and-conquer algorithm for unconstrained large-scale black-box optimization. ACM Trans Math Softw 42(2):13MathSciNetCrossRef Mei Y, Omidvar MN, Li X, Yao X (2016a) A competitive divide-and-conquer algorithm for unconstrained large-scale black-box optimization. ACM Trans Math Softw 42(2):13MathSciNetCrossRef
go back to reference Mei Y, Zhang M, Nguyen S (2016b) Feature selection in evolving job shop dispatching rules with genetic programming. In: Proceedings of the genetic and evolutionary computation conference (GECCO). ACM, pp 365–372 Mei Y, Zhang M, Nguyen S (2016b) Feature selection in evolving job shop dispatching rules with genetic programming. In: Proceedings of the genetic and evolutionary computation conference (GECCO). ACM, pp 365–372
go back to reference Mei Y, Su N, Xue B, Zhang M (2017) An efficient feature selection algorithm for evolving job shop scheduling rules with genetic programming. IEEE Trans Emerg Top Comput Intell 1(5):339–353CrossRef Mei Y, Su N, Xue B, Zhang M (2017) An efficient feature selection algorithm for evolving job shop scheduling rules with genetic programming. IEEE Trans Emerg Top Comput Intell 1(5):339–353CrossRef
go back to reference Miller JF, Thomson P (2000) Cartesian genetic programming. In: Genetic programming. Springer, Berlin, pp 121–132 Miller JF, Thomson P (2000) Cartesian genetic programming. In: Genetic programming. Springer, Berlin, pp 121–132
go back to reference Moore JH, Hill DP, Saykin A, Shen L (2013) Exploring interestingness in a computational evolution system for the genome-wide genetic analysis of Alzheimer’s Disease. Springer, New York Moore JH, Hill DP, Saykin A, Shen L (2013) Exploring interestingness in a computational evolution system for the genome-wide genetic analysis of Alzheimer’s Disease. Springer, New York
go back to reference Moraglio A, Krawiec K, Johnson CG (2012) Geometric semantic genetic programming. In: International conference on parallel problem solving from nature. Springer, Berlin, pp 21–31 Moraglio A, Krawiec K, Johnson CG (2012) Geometric semantic genetic programming. In: International conference on parallel problem solving from nature. Springer, Berlin, pp 21–31
go back to reference Neshatian K, Zhang M (2009) Pareto front feature selection: using genetic programming to explore feature space. In: Proceedings of the genetic and evolutionary computation conference, GECCO 2009. Montreal, Québec, Canada, pp 1027–1034 Neshatian K, Zhang M (2009) Pareto front feature selection: using genetic programming to explore feature space. In: Proceedings of the genetic and evolutionary computation conference, GECCO 2009. Montreal, Québec, Canada, pp 1027–1034
go back to reference O’Neill M, Ryan C (2001) Grammatical evolution. IEEE Trans Evol Comput 5(4):349–358CrossRef O’Neill M, Ryan C (2001) Grammatical evolution. IEEE Trans Evol Comput 5(4):349–358CrossRef
go back to reference Riley M, Mei Y, Zhang M (2016) Improving job shop dispatchingrules via terminal weighting and adaptive mutation in genetic programming. Vancouver, BC, Canada, pp 3362 – 3369 Riley M, Mei Y, Zhang M (2016) Improving job shop dispatchingrules via terminal weighting and adaptive mutation in genetic programming. Vancouver, BC, Canada, pp 3362 – 3369
go back to reference Rojas F, Meza F (2015) A parallel distributed genetic algorithm for the prize collecting steiner tree problem. In: International conference on computational science and computational intelligence (CSCI), pp. 643–646 Rojas F, Meza F (2015) A parallel distributed genetic algorithm for the prize collecting steiner tree problem. In: International conference on computational science and computational intelligence (CSCI), pp. 643–646
go back to reference Rosenwald A, Wright G, Chan WC, Connors JM, Campo E, Fisher RI, Gascoyne RD, Muller-Hermelink HK, Smeland EB, Giltnane JM (2002) The use of molecular profiling to predict survival after chemotherapy for diffuse large-b-cell lymphoma. New Engl J Med 346(25):1937–1947CrossRef Rosenwald A, Wright G, Chan WC, Connors JM, Campo E, Fisher RI, Gascoyne RD, Muller-Hermelink HK, Smeland EB, Giltnane JM (2002) The use of molecular profiling to predict survival after chemotherapy for diffuse large-b-cell lymphoma. New Engl J Med 346(25):1937–1947CrossRef
go back to reference Sandin I, Andrade G, Viegas F, Madeira D (2012) Aggressive and effective feature selection using genetic programming. In: IEEE congress on evolutionary computation (CEC), pp 1–8 Sandin I, Andrade G, Viegas F, Madeira D (2012) Aggressive and effective feature selection using genetic programming. In: IEEE congress on evolutionary computation (CEC), pp 1–8
go back to reference Schmidt M, Lipson H (2009) Distilling free-form natural laws from experimental data. Science 324(5923):81–85CrossRef Schmidt M, Lipson H (2009) Distilling free-form natural laws from experimental data. Science 324(5923):81–85CrossRef
go back to reference Shao S, Liu X, Zhou M, Zhan J, Liu X, Chu Y, Chen H (2012) A gpu-based implementation of an enhanced GEP algorithm. In: Conference on genetic and evolutionary computation, pp 999–1006 Shao S, Liu X, Zhou M, Zhan J, Liu X, Chu Y, Chen H (2012) A gpu-based implementation of an enhanced GEP algorithm. In: Conference on genetic and evolutionary computation, pp 999–1006
go back to reference Vladislavleva E, Smits G, Den Hertog D (2009) Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Trans Evol Comput 13(2):333–349CrossRef Vladislavleva E, Smits G, Den Hertog D (2009) Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Trans Evol Comput 13(2):333–349CrossRef
go back to reference Xue B, Zhang M, Browne WN (2013) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Syst Man Cybern 43(6):1656–1671 Xue B, Zhang M, Browne WN (2013) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Syst Man Cybern 43(6):1656–1671
go back to reference Xue B, Zhang M, Browne WN, Yao X (2016) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput 20(4):606–626CrossRef Xue B, Zhang M, Browne WN, Yao X (2016) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput 20(4):606–626CrossRef
go back to reference Yao X, Liu Y, Lin GM (1999) Evolutionary programming made faster. IEEE Trans Evol Comput 3(2):82–102CrossRef Yao X, Liu Y, Lin GM (1999) Evolutionary programming made faster. IEEE Trans Evol Comput 3(2):82–102CrossRef
go back to reference Zhai Y, Ong YS, Tsang IW (2014) The emerging “big dimensionality”. IEEE Comput Intell Mag 9(3):14–26CrossRef Zhai Y, Ong YS, Tsang IW (2014) The emerging “big dimensionality”. IEEE Comput Intell Mag 9(3):14–26CrossRef
go back to reference Zhong J, Cai W (2015) Differential evolution with sensitivity analysis and the powell’s method for crowd model calibration. J Comput Sci 9:26–32CrossRef Zhong J, Cai W (2015) Differential evolution with sensitivity analysis and the powell’s method for crowd model calibration. J Comput Sci 9:26–32CrossRef
go back to reference Zhong J, Ong YS, Cai W (2016) Self-learning gene expression programming. IEEE Trans Evol Comput 20(1):65–80CrossRef Zhong J, Ong YS, Cai W (2016) Self-learning gene expression programming. IEEE Trans Evol Comput 20(1):65–80CrossRef
go back to reference Zhong J, Cai W, Lees M, Luo L (2017a) Automatic model construction for the behavior of human crowds. Appl Soft Comput 56:368–378CrossRef Zhong J, Cai W, Lees M, Luo L (2017a) Automatic model construction for the behavior of human crowds. Appl Soft Comput 56:368–378CrossRef
go back to reference Zhong J, Feng L, Ong Y-S (2017b) Gene expression programming: a survey. IEEE Comput Intell Mag 12(3):54–72CrossRef Zhong J, Feng L, Ong Y-S (2017b) Gene expression programming: a survey. IEEE Comput Intell Mag 12(3):54–72CrossRef
go back to reference Zhou C, Xiao W, Tirpak TM, Nelson PC (2003) Evolving accurate and compact classification rules with gene expression programming. IEEE Trans Evol Comput 7(6):519–531CrossRef Zhou C, Xiao W, Tirpak TM, Nelson PC (2003) Evolving accurate and compact classification rules with gene expression programming. IEEE Trans Evol Comput 7(6):519–531CrossRef
Metadata
Title
A fast parallel genetic programming framework with adaptively weighted primitives for symbolic regression
Authors
Zhixing Huang
Jinghui Zhong
Liang Feng
Yi Mei
Wentong Cai
Publication date
30-09-2019
Publisher
Springer Berlin Heidelberg
Published in
Soft Computing / Issue 10/2020
Print ISSN: 1432-7643
Electronic ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-019-04379-4

Other articles of this Issue 10/2020

Soft Computing 10/2020 Go to the issue

Premium Partner