Top

Published in:

2019 | OriginalPaper | Chapter

d(Tree)-by-dx: Automatic and Exact Differentiation of Genetic Programming Trees

Authors : Peter Rockett, Yuri Kaszubowski Lopes, Tiantian Dou, Elizabeth A. Hathway

Published in: Hybrid Artificial Intelligent Systems

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Genetic programming (GP) has developed to the point where it is a credible candidate for the ‘black box’ modeling of real systems. Wider application, however, could greatly benefit from its seamless embedding in conventional optimization schemes, which are most efficiently carried out using gradient-based methods. This paper describes the development of a method to automatically differentiate GP trees using a series of tree transformation rules; the resulting method can be applied an unlimited number of times to obtain higher derivatives of the function approximated by the original, trained GP tree. We demonstrate the utility of our method using a number of illustrative gradient-based optimizations that embed GP models.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Particle Swarm Optimization-Based CNN-LSTM Networks for Anomalous Query Access Control in RBAC-Administered Model

next chapter Genetic Algorithm-Based Deep Learning Ensemble for Detecting Database Intrusion via Insider Attack

We have used NLopt version 2.5 downloadable from https://github.com/stevengj/nlopt/archive/v2.5.0.tar.gz.

Bartholomew-Biggs, M.C., Brown, S., Christianson, B., Dixon, L.C.W.: Automatic differentiation of algorithms. J. Comput. Appl. Math. 124, 171–190 (2000). https://doi.org/10.1016/S0377-0427(00)00422-2MathSciNetCrossRefMATH

Baydin, A.G., Pearlmutter, B.A., Radul, A.A., Siskind, J.M.: Automatic differentiation in machine learning: a survey. J. Mach. Learn. Res. 18(1), 5595–5637 (2017). http://jmlr.org/papers/v18/17-468.htmlMathSciNetMATH

Camacho, E.F., Bordons, C.: Model Predictive Control, 2nd edn. Springer, London (2004)MATH

Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. Society for Industrial and Applied Mathematics (2009). https://doi.org/10.1137/1.9780898718768

Gandomi, A.H., Alavi, A.H., Ryan, C. (eds.): Handbook of Genetic Programming Applications. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20883-1CrossRef

Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison Wesley, Reading (1989)MATH

Hock, W., Schittkowski, K.: Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems, vol. 187. Springer, Heidelberg (1981). https://doi.org/10.1007/978-3-642-48320-2CrossRefMATH

Izzo, D., Biscani, F., Mereta, A.: Differentiable genetic programming. In: 20th European Conference (EuroGP 2017), Amsterdam, The Netherlands, pp. 35–51 (2017). https://doi.org/10.1007/978-3-319-55696-3_3

Johnson, S.G.: The NLopt nonlinear-optimization package (2019). http://ab-initio.mit.edu/nlopt

10.

Keijzer, M.: Improving symbolic regression with interval arithmetic and linear scaling. In: Ryan, C., Soule, T., Keijzer, M., Tsang, E., Poli, R., Costa, E. (eds.) EuroGP 2003. LNCS, vol. 2610, pp. 70–82. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-36599-0_7CrossRef

11.

Kraft, D.: Algorithm 733: TOMP-Fortran modules for optimal control calculations. ACM Trans. Math. Softw. 20(3), 262–281 (1994). https://doi.org/10.1145/192115.192124MathSciNetCrossRefMATH

12.

Mousavi Astarabadi, S.S., Ebadzadeh, M.M.: Avoiding overfitting in symbolic regression using the first order derivative of GP trees. In: Genetic and Evolutionary Computation Conference (GECCO Companion 2015), Madrid, Spain, pp. 1441–1442, 11–15 July 2015. https://doi.org/10.1145/2739482.2764662

13.

Ni, J., Drieberg, R.H., Rockett, P.I.: The use of an analytic quotient operator in genetic programming. IEEE Trans. Evol. Comput. 17(1), 146–152 (2013). https://doi.org/10.1109/TEVC.2012.2195319CrossRef

14.

Rockett, P., Hathway, E.A.: Model-predictive control for non-domestic buildings: critical review and prospects. Build. Res. Inform. 45(5), 556–571 (2017). https://doi.org/10.1080/09613218.2016.1139885CrossRef

15.

Rudin, W.: Principles of Mathematical Analysis. McGraw-Hill, New York (1976)MATH

16.

Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006). https://doi.org/10.1007/s10107-004-0559-yMathSciNetCrossRefMATH

17.

Waltz, R.A., Morales, J.L., Nocedal, J., Orban, D.: An interior algorithm for nonlinear optimization that combines line search and trust region steps. Math. Program. 107(3), 391–408 (2006). https://doi.org/10.1007/s10107-004-0560-5MathSciNetCrossRefMATH

Title: d(Tree)-by-dx: Automatic and Exact Differentiation of Genetic Programming Trees
Authors: Peter Rockett
Yuri Kaszubowski Lopes
Tiantian Dou
Elizabeth A. Hathway
Publisher: Springer International Publishing
Book: Hybrid Artificial Intelligent Systems
Print ISBN: 978-3-030-29858-6

Electronic ISBN: 978-3-030-29859-3

Copyright Year: 2019
DOI: https://doi.org/10.1007/978-3-030-29859-3_12

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner