Top

Published in:

2024 | OriginalPaper | Chapter

A Note on the Hwang-Kim’s Universal Activation Function

Authors : Maria Vasileva, Nikolay Kyurkchiev

Published in: New Trends in the Applications of Differential Equations in Sciences

Publisher: Springer Nature Switzerland

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In this talk we give some notes on Universal activation function UA(t) proposed from Seung-Yeon Hwang and Leong-Joon Kim (Computers, Materials and Continua 75:2, 2022). First of all, we will note that this new activation function is, in a certain sense, a ”superposition” of activation functions suggested by Jun (Mathematics 7:3, 2019) and Vasileva and Kyurkchiev (Int. J. Differ. Equ. and Appl. 21:1, 2022). This fact is missing in the work of Hwang and Kim. Our main goal is to investigate the smooth approximation to UA(t) using Gaussian error function. The theoretical results are important in elucidating the intrinsic properties of this new universal activation function. We present simple dynamic programming module for graphical representation of smooth approximation of UA(t) developed by CAS Wolfram Mathematica.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Novel Monte Carlo Algorithm for Linear Algebraic Systems

next chapter Special Lattice and Digital Sequences for Multidimensional Air Pollution Modelling

Jagtap, A. D., Karniadakis, G. Em.: How important are activation functions in regression and classification? A survey, performance comparison, and future directions. Journal of Machine Learning for Modeling and Computing 4(1), 21–75 (2023). https://doi.org/10.1615/JMachLearnModelComput.2023047367.

Ohn, I., Kim, Y.: Smooth function approximation by deep neural networks with general activation functions. Entropy 21(7), 627 (2019). https://doi.org/10.3390/e21070627.

Cybenko, G.: Approximation by superpositions of a sigmoidal function. Math. Control Signal Systems 2, 303–314 (1989). https://doi.org/10.1007/BF02551274.

Chen, Z., Cao, F.: The approximation operators with sigmoidal functions. Comput. Math. Appl. 58(4), 758–765 (2009). https://doi.org/10.1016/j.camwa.2009.05.001.

Guliyev, N., Ismailov, V.: A single hidden layer feedforward network with only one neuron in the hidden layer san approximate any univariate function. Neural Computation 28(7), 1289–1304 (2016). https://doi.org/10.1162/NECO_a_00849.

Costarelli, D., Spigler, R.: Approximation results for neural network operators activated by sigmoidal functions. Neural Networks 44, 101–106 (2013). https://doi.org/10.1016/j.neunet.2013.03.015.

Costarelli, D., Spigler, R.: Constructive Approximation by Superposition of Sigmoidal Functions. Anal. Theory Appl. 29(2), 169–196 (2013). https://doi.org/10.4208/ata.2013.v29.n2.8.

Costarelli, D., Vinti, G.: Pointwise and uniform approximation by multivariate neural network operators of the max-product type. Neural Networks 81, 81–90 (2016). https://doi.org/10.1016/j.neunet.2016.06.002.

Kyurkchiev, N., Nikolov, G.: Comments on some new classes of sigmoidal and activation functions. Applications. Dyn. Syst. Appl. 28(4), 789–808 (2019). https://doi.org/10.12732/dsa.v28i4.1.

10.

Kyurkchiev, N., Markov, S.: Sigmoid Functions: Some Approximation and Modelling Aspects. Some Moduli in Programming Environment MATHEMATICA. LAP LAMBERT Academic Publishing: Saarbrucken, Germany (2015) ISBN 978-3-659-76045-7.

11.

Kyurkchiev, N., Markov, S.: On the Hausdorff distance between the Heaviside step function and Verhulst logistic function. J. Math. Chem. 54(1), 109–119 (2016). https://doi.org/10.1007/s10910-015-0552-0.

12.

Kyurkchiev, N.: A note on the Volmer’s activation (VA) function. C. R. Acad. Bulg. Sci. 70(6), 769–776 (2017).

13.

Kyurkchiev, N., Iliev, A., Markov, S.: Some techniques for recurrence generating of activation functions. LAP LAMBERT Academic Publishing (2017), ISBN: 978-3-330-33143-3.

14.

Kyurkchiev, N., Iliev, A., Rahnev, A.: Some Families of Sigmoid Functions: Applications to Growth Theory. LAP LAMBERT Academic Publishing (2019), ISBN: 978-613-9-45608-6.

15.

Kyurkchiev, N., Iliev, A., Rahnev, A.: A new class of activation functions based on the correcting amendments of Gompertz-Makeham type. Dyn. Syst. Appl. 28(2), 243–257 (2019). http://dx.doi.org/10.12732/dsa.v28i2.2.

16.

Kyurkchiev, N., Andreev, A.: Approximation and antenna and filter synthesis: Some moduli in programming environment Mathematica, LAP LAMBERT Academic Publishing: Saarbrucken, Germany, (2014), ISBN 978-3-659-53322-8.

17.

Pavlov, N.; Iliev, A.; Rahnev, A.; Kyukchiev, N.: Some software reliability models: Approximation and modeling aspects. LAP LAMBERT Academic Publishing: Saarbrucken, Germany (2018), ISBN: 978-613-9-82805-0.

18.

Iliev, A.; Kyukchiev, N.; Rahnev, A.; Terzieva, T.: Some models in the theory of computer viruses propagation. LAP LAMBERT Academic Publishing: Saarbrucken, Germany (2019), ISBN: 978-620-0-00826-8.

19.

Hwang, S.-Y., Kim, L.-J.: A universal activation function for deep learning. Computers, Materials and Continua 75(2), 3553–3569 (2023). https://doi.org/10.32604/cmc.2023.037028.

20.

Yun, B. I.: A Neural network approximation based on a parametric sigmoidal function. Mathematics 7(3), 262 (2019). https://doi.org/10.3390/math7030262.

21.

Vasileva, M., Kyurkchiev, N.: Investigations on the smooth approximation to POLY(X; N) using Gaussian error function. Related problems, Int. J. of Differ. Equ. and Appl. 21(1), 93–104 (2022). https://doi.org/10.12732/ijdea.v21i1.8.

22.

Bagul, Y. J., C. Chesneau, C.: Sigmoid functions for the smooth approximation to the absolute value function. Moroccan J. of Pure and Appl. Anal. (MJPAA) 7(1), 12–19 (2021). https://doi.org/10.2478/mjpaa-2021-0002.

23.

Hausdorff, F.: Set Theory. 2nd ed. Chelsea Publ., New York, NY, USA (1962).

24.

Sendov, B.L.: Hausdorff approximations. In Mathematics and Its Applications, Springer Science & Business Media: Berlin/Heidelberg, Germany, Vol. 50, 1–367 (1990).

25.

Kyurkchiev, N.: Comments on the Yun’s algebraic activation function. Some extensions in the trigonometrig case. Dyn. Syst. Appl. 28(3), 533–543 (2019). https://doi.org/10.12732/dsa.v28i3.1.

Title: A Note on the Hwang-Kim’s Universal Activation Function
Authors: Maria Vasileva
Nikolay Kyurkchiev
Publisher: Springer Nature Switzerland
Book: New Trends in the Applications of Differential Equations in Sciences
Print ISBN: 978-3-031-53211-5

Electronic ISBN: 978-3-031-53212-2

Copyright Year: 2024
DOI: https://doi.org/10.1007/978-3-031-53212-2_40

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"