nach oben

Neural Computing and Applications

Erschienen in:

26.04.2017 | EANN 2016

Constructive lower bounds on model complexity of shallow perceptron networks

verfasst von: Věra Kůrková

Erschienen in: Neural Computing and Applications | Ausgabe 7/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Limitations of shallow (one-hidden-layer) perceptron networks are investigated with respect to computing multivariable functions on finite domains. Lower bounds are derived on growth of the number of network units or sizes of output weights in terms of variations of functions to be computed. A concrete construction is presented with a class of functions which cannot be computed by signum or Heaviside perceptron networks with considerably smaller numbers of units and smaller output weights than the sizes of the function’s domains. A subclass of these functions is described whose elements can be computed by two-hidden-layer perceptron networks with the number of units depending on logarithm of the size of the domain linearly.

Vorheriger Artikel Special issue: Engineering applications of neural networks

Nächster Artikel Application of feature selection methods for automated clustering analysis: a review on synthetic datasets

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

Ba LJ, Caruana R (2014) Do deep networks really need to be deep? In: Ghahrani Z et al (eds) Advances in neural information processing systems, vol 27, pp 1–9

Barron AR (1992) Neural net approximation. In: Narendra K (ed) Proceedings 7th Yale workshop on adaptive and learning systems, pp 69–72. Yale University Press

Barron AR (1993) Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans Inf Theory 39:930–945MathSciNetCrossRefMATH

Bengio Y (2009) Learning deep architectures for AI. Foundations and Trends in Machine Learning 2:1–127CrossRefMATH

Bengio Y, Delalleau O, Roux NL (2006) The curse of highly variable functions for local kernel machines. In: Advances in neural information processing systems 18, pp 107–114. MIT Press

Bengio Y, LeCun Y (2007) Scaling learning algorithms towards AI. In: Bottou LO, Chapelle D, DeCoste, Weston J (eds) Large-Scale Kernel Machines. MIT Press

Bianchini M, Scarselli F (2014) On the complexity of neural network classifiers: a comparison between shallow and deep architectures. IEEE Trans Neural Netw Learning Syst 25(8):1553–1565CrossRef

Candès EJ (2008) The restricted isometric property and its implications for compressed sensing. C R Acad Sci Paris I 346:589–592CrossRefMATH

Candès EJ, Tao T (2005) Decoding by linear programming. IEEE Trans Inf Process 51:4203–4215MathSciNetCrossRefMATH

10.

Cover T (1965) Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Trans Electron Comput 14:326–334CrossRefMATH

11.

Erdös P, Spencer JH (1974) Probabilistic methods in combinatorics. Academic Press

12.

Fine TL (1999) Feedforward neural network methodology. Springer, Berlin HeidelbergMATH

13.

Gnecco G, Sanguineti M (2011) On a variational norm tailored to variable-basis approximation schemes. IEEE Trans Inf Theory 57:549–558MathSciNetCrossRefMATH

14.

Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554MathSciNetCrossRefMATH

15.

Ito Y (1992) Finite mapping by neural networks and truth functions. Mathematical Scientist 17:69–77MathSciNetMATH

16.

Kainen PC, Kůrková V, Sanguineti M (2012) Dependence of computational models on input dimension: tractability of approximation and optimization tasks. IEEE Trans Inf Theory 58:1203–1214MathSciNetCrossRefMATH

17.

Kainen PC, Kůrková V, Vogt A (1999) Approximation by neural networks is not continuous. Neurocomputing 29:47–56CrossRef

18.

Kainen PC, Kůrková V, Vogt A (2000) Geometry and topology of continuous best and near best approximations. J Approx Theory 105:252–262MathSciNetCrossRefMATH

19.

Kainen PC, Kůrková V, Vogt A (2007) A Sobolev-type upper bound for rates of approximation by linear combinations of Heaviside plane waves. J Approx Theory 147:1–10MathSciNetCrossRefMATH

20.

Kecman V (2001) Learning and soft computing. MIT Press, CambridgeMATH

21.

Kůrková V (1997) Dimension-independent rates of approximation by neural networks. In: Warwick K, Kárný M (eds) Computer-intensive methods in control and signal processing. The curse of dimensionality, pp 261–270. Birkhäuser, Boston

22.

Kůrková V (2008) Minimization of error functionals over perceptron networks. Neural Comput 20:250–270MathSciNetMATH

23.

Kůrková V (2012) Complexity estimates based on integral transforms induced by computational units. Neural Netw 33:160–167CrossRefMATH

24.

Kůrková V (2016) Lower bounds on complexity of shallow perceptron networks. In: Jayne C, Iliadis L (eds) Engineering applications of neural networks. Communications in computer and information sciences, vol 629, pp 283–294. Springer

25.

Kůrková V, Kainen PC (1994) Functionally equivalent feedforward neural networks. Neural Comput 6 (3):543–558CrossRef

26.

Kůrková V, Kainen PC (1996) Singularities of finite scaling functions. Appl Math Lett 9(2):33–37MathSciNetCrossRefMATH

27.

Kůrková V, Kainen PC (2014) Comparing fixed and variable-width Gaussian kernel networks. Neural Netw 57:23–28CrossRefMATH

28.

Kůrková V, Sanguineti M (2002) Comparison of worst-case errors in linear and neural network approximation. IEEE Trans Inf Theory 48:264–275MathSciNetCrossRefMATH

29.

Kůrková V, Sanguineti M (2008) Approximate minimization of the regularized expected error over kernel models. Math Oper Res 33:747–756MathSciNetCrossRefMATH

30.

Kůrková V, Sanguineti M (2016) Model complexities of shallow networks representing highly varying functions. Neurocomputing 171:598–604CrossRef

31.

Kůrková V, Savický P, Hlaváčková K (1998) Representations and rates of approximation of real-valued Boolean functions by neural networks. Neural Netw 11:651–659CrossRef

32.

LeCunn Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444CrossRef

33.

MacWilliams F, Sloane NJA (1977) The theory of error-correcting codes. North-Holland, Amsterdam

34.

Maiorov V, Pinkus A (1999) Lower bounds for approximation by MLP neural networks. Neurocomputing 25:81–91CrossRefMATH

35.

Maiorov VE, Meir R (2000) On the near optimality of the stochastic approximation of smooth functions by neural networks. Adv Comput Math 13:79–103MathSciNetCrossRefMATH

36.

Mhaskar HN, Liao Q, Poggio T (2016) Learning functions: when is deep better than shallow. Center for brains, minds & machines CBMM Memo No. 045v3, pp 1–12

37.

Mhaskar HN, Liao Q, Poggio T (2016) Learning functions: when is deep better than shallow. Center for brains, minds & machines CBMM Memo No. 045v4, pp 1–12

38.

Sloane NJA A library of Hadamard matrices. http://www.research.att.com/njas/hadamard/

39.

Sussman HJ (1992) Uniqueness of the weights for minimal feedforward nets with a given input-output map. Neural Netw 5(4):589–593CrossRef

40.

Sylvester J (1867) Thoughts on inverse orthogonal matrices, simultaneous sign successions, and tessellated pavements in two or more colours, with applications to Newton’s rule, ornamental tile-work, and the theory of numbers. Phil Mag 34:461– 475CrossRef

Titel: Constructive lower bounds on model complexity of shallow perceptron networks
verfasst von: Věra Kůrková
Publikationsdatum: 26.04.2017
Verlag: Springer London
Erschienen in: Neural Computing and Applications / Ausgabe 7/2018
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-017-2965-0

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Weitere Artikel der Ausgabe 7/2018

Electrical load forecasting based on self-adaptive chaotic neural network using Chebyshev map

Combining bag-of-words and sentiment features of annual reports to predict abnormal stock returns

Linguistic hesitant intuitionistic fuzzy decision-making method based on VIKOR

Novel intuitionistic fuzzy soft multiple-attribute decision-making methods

2D recurrent neural networks: a high-performance tool for robust visual tracking in dynamic scenes

A hybrid deep learning neural approach for emotion recognition from facial expressions for socially assistive robots