Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben

2010 | Buch

Introduction to Neural Networks Kapitel lesen Erstes Kapitel lesen

Sensitivity Analysis for Neural Networks

verfasst von: Daniel S. Yeung, Ian Cloete, Daming Shi, Wing W. Y. Ng

Verlag: Springer Berlin Heidelberg

Buchreihe : Natural Computing Series

Enthalten in: Springer Professional "Wirtschaft+Technik" , Springer Professional "Technik" , Springer Professional "Wirtschaft"

Einloggen, um Zugang zu erhalten
insite
SUCHEN

Über dieses Buch

Artificial neural networks are used to model systems that receive inputs and produce outputs. The relationships between the inputs and outputs and the representation parameters are critical issues in the design of related engineering systems, and sensitivity analysis concerns methods for analyzing these relationships. Perturbations of neural networks are caused by machine imprecision, and they can be simulated by embedding disturbances in the original inputs or connection weights, allowing us to study the characteristics of a function under small perturbations of its parameters.

This is the first book to present a systematic description of sensitivity analysis methods for artificial neural networks. It covers sensitivity analysis of multilayer perceptron neural networks and radial basis function neural networks, two widely used models in the machine learning field. The authors examine the applications of such analysis in tasks such as feature selection, sample reduction, and network optimization. The book will be useful for engineers applying neural network sensitivity analysis to solve practical problems, and for researchers interested in foundational problems in neural networks.

Inhaltsverzeichnis

Frontmatter
Chapter 1. Introduction to Neural Networks
Abstract
The human brain consists of ten billion densely interconnected nerve cells, called neurons; each connected to about 10,000 other neurons, with 60 trillion connections, synapses, between them. By using multiple neurons simultaneously, the brain can perform its functions much faster than the fastest computers in existence today. On the other hand, a neuron can be considered as a basic information-processing unit, whereas our brain can be considered as a highly complex, nonlinear and parallel biological information-processing network, in which information is stored and processed simultaneously. Learning is a fundamental and essential characteristic of biological neural networks. The ease with which they can learn led to attempts to emulate a biological neural network in a computer.
Daniel S. Yeung, Ian Cloete, Daming Shi, Wing W.Y. Ng
Chapter 2. Principles of Sensitivity Analysis
Abstract
Sensitivity refers to how a neural network output is influenced by its input and/or weight perturbations. Sensitivity analysis dates back to the 1960s, when Widrow investigated the probability of misclassification caused by weight perturbations, which are caused by machine imprecision and noisy input (Widrow and Hoff, 1960). In network hardware realization, such perturbations must be analyzed prior to its design, since they significantly affect network training and generalization. The initial idea of sensitivity analysis has been extended to the optimization of neural networks, such as through sample reduction, feature selection, and critical vector learning.
Daniel S. Yeung, Ian Cloete, Daming Shi, Wing W.Y. Ng
Chapter 3. Hyper-Rectangle Model
Abstract
In this chapter, we discuss a hyper rectangle model, instead of the traditional hypersphere, which is employed as the mathematical model to represent an MLP’s input space. The hyper-rectangle approach does not demand that the input deviation be very small as the derivative approach requires, and the mathematical expectation used in the hyper-rectangle model reflects the network’s output deviation more directly and exactly than the variance does. Moreover, this approach is applicable to the MLP that deals with infinite input patterns, which is an advantage of the MLP over other discrete feedforward networks like Madalines.
Daniel S. Yeung, Ian Cloete, Daming Shi, Wing W.Y. Ng
Chapter 4. Sensitivity Analysis with Parameterized Activation Function
Abstract
Among all the traditional methods introduced in Chap. 2, none has involved activation function in the calculation of sensitivity analysis. This chapter attempts to generalize Piché’s method by parameterizing antisymmetric squashing activation functions, through which a universal expression of MLP’s sensitivity will be derived without any restriction on input or output perturbations.
Daniel S. Yeung, Ian Cloete, Daming Shi, Wing W.Y. Ng
Chapter 5. Localized Generalization Error Model
Abstract
The generalization error bounds found by current error models using the number of effective parameters of a classifier and the number of training samples are usually very loose. These bounds are intended for the entire input space. However, support vector machines, radial basis function neural networks and multilayer Perceptron neural networks are local learning machines for solving problems and thus treat unseen samples near the training samples as more important. In this chapter, we describe a localized generalization error model which bounds the generalization error from above within a neighborhood of the training samples using a stochastic sensitivity measure (Yeung et al., 2007 and Ng et al., 2007). This model is then used to develop an architecture selection technique for a classifier with maximal coverage of unseen samples by specifying a generalization error threshold.
Daniel S. Yeung, Ian Cloete, Daming Shi, Wing W.Y. Ng
Chapter 6. Critical Vector Learning for RBF Networks
Abstract
One of the most popular neural network models, the radial basis function (RBF) network attracts a lot of attention due to its improved approximation ability as well as the construction of its architecture. Bishop (1991) concluded that an RBF network can provide a fast, linear algorithm capable of representing complex nonlinear mappings. Park and Sandberg (1993) further showed that an RBF network can approximate any regular function. In a statistical sense, the approximation ability is a special case of statistical consistency. Hence, Xu et al. (1994) presented upper bounds for the convergence rates of the approximation error of RBF networks, and constructively proved the existence of a consistent point-wise estimator for RBF networks. Their results can be a guide to optimize the construction of an RBF network, which includes the determination of the total number of radial basis functions along with their centers and widths. This is an important problem to address because the performance and training of an RBF network depend very much on these parameters.
Daniel S. Yeung, Ian Cloete, Daming Shi, Wing W.Y. Ng
Chapter 7. Sensitivity Analysis of Prior Knowledge1
Abstract
The paradigm of Knowledge-Based Neurocomputing (Cloete et al., 2000b) addresses the encoding, extraction and refinement of symbolic knowledge in a neurocomputing paradigm. Prior symbolic knowledge derived outside of neural networks can be encoded in neural network form, and then further trained. Classification rules of various forms (Cloete, 1996 and 2000) are most often used as prior symbolic knowledge, and then transformed into a neural network that produces the same classification. In the transformation process, one would like to retain the flexibility in the network for further training, and encode the knowledge in such a way that it is not destroyed by further training, but can be revised. In addition, one would like the possibility to improve the classification by discovering new rules if needed.
Daniel S. Yeung, Ian Cloete, Daming Shi, Wing W.Y. Ng
Chapter 8. Applications
Abstract
The curse of dimensionality is always problematic in pattern classification problems. In this chapter, we provide a brief comparison of the major methodologies for reducing input dimensionality and summarize them in three categories: correlation among features, transformation and neural network sensitivity analysis. Furthermore, we propose a novel method for reducing input dimensionality that uses a stochastic RBFNN sensitivity measure. The experimental results are promising for our method of reducing input dimensionality.
Daniel S. Yeung, Ian Cloete, Daming Shi, Wing W.Y. Ng
Backmatter
Metadaten
Titel
Sensitivity Analysis for Neural Networks
verfasst von
Daniel S. Yeung
Ian Cloete
Daming Shi
Wing W. Y. Ng
Copyright-Jahr
2010
Verlag
Springer Berlin Heidelberg
Electronic ISBN
978-3-642-02532-7
Print ISBN
978-3-642-02531-0
DOI
https://doi.org/10.1007/978-3-642-02532-7

Premium Partner

    Bildnachweise