On the relations between discriminant analysis and multilayer perceptrons

doi:10.1016/0893-6080(91)90071-C

Neural Networks

Volume 4, Issue 3, 1991, Pages 349-360

https://doi.org/10.1016/0893-6080(91)90071-C Get rights and content

Abstract

We study the relations between discriminant analysis and multilayer perceptrons used for classification tasks. We first consider linear networks and prove the formal equivalence between the two techniques in this case. We then present a set of experiments on problems with increasing degree of nonlinearity. This allows to study the extension of this result to nonlinear nets and to investigate data transformations in the successive layers of these nets. Finally, we show evidence of generic properties of MLPs classifiers.

References (18)

P. Baldi et al.
Neural networks and principal component analysis: learning from example without local minima
Neural Networks
(1989)
K.I. Funahashi
On the approximate realization of continuous mappings by neural networks
Neural Networks
(1989)
K. Hornik et al.
Multilayer feedforward networks are universal approximators
Neural Networks
(1989)
H. Bourlard et al.
Autoassociation by multilayer perceptrons and singular value decomposition
Biological Cybernetics
(1988)
G.W. Cottrel et al.
Image compression by back propagation: an example of extensional programming
P. Devijver et al.
Statistical Pattern Recognition
(1982)
R.O. Duda et al.
Pattern Classification and Scene Analysis
(1973)
R.A. Fisher
The use of multiple measurements in taxonomic problems
Annals of Eugenics
(1936)
F. Fogelman-Soulie et al.
Evaluation of network architectures on test learning tasks

There are more references available in the full text version of this article.

Cited by (94)

Hybrid neural networks for big data classification
2020, Neurocomputing
Citation Excerpt :
Neural networks are models with a wide variety of applications, such as pattern recognition, signal classification, applications for robot control, among others [1–4]. Neural networks are able to learn the distinctive features of a set of data grouped into classes; this learning process is called the training of the neural network and in this way is that the network manages to classify such input patterns [5]. The training of this type of algorithms is divided into two main branches, supervised and unsupervised training [6].
Two new hybrid neural architectures combining morphological neurons and perceptrons are introduced in this paper. The first architecture, called Morphological - Linear Neural Network (MLNN) consists of a hidden layer of morphological neurons and an output layer of classical perceptrons has the capability of extracting features. The second architecture, called Linear-Morphological Neural Network (LMNN) is composed of one or several perceptron layers as a feature extractor, it is then followed by an output layer of morphological neurons for non-linear classification. Both architectures are trained by stochastic gradient descent. One of the main contributions of this paper is to show that the morphological layer offers a greater capacity to extract features than the perceptron layer. This claim is supported both theoretically and experimentally. We prove that the morphological layer possesses a greater capacity per computation unit to segment the 2D input space than the perceptron layer. In other words, adding more hyper-boxes produces more response regions than adding hyperplanes. From an empirical point of view, we test the two new models on 25 standard datasets at low dimensionality and one big data dataset. The result is that MLNN requires a lesser number of learning parameters than the other tested architectures while achieving better accuracies.
Neural networks and statistical techniques: A review of applications
2009, Expert Systems with Applications
Neural networks are being used in areas of prediction and classification, the areas where statistical methods have traditionally been used. Both the traditional statistical methods and neural networks are looked upon as competing model-building techniques in literature. This paper carries out a comprehensive review of articles that involve a comparative study of feed forward neural networks and statistical techniques used for prediction and classification problems in various areas of applications. Tabular presentations highlighting the important features of these articles are also provided. This study aims to give useful insight into the capabilities of neural networks and statistical methods used in different kinds of applications.
Adaptive Machine Learning Algorithms with Python: Solve Data Analytics and Machine Learning Problems on Edge Devices
2022, Adaptive Machine Learning Algorithms with Python: Solve Data Analytics and Machine Learning Problems on Edge Devices
Course performance prediction for basic courses of universities based on support vector machine
2019, Journal of Physics: Conference Series
Connectionist approaches for automatic speaker recognition
2019, ESCA Workshop on Automatic Speaker Recognition, Identification, and Verification, ASRIV 1994
MCA-NN: Multiple Correspondence Analysis Based Neural Network for Disaster Information Detection
2017, Proceedings - 2017 IEEE 3rd International Conference on Multimedia Big Data, BigMM 2017

View all citing articles on Scopus

View full text

Original contributionOn the relations between discriminant analysis and multilayer perceptrons

Abstract

Neural Networks

Neural Networks

Neural Networks

Autoassociation by multilayer perceptrons and singular value decomposition

Biological Cybernetics

Image compression by back propagation: an example of extensional programming

Statistical Pattern Recognition

Pattern Classification and Scene Analysis

The use of multiple measurements in taxonomic problems

Annals of Eugenics

Evaluation of network architectures on test learning tasks

Original contribution
On the relations between discriminant analysis and multilayer perceptrons