Neural Network Classification in Exponential Models with Unknown Statistics

This contribution demonstrates the possibility to achieve a Bayesian (or nearly Bayesian) classification of exponentially distributed data by perceptrons with at most two hidden layers. The number of hidden layers depends on how much is known about the sufficient statistics figuring in the corresponding exponential distributions. A practical applicability is illustrated by classification of normally distributed data. Experiments with such data proved that, in the learning based on correct classification information, the error backpropagation rule is able to create in the hidden layers surprisingly good approximations of apriori unknown sufficient statistics. This enables the trained network to imitate Bayesian classifiers and to achieve minimum classification errors.