6. Brain-Inspired Machine Learning Algorithm: Neural Network Optimization

An artificial neural network (ANN) is a network inspired by biological neural networks (the central nervous systems of animals, in particular the brain) which are used to estimate or approximate functions that can depend on a large number of inputs that are generally unknown [1–7]. All inputs for the given neuron are multiplied by their respective weights, summed up, and then transformed using an activation function. Activation functions usually restrict the range of values of the neuron output, for example, from minus one to one. This transformed value then serves as the input for the next neuron. A neural network consists of three types of layers—input, hidden, and output as depicted in Fig. 6.1. The input layer contains the data supplied to the network. In the hidden layers the transformation of the inputs takes place and the output layer produces the estimate. The number of hidden layers is not limited. Several activation functions exist and they also influence the performance of the ANNs. Perhaps the simplest activation function is threshold function. If the signal from a neuron is larger or equal to a certain set value, then the output value is equal to 1 else the output value is equal to zero. This type of signal is good for binary problems. In general, ANNs learn from the data and they are able to improve in iterations by adjusting the weights [7].