A wavelet-based neural network scheme for supervised and unsupervised learning

We introduce a scheme for supervised and unsupervised learning based on successive decomposition of random inputs by means of wavelet basis. To each successive layer corresponds a different wavelet basis with more null moments than its predecessor. We consider two types of operations in the scheme: firstly, a input signal treatment phase—the awake phase—and, secondly, a reorganizing phase of the random wavelet coefficients obtained in the previous awake phase—the asleep phase. The next awake phase input treatment will include a feedback derived from the previous asleep phase. The set of random wavelet coefficients of the deepest layer—at each stage of the learning process—is supposed to be Gaussian distributed, and the corresponding sequence of Gaussian distributions constitutes the inner representation of the world in the scheme. We show that in the case of a constant average value of the inputs in each successive awake phases, the mean value of the feedback converges to a multiple of the constant expected value of the inputs. We show a general result on the stabilization of the Gaussian distribution corresponding to the deepest layer, and we show that when the estimated means and covariances converge, then the sequence of Gaussian distributions of the inner representation of the world in the scheme also converges. We present an example of a neural network scheme for supervised learning corresponding to extracting a signal from data corrupted with Gaussian noise.