Improving Visual Feature Representations by Biasing Restricted Boltzmann Machines with Gaussian Filters

Advances in unsupervised learning have allowed the efficient learning of feature representations from large sets of unlabeled data. This paper evaluates visual features learned through unsupervised learning, specifically comparing biasing methods using Gaussian filters on a single-layer network. Using the restricted Boltzmann machine, features emerging through training on image data are compared by classification performance on standard datasets. When Gaussian filters are convolved with adjacent hidden layer activations from a single example during training, topographies emerge where adjacent features become tuned to slightly varying stimuli. When Gaussian filters are applied to the visible nodes, images become blurrier; training on these images leads to less localized features being learned. The networks are trained and tested on the CIFAR-10, STL-10, COIL-100, and MNIST datasets. It is found that the induction of topography or simple image blurring during training produce better features as evidenced by the consistent and notable increase in classification results.