Discrete Cosine Transform Spectral Pooling Layers for Convolutional Neural Networks

Pooling operations for convolutional neural networks provide the opportunity to greatly reduce network parameters, leading to faster training time and less data overfitting. Unfortunately, many of the common pooling methods such as max pooling and mean pooling lose information about the data (i.e., they are lossy methods). Recently, spectral pooling has been utilized to pool data in the spectral domain. By doing so, greater information can be retained with the same network parameter reduction as spatial pooling. Spectral pooling is currently implemented in the discrete Fourier domain, but it is found that implementing spectral pooling in the discrete cosine domain concentrates energy in even fewer spectra. Although Discrete Cosine Transforms Spectral Pooling Layers (DCTSPL) require extra computation compared to normal spectral pooling, the overall time complexity does not change and, furthermore, greater information preservation is obtained, producing networks which converge faster and achieve a lower misclassification error.