Comparing Training of Sparse to Classic Neural Networks for Binary Classification in Medical Data

Sparse Neural Networks are increasing in popularity and provide the opportunity for compact and efficient models for resource-constrained environments which are expanding as the number of IoT devices is increasing and as the Edge Computing and Fog paradigms are gaining traction. We investigate and evaluate sparsifying the training of Convolutional Neural Networks for the task of binary classification on medical datasets. We considered low (i.e., \(28\times 28\)) grey-scale resolution images that are memory-friendly and suitable for storing and analysing on lightweight devices. We found out that high sparsification strategies (above 75%) can achieve comparable performances with that of the fully connected counterpart while allowing for a reduction in inference time and peak memory usage, beneficial for resource-constrained environments part of Edge Computing. It is important to note that, as might be expected, after 90% sparsity, the performance can oscillate, and the results can vary significantly.