Spread-Learned Spatial Features to Improve Tick-Shape Networks

The rapid growth of mobile devices and embedded systems requires lightweight networks with high performance in reasonable complexity. Recently, TickNets have been proposed to meet that requirement via connecting several tick-shape backbones. However, their performance is still at modest levels due to the lack of spatial features exploited from a basic tick-shape backbone. To mitigate this problem, an efficient perceptron is proposed to take into account spread-learned spatial features for improving the learning ability of tick-shape networks. Accordingly, this spread-learned feature extractor is simply done by adding a Full Residual Point-Depth-Point (FR-PDP) block to the beginning of a basic backbone. Such strategy will ensure two practical benefits for the tick-shape networks: i) Exploiting the identical FR-PDP-based features in a tick-shape backbone; and ii) Extracting more discriminative spatial features for the learning process. Finally, STickNets are formed by simply connecting several spread-learned tick-shape backbones. Experimental results on various benchmark datasets have indicated that our proposal has significantly boosted the performance of tick-shape networks. In particular, STickNet-basic is enhanced by \(\sim \)3.5% on CIFAR-100, up to 7.4% on Stanford Dogs. The implementation code is available at https://github.com/ngochc/STicknets.