Routing Attention Shift Network for Image Classification and Segmentation

Deep neural networks as fundamental tools of deep learning have evolved remarkably in various tasks; however, the computational complexity and resources costs rapidly increased when using deeper networks, which challenges the deployment of the resource-limited devices. Recently, shift operation is considered as an alternative.to depthwise separable convolutions, using 60% fewer parameters compared spatial convolutions. Its basic block is composed by shift operations and 1 \(\times \) 1 convolution in the intermediate feature maps. Previous works focus on optimizing the redundancy of the correlation between shift groups, making shift to be a learnable parameter, which yields more time to train and higher computation. In this paper, we propose a “dynamic routing” strategy to seek the best movement for shift operation based on attention mechanism, termed Routing Attention Shift Layer (RASL), which measures the contribution of channels to the outputs without back propagation. Moreover, the proposed RASL shows strong generalization to many tasks. Experiments on both classification and semantic segmentation tasks demonstrate the superior performance of the proposed methods.