29. MPSiam: A Fast Multiplexing Siamese Tracking Network

Siamese trackers have achieved remarkable performance in accuracy. However, the high memory cost and inference speed have restricted the deployment of the state-of-the-art trackers in mobile applications. To address this issue, this paper presents a backbone consisting of multiplexing convolution blocks that newly proposed by us, which combine the spatial multiplexing operation and channel multiplexing operation. The spatial multiplexing operation is inspired by the subpixel convolution in super-resolution tasks. The channel multiplexing operation is inspired by the channel shuffle in ShuffleNet. These two modules can be used to effectively optimize the multiply–accumulate (MACC) operation, by multiplying the number of operations and then adding it to a network. We employ this new module to build a novel lightweight backbone for the SiamRPN++ tracker. We trained this model and evaluated its performances on the VOT2018 and OTB2015 datasets. Our model is compressed to 43 MB, the inference time was 83 FPS, and the experiments were carried out in a single NVIDIA 2080Ti GPU. Our model is superior to MobileNetv2-SiamRPN++, which has a model size of 58 MB and the inference time of 55 FPS, and our method also managed to reduce the MACC from 1.2 to 0.5 B. Compared with SiamRPN++ with Resnet50 backbone, our model achieved a compression rate of 4.8\(\times \) and speedup of 3.3\(\times \), just losing 3% EAO.