Battle on Edge - Comparison of Convolutional Neural Networks Inference Speed on Two Various Hardware Platforms

Several reasons influenced the tendency to move the first level of machine learning data processing to the edge of the information system. Edge-generated data is typically processed by so-called edge devices with low processing power and low power consumption. In addition to well-known SoC (System on Chip) manufacturers that are usually used as an edge device, some manufacturers in this market base their processor design on open source. This paper compares two different SoC, one based on the ARM (Advanced RISC Machines) architecture and the other on the open-source RISC-V (Reduced Instruction Set Computer) architecture. The specificity of the analysed SoC based on the RISC-V architecture is an additional processor for speed up calculations common in neural networks. Since the architectures differ, we compare two SoC of similar price. The comparison’s focus is an analysis of the inference performance with the different number of filters in the first layer of the convolutional neural network used to detect handwritten digits. The process of convolutional neural network’s training occurs in the cloud and uses a well-known database of handwritten digits – MNIST (Modified National Institute of Standards and Technology). In the SoC based on the RISC-V architecture, a reduced dependence of the inference speed on the number of filters at the first level of the convolutional neural network was observed.