A High-Speed Neural Architecture Search Considering the Number of Weights

Neural architecture search (NAS) is a promising method to ascertain network architecture automatically and to build a suitable network for a particular application without any human intervention. However, NAS requires huge computation resources to find the optimal parameters of a network in the training phase of each search. Because a trade-off generally exists between model size and accuracy in deep learning models, the model size tends to increase in pursuit of higher accuracy. In applications with limited resources, such as edge AI, reducing the network weight might be more important than improving its accuracy. Alternatively, achieving high accuracy with maximum resources might be more important. The objective of this research is to find a model with sufficient accuracy with a limited number of weights and to reduce the search time. We improve the Differentiable Network Search (DARTS) algorithm, one of the fastest NAS methods, by adding another constraint to the loss function, which limits the number of network weights. We evaluate the proposed algorithm using three constraints. Compared to the conventional DARTS algorithm, the proposed algorithm reduces the search time by up to 40% when the model size range is set properly. It achieves comparable accuracy with that of DARTS.