Improve Semi-supervised Learning with Metric Learning Clusters and Auxiliary Fake Samples

Because it is very expensive to collect a large number of labeled samples to train deep neural networks in certain fields, semi-supervised learning (SSL) researcher has become increasingly important in recent years. There are many consistency regularization-based methods for solving SSL tasks, such as the \(\Pi \) model and mean teacher. In this paper, we first show through an experiment that the traditional consistency-based methods exist the following two problems: (1) as the size of unlabeled samples increases, the accuracy of these methods increases very slowly, which means they cannot make full use of unlabeled samples. (2) When the number of labeled samples is vary small, the performance of these methods will be very low. Based on these two findings, we propose two methods, metric learning clustering (MLC) and auxiliary fake samples, to alleviate these problems. The proposed methods achieve state-of-the-art results on SSL benchmarks. The error rates are 10.20%, 38.44% and 4.24% for CIFAR-10 with 4000 labels, CIFAR-100 with 10,000 labels and SVHN with 1000 labels by using MLC. For MNIST, the auxiliary fake samples method shows great results in cases with the very few labels.