More Teachers Make Greater Students: Compression of CycleGAN

Generative Adversarial Networks (GANs) have obtained outstanding performance in image-to-image translation. Nevertheless, their applications are greatly limited due to high computational costs. Although past work on compressed GANs has yielded rich results, most still come at the expense of image quality. Therefore, in order to generate high-quality images and simplify the process of distillation, we propose a framework with more generators and fewer discriminators (MGFD) strategy to enhance the online knowledge distillation with high-quality images. First, we introduce the Inception-enhanced residual block into our enhanced teacher generator, which significantly improves image quality at a low cost. Then, the multi-granularity online knowledge distillation method is adopted and simplified by selecting wider Inception-enhanced teacher generator. In addition, we also combine the intermediate layer distillation losses to help student generator to obtain diverse features and more supervised signals from the intermediate layer for better transformations. Experiments demonstrate that our framework can significantly reduce computational costs and generate more natural images.