Multi-branch Semantic GAN for Infrared Image Generation from Optical Image

Infrared remote sensing images capture the information of ground objects by their thermal radiation differences. However, the facility required for infrared imaging is not only priced high but also demands strict testing conditions. Thus it becomes an important topic to seek a way to convert easily-obtained optical remote sensing images into infrared remote sensing images. The conventional approaches cannot generate satisfactory infrared images due to the challenge of this task and many unknown parameters to be determined. In this paper, we proposed a novel multi-branch semantic GAN (MBS-GAN) for infrared image generation from the optical image. In the proposed model, we draw on the idea from Ensemble Learning and propose to use more than one generator to synthesize the infrared images with different semantic information. Specially, we integrate scene classification into image transformation to train models with scene information, which assists learned generation models to capture more semantic characteristics. The generated images are evaluated by PSNR, SSIM and cosine similarity. The experimental results prove that this proposed method is able to generate images retaining the infrared radiation characteristics of ground objects and performs well in converting optical images to infrared images.