Hyperspectral and Multispectral Image Fusion via Regularization on Non-local Structure Tensor Total Variation

Hyperspectral and multispectral image fusion plays an important role in the fields such as image restoration, space robotics, remote sensing, computer vision, etc. The fused images take advantages of the high spectral resolutions of hyperspectral images and the high spatial resolutions of multispectral images. So there has been a recent outgrowth of interests of research in this field. However, the local structural image regularity and non-local image self-similarity are not yet explored in existing fusion methods. This paper develops a novel method for fusing hyperspectral and multispectral images with non-local structure tensor total variation regularization. Specifically, the weight function to measure self-similarity in non-local domain is introduced on the basis of the definition of structure tensor. Therefore, the proposed method can exploit the regularity and self-similarity of spatial-spectral image’s non-local structures simultaneously. In addition to the general formulation of non-local structure tensor variation, this paper presents the discrete non-local structure tensor variation for hyperspectral and multispectral image fusion whose measurements are discrete. This method can provide an alternative way to measure the image variation of multi-band image. The resulting problem is solved by alternating direction method of multipliers (ADMM), a simple but powerful algorithm for convex optimization problems. The optimization problem is handled by a splitting variables strategy, which is implemented by introducing four auxiliary variables regarding to the required fused image. The iteration related to the proposed non-local structure tensor regularization term is solved by the proximal mapping method. The solutions of other iterations are given through fast Fourier transforms. Extensive computational experiments are performed on various datasets. The results of index evaluation demonstrate the effectiveness of the proposed method. These results also indicate non-local structure tensor total variation regularization has obvious advantages over the state-of-the-art fusion methods.