Single image super-resolution via subspace projection and neighbor embedding

Abstract

In this paper, we present a novel learning-based single image super-resolution algorithm to address the problems of inefficient learning and improper estimation in coping with nonlinear high-dimensional feature data. Our method named as subspace projection and neighbor embedding (SPNE) first projects the high-dimensional data into two different subspaces respectively, i.e., kernel principal component analysis (KPCA) subspace and modified locality preserving projection (MLPP) subspace to obtain the global and local structures of data. In an optimal low-dimensional feature space, the k-nearest neighbors of each input low-resolution (LR) image patch can be found for efficient learning. Then within similarity measures and proportional factors, the k embedding weights are used to estimate high-frequency information from a training dataset. Finally, we apply iterative back projection (IBP) to further enhance the super-resolution results. Experiments on simulative and actual LR images demonstrate that the proposed approach outperforms the existing NE-based super-resolution methods in terms of visual quality and some selected objective metrics.

Introduction

.In many practical applications, such as computer vision, medical imaging, safety surveillance, and preprocessing of hyper-spectral fusion, high resolution (HR) images are usually desired to deal with. However, images from the existing digital acquisition system do not meet people׳s requirement. In order to alleviate the contradiction, image super-resolution (SR) reconstruction technique becomes more and more popular to estimate a target HR image from one or more low resolution (LR) images [1], [2].

In recent years, image super-resolution algorithms have achieved great success. Generally speaking, those methods can be separated into three categories: interpolation-based [3], [4], multi-frame-based [5], [6], [7], and learning-based [8]. Although interpolation-based methods can enlarge the size of image in any magnification factor, the results often have blurring artifacts. The multi-frame-based methods use a simplified imaging degradation model with some terms (e.g., motion warping, optical dimming, low sampling, and random noise). However, these methods often depend on registration precision among LR images and have difficulty in estimating a HR image when magnification factor is large. Relatively, learning-based methods do not face the above problems. The goal of these methods is to estimate the missing high frequency information from a training set consisting of many LR–HR image patch pairs. Freeman et al. [9] first proposed a learning-based SR algorithm using a Markov random field model to learn the relationship between LR and corresponding HR image patches. But this method is very sensitive to examples in the training set. Chang et al. [10] developed a neighbor embedding (NE) based method, which introduced locally linear embedding (LLE) to estimate high frequency details linearly combining the k-nearest neighbors in the training set. Then, many researchers extended the NE-based method [11], [12], [13], [14], [15], [16], [17]. Chan et al. [13] emphasized on edge detection and feature selection using first-order gradient feature and normalized luminance feature together to find the k-nearest neighbors. In order to represent complicated texture structures, Zhang et al. [14] proposed a novel NE-based SR algorithm through a partially supervised distance measurement with class information. Recently, Gao et al. [15] applied a joint learning technique to map LR and HR feature spaces onto a unified feature subspace by grouping patch pairs. Yang et al. [18] presented a sparse coding based SR approach to generate HR output from an over-complete dictionary. Then, they proposed a coupled dictionary training method [29] for reconstructing well the underlying HR image patches within the coupled dictionary. Jia et al. [30] introduced a local parametric regression approach over the learned coupled dictionaries across source and target image spaces using sparse representation. Gao et al. [31] presented a sparse neighbor selection scheme simultaneously combining neighbor search process and reconstruction weights estimation process.

In order to represent LR image patches, we need to extract their texture features. Due to high dimensionality [19], the extracted features are often nonlinear. Simply applying linear transformation for high-dimensional data may lead to an over-fitting problem [20] during the training process, as well as provide poor results. Kernel techniques [21] are capable of dealing with nonlinear problem. However, after applying kernel techniques, the feature data has higher dimensionality and greater redundancy as well. To further address this problem, dimensionality reduction technique is a feasible way. In general, principal component analysis (PCA) is one of representative global dimensionality reduction methods, while locality preserving projection (LPP) is a local dimensionality reduction method [22]. Global and local structures are both significant for learning high-dimensional data. Therefore, we exploit kernel PCA (KPCA) and modified LPP (MLPP) to reserve both global and local structures of high-dimensional feature data. Within the proposed approach, we can adaptively find the 0k-nearest neighbors with higher matching precision for each test LR patch.

Our approach is based on NE, representing a HR image patch as a linear combination of its neighbors. Here we propose a novel subspace projection and neighbor embedding (SPNE) for single image super-resolution reconstruction. The neighbor search is implemented by KPCA transformation, followed by MLPP transformation. Then, embedding weights are estimated by a new metric, combining similarity measures and proportional factors. Using the learnt high-frequency details, the target HR image patches can be obtained corresponding to the test LR image patches. In order to achieve better results in SR reconstruction, we apply iterative back projection (IBP) for enhancement. The proposed approach preserves global and local structures of feature data using KPCA and MLPP transformations. For each test LR image patch, its k-nearest neighbors can be searched with high precision. The novel metric for embedding weights is crucial in estimating target HR image patches. Experimental results on simulative and actual test images demonstrate that the proposed SPNE algorithm achieves much higher SR reconstruction performance than the existing SR methods [13], [14], [15].

The remainder of this paper is organized as follows. Section 2 describes kernel principal component analysis (KPCA). The SPNE-based image super-resolution algorithm is proposed in Section 3. Sections 4 and 5 present experimental results and conclusion, respectively.