Denoising by second order statistics
Highlights
► We present a new approach for deducing data fidelity terms for variational denoising methods. ► Our approach extends the classical MAP approach by an additional variable transformation. ► Hereby, the removed noise is forced to follow the statistics of the assumed noise model. ► Examples for additive Gaussian noise show the good properties of our new approach. ► It is demonstrated to be particularly well suited for data containing high frequency structures.
Introduction
Measured signals and images are often corrupted by noise which makes their denoising and reconstruction a central aim in digital signal and image processing. Especially for data of low quality reliable and robust reconstruction methods are required. In the last decades many methods have been proposed for denoising corrupted data. A commonly applied approach is to solve a variational problem, where one has to minimize a functional consisting of a data fidelity term and a regularization term. The functional is usually deduced by a maximum a posteriori strategy, which requires some knowledge about the noise statistics and the distribution of the original data. In the literature, e.g., when considering detector noise or in case of high photon counts, where the Poisson distribution can be well approximated by a Gaussian one, it is often assumed that the corrupted data follows an additive noise model. This means that our given noisy data is modeled as where is the unknown noise-free data and the noise vector is a realization of a random vector defined with respect to a continuous probability space . As usual, represents here the sample space, denotes the and represents the probability measure. The vectors g and f0 are assumed to be realizations of independent N-dimensional random vectors and , respectively, so that .
To deduce an estimate of f0 by a maximum a posteriori (MAP) strategy, one usually setscf., e.g., [1], [2], [3], where is the conditional probability density function for observing f given G=g. By Bayes' theorem we know thathere is the so-called likelihood, which is usually closely related to the density of the noise, pF is some a priori density of F and pG is the density of G. Since we consider additive noise, it holds that , where and denotes the density of . Moreover, inserting (2) in (1) yieldshere the terms and imply that we search for the most likely vectors and under the condition that . If the components of the random vector are pairwise independent and identically distributed (i.i.d.) as it is often assumed, thenFor the special case that , this leads toTo determine , at least some estimate of the a priori density pF is required. Assuming that for some constant and a nonnegative function , the minimization problem (3) with (5) is finally equivalent tohere the amount of filtering is controlled by the parameter , which steers the influence of the two terms within the functional. If J is assumed to be , where D is a discrete first derivative operator, we obtain by this approach the regularization method proposed by Tikhonov and Miller (TM) in [4], which we will shortly call MAP-TM. By this choice for J the initial signal is assumed to have small first derivatives, i.e., to be of a certain degree of smoothness (in H1 for the continuous setting). Unfortunately, if the signal contains jumps, the TM regularization will oversmooth them. To overcome this drawback, J is often set to , which is the discrete one-dimensional version of the total variation regularizer (TV). The corresponding denoising method (6) leads to the classical approach of Rudin et al. [5], which is well known for its discontinuity preserving properties. In the following, we will refer to this method as MAP-TV and we will use it as well as the MAP-TM approach as reference methods for our numerical experiments.
Now, if we forget about the regularization term for a moment and have again a closer look at our data fidelity term in (4), where is assumed to be i.i.d., we see that this data fidelity term is minimal whenever all components maximize . Consequently, without the regularization term or equivalently for , our reconstructed noise vector would be a constant vector of value and thus, . These estimates may seem reasonable for a signal length N close to one. However, since the vector is i.i.d., we may expect for larger N that the empirical distribution of the components of our estimated noise vector resembles the distribution of . In principal, to check how good a set of samples coincides with a given distribution we could for example apply the Kolmogorov–Smirnov [6] or the Anderson–Darling test [7].
Outline. In the following, we show that it is possible to modify the standard MAP approach so that the reconstructed noise vector is forced to resemble the statistical properties of the assumed noise model. To this purpose, a suitable variable transformation is applied to the random vector before computing the MAP estimates. In Section 2 our new approach is presented and we investigated two different transformations with respect to their benefits and shortcomings. These transformations incorporate estimates of higher moments of into the resulting minimization problems to force the reconstructed noise vector to have the desired statistics. In Section 3 we discuss a first implementation of our approach for one-dimensional data and present numerical results. Finally, we summarize our new findings and finish with concluding remarks in Section 4.
Related work. The idea of using higher-order statistics for restoring corrupted data can for example be found in blind source separation techniques, cf. [8], [9], [10]. Moreover, it has been used for wavelet based denoising methods as, e.g., presented in [11], [12], [13] and in approaches combining the empirical mode decomposition with higher-order statistical estimates, see, e.g., [14]. In contrast to the works of Hofinger [15], [16] we introduce here a representation of the noise distribution that depends both on moments and especially on the correlation of the random variables . We also embed noise correlation, cf. [17], in a concise formalism that allows to achieve de-correlated estimates of the original noise components if the are independent, a result that is achieved to some extent ad hoc with non-local means [18], [19] according to empirical studies.
Section snippets
A new denoising approach
For simplicity, we assume in the following that the random variables are again i.i.d. with expectation value and variance . Hence, the components of the vector can be considered as samples of the same random variable. Computing the MAP estimator and the corresponding noise vector from Eq. (3) is equivalent to solving the minimization problemfor the given noisy data . Since the term does not
Minimizaton problem
To demonstrate the capability of our new denoising approach introduced in (8), (14) we proceed with numerical examples. In the following, we want to denoise signals corrupted by additive white Gaussian noise by minimizing the second order statistics functionalwith respect to so that is our reconstruction of the original signal f0. Here, the prior term from (8) is set to and guarantees that the reconstructed
Conclusions
We have shown that the standard maximum likelihood estimation approach for denoising signals can be generalized by introducing an additional transformation of the random variables modeling the noise. This transformation allows to consider also pixel correlations within the noise vectors and helps to obtain a reconstructed noise vector, which resembles the statistical properties of the assumed noise model. The transformation of our choice leads to a nonconvex minimization problem. A local
References (30)
- et al.
Nonlinear total variation based noise removal algorithms
Physica D
(1992) - et al.
Blind separation of sources using higher-order cumulants
Signal Processing
(1999) - et al.
Blind separation of any source distributions via high-order statistics
Signal Processing
(2007) - et al.
Wavelet packets and de-noising based on higher-order-statistics for transient detection
Signal Processing
(2001) - et al.
ECG signal denoising using higher order statistics in wavelet subbands
Biomedical Signal Processing and Control
(2010) - et al.
A moment-based nonlocal-means algorithm for image denoising
Information Processing Letters
(2009) - A.B. Hamza, H. Krim, A variational approach to maximum a posteriori estimation for image denoising, in: M.A.T....
- et al.(2005)
Model distortions in Bayesian MAP reconstruction
Inverse Problems and Imaging
(2007)Regularization of incorrectly posed problems
Soviet Mathematics Doklady
(1963)