A new pre-whitening transform domain LMS algorithm and its application to speech denoising

doi:10.1016/j.sigpro.2016.06.021

Signal Processing

Volume 130, January 2017, Pages 118-128

https://doi.org/10.1016/j.sigpro.2016.06.021 Get rights and content

Highlights

•
A novel pre-whitening transform domain LMS algorithm is proposed.
•
This pre-whitening is introduced before applying the transform to reinforce its decorrelation using a simple finite impulse response decorrelation filter of order one.
•
In adaptive speech denoising, the proposed algorithm outperforms the conventional algorithms for different transforms in terms of the computational complexity, mean square error convergence speed, reached steady state level, residual noise in the denoised signal, steady state excess mean square error, misadjustment and output SNR.

Abstract

In this paper, we propose a new pre-whitening transform domain LMS algorithm. The main idea is to introduce a pre-whitening using a simple finite impulse response decorrelation filter of order one before applying the transform to reinforce its decorrelation. The resulting algorithm has the advantage of using any transform even with low decorrelation. This advantage can be exploited to consider transforms having lower computational and structural complexities than those of the classical transforms. For this purpose, we also investigate the use of other transforms, namely the parametric Fourier and Hartley transforms. This investigation is accomplished by studying the eigenvalue spreads obtained by a given parametric transform and then finding the value of the parameter corresponding to the minimum eigenvalue spread, which is equivalent to the best mean square error (MSE) convergence behavior. This approach provides new attractive transforms for the proposed algorithm. Moreover, we consider the adaptive speech denoising as an application to evaluate the performance of the proposed algorithm. The comparisons between the proposed and conventional algorithms for different transforms are performed in terms of the computational complexity, MSE convergence speed, reached steady state level, residual noise in the denoised signal, steady state excess MSE, misadjustment and output SNR.

Introduction

The least mean square (LMS) algorithm is the mostly used in the adaptive filtering for its simplicity and robustness [1]. However, it has a slow convergence in the case of highly correlated input signals [2], [3]. This is due to the fact that the autocorrelation matrix of the input signal has a large eigenvalue spread. To overcome this problem by reducing the eigenvalue spread, whitening or decorrelated adaptive algorithms have been proposed for time-domain LMS [4], [5], [6]. In [4], the authors proposed a joint decorrelation of both the input and error signals. The decorrelation is achieved in the time domain using an adaptive decorrelation filter based on the concept of prediction. The resulting decorrelated normalized LMS (NLMS) structure has the advantage of improving the mean square error (MSE) convergence speed and steady state compared to the conventional LMS and NLMS algorithms. The decorrelation of the input signal can also be achieved by an orthogonal transformation followed by power normalization [7]. These two operations have led to a new configuration named transform-domain LMS (TDLMS) adaptive filters, which outperform the time-domain LMS adaptive filters in terms of the MSE convergence speed and steady state [8], [9], [10]. The orthogonal transforms such as the discrete Fourier transform (DFT), the discrete Hartley transform (DHT) and the discrete cosine transform (DCT) have been used in the TDLMS [7]. The resulting adaptive filters have been named as DFT–LMS, DHT–LMS and DCT–LMS, respectively [11]. The convergence speed of these filters depends on the used transform [8], [11], [12]. In general, the DCT–LMS filter presents a convergence performance better than those of the DFT–LMS and DHT–LMS filters [11]. This is mainly due to the fact that the DCT is suboptimal in terms of decorrelation [13], [14]. Therefore, it is highly desirable to use in the TDLMS adaptive filters a transform having better decorrelation. However, the existing transforms are fixed and hence it is not possible to increase their decorrelation ability. Since the introduction of a decorrelation in the time-domain LMS has brought interesting improvements, it is important to investigate the introduction of such a decorrelation in the TDLMS. However, to the best of author’s knowledge, this investigation has not been reported in the literature.

One of the applications of LMS adaptive filters is speech ;denoising. It is a very crucial operation to enhance the quality and intelligibility of the voice and reduce communication fatigue in modern communication systems such as mobile phones, hands-free telephony and voice-controlled systems, which are generally used in noisy environments [15]. Most of the audio frequency acoustic noises, such as computer fan noise and noise from people and cars, have low-frequency spectra and hence are colored [15], [16]. These noises often corrupt the speech signal, which is also colored. Therefore, it is important to consider colored noises in designing speech denoising techniques.

In this paper, we propose a new pre-whitening TDLMS (PW-TDLMS) algorithm. It maintains the structure of the conventional TDLMS algorithm and introduces a pre-whitenning before applying the transform. This is a novel and interesting strategy to reinforce the decorrelation of the used fixed transform in the TDLMS. The proposed pre-whitening is achieved by using a simple finite impulse response (FIR) decorrelation filter of order one based on a fixed prediction concept. The resulting PW-TDLMS has the advantage of using any transform even with low decorrelation. For this purpose, we investigate the use of other transforms such as the parametric DFT and DHT transforms [17] and study the performance of the proposed PW-TDLMS algorithm in terms of MSE convergence speed and steady state. We carry out this comparative study by considering adaptive speech denoising as an application of the proposed algorithm and show the simulation results of the proposed PW-TDLMS and TDLMS for different transforms. Moreover, we compute the eigenvalue spread of the autocorrelation matrix obtained after applying the parametric DFT or DHT transform and power normalization in the case of highly correlated Markov-1 noise, which is equivalent to the first order autoregressive (AR) process [11], for different values of the independent parameter of the transform. The transform with the value of the parameter corresponding to a good compromise between the eigenvalue spread and computational complexity is selected for the proposed PW-TDLMS. It should be noted that the eigenvalue spreads in the case of the DCT and classical DFT and DHT transforms are known in the literature [9], [11], [18].

The reminder of this paper is organized as follows. Section 2 briefly presents the parametric DFT and DHT. In Section 3, we analyze the TDLMS adaptive filter by considering the stability, steady state and convergence performances in the case of a first order AR process. For the convergence analysis, we review the eigenvalue spreads in the cases of the DFT, DHT and DCT, and then find the eigenvalue spreads in the cases of the parametric DFT and DHT. The proposed PW-TDLMS adaptive filter is developed and compared with the conventional TDLMS adaptive filter in Section 4 in terms of the eigenvalue spreads in the cases of the DCT, DFT, DHT, parametric DFT and DHT transforms. Section 5 presents the computational complexities of the DCT–LMS and proposed parametric DHT-based PW-LMS algorithms. In order to compare the performance of the proposed PW-TDLMS with that of the TDLMS for different transforms, we consider in Section 6 the speech denoising application for the cases of speech-like and real speech signals. The simulation results and comparisons are given therein in terms of the MSE convergence speed, reached steady state level, residual noise in the denoised signal, steady state excess MSE, misadjustment and output SNR. Some concluding remarks are given in Section 7.

Section snippets

Parametric discrete Fourier and Hartley transforms

The three-parameter DFT transform of a complex sequence $x (k)$ of order $N = 2^{r}$ , $r > 3$ , is defined in [17] as $X^{a, b, c} (n) = \sum_{k = 0}^{N - 1} x (k) v_{F^{a, b, c}} (nk \mod N), 0 \leq n \leq N - 1$ where $v_{F^{a, b, c}} (i)$ , $0 \leq i \leq N - 1$ , are the entries of the parametric kernel vector given by $V_{F^{a, b, c}} = [\begin{matrix} 1 & \begin{matrix} V & c & \begin{matrix} - j V & - 1 \end{matrix} \end{matrix} & \begin{matrix} - V & \begin{matrix} - c & j V \end{matrix} \end{matrix} \end{matrix}]$ with the vector $V = [\begin{matrix} W_{N}^{1} & \dots & \begin{matrix} W_{N}^{(N / 16) - 1} & a & \begin{matrix} W_{N}^{(N / 16) + 1} & \dots & \begin{matrix} W_{N}^{(N / 8) - 1} & b & \begin{matrix} W_{N}^{(N / 8) + 1} & \dots & \begin{matrix} W_{N}^{(3 N / 16) - 1} & - j a^{*} & W_{N}^{(3 N / 16) + 1} \end{matrix} \end{matrix} \end{matrix} \end{matrix} \end{matrix} \end{matrix} \begin{matrix} \dots & W_{N}^{(N / 4) - 1} \end{matrix}],$

$W_{N} = \exp (- j (2 π / N))$ , $j = \sqrt{- 1}$ , ${(.)}^{*}$ denotes the complex conjugate transpose, and $a$ , $b$ and $c$ are three nonzero parameters. The matrix form of (1) can

Transform domain LMS adaptive filter analysis

In the TDLMS adaptive filter presented in Fig. 1, the correlated tap delayed input vector $x_{k} = {[x_{k}, x_{k - 1}, \dots, x_{k - N + 1}]}^{T}$ is transformed into the vector $u_{k} = T_{N} x_{k}$ using a fixed data-independent orthogonal transform matrix $T_{N}$ , where N is the filter length. The resulting vector $u_{k}$ is less correlated then the vector $x_{k}$ . The transformed vector is then power normalized using the diagonal matrix $P_{k}$ defined in [9], [11], [19] as $P_{k} = diag ([σ_{k}^{2} (i), i = 0,1, \dots, N - 1])$ where $σ_{k}^{2} (i)$ is the power estimate of the ith input $u_{k} (i)$

Proposed pre-whitening transform domain LMS adaptive filter

In this section, we propose a new pre-whitening TDLMS (PW-TDLMS) adaptive noise canceler by introducing a pre-whitening filter $g$ at the input of the TDLMS as presented in Fig. 4. The purpose of this filter is to obtain a decorrelated version ${\tilde{x}}_{k}$ of the signal $x_{k}$ at the input of TDLMS. This has a direct impact on further reducing the eigenvalue spread of the autocorrelation matrix $S_{N}$ obtained after transformation and power-normalization of the signal ${\tilde{x}}_{k}$ in TDLMS and consequently on accelerating

Computational complexity

According to [24], the computational complexity of the conventional TDLMS (without including the complexity of the used transform) is $6 N + 1$ multiplications and $3 N$ additions for each iteration. The computational complexity of the proposed PW-TDLMS is increased by only one addition and one multiplication due to the first order pre-whitening filter. Therefore, by including the complexity of the transform, the computational complexities of the DCT–LMS and proposed PW- ${DHT}^{- π / 6}$ -LMS are given in Table 3

Simulation results

In this section, we present a set of simulations to test and compare the performance of the proposed PW-TDLMS with that of the TDLMS using the DCT, DFT, DHT, DFT^-π/6 and DHT^-π/6 transforms. The algorithms are implemented as adaptive noise cancelers as shown in Fig. 4.

Conclusion

In this paper, a new pre-whitening transform domain LMS (PW-TDLMS) algorithm has been developed by introducing a pre-whitening using a simple finite impulse response decorrelation filter of order one before applying the transform to reinforce its decorrelation. It has been shown that the proposed PW-TDLMS algorithm significantly decreases the eigenvalue spreads compared to the conventional TDLMS for all considered transforms, namely the DCT, DFT, DHT, and parametric DFT and DHT. In both the

References (27)

E.V. Kuhn et al.
Analysis of the TDLMS algorithm operating in a nonstationary environment
Digit. Signal Process.
(2015)
J. Lu et al.
A modified frequency-domain block LMS algorithm with guaranteed optimal steady-state performance
Signal Process.
(2014)
K. Mayyas
New transform-domain adaptive algorithms for acoustic echo cancellation
Digit. Signal Process.
(2003)
D.I. Kim et al.
Performance analysis of the DCT-LMS adaptive filtering algorithm
Signal Process.
(2000)
K. Ngo et al.
Improved prediction error filters for adaptive feedback cancellation in hearing aids
Signal Process.
(2013)
Y. Hu et al.
Subjective comparison and evaluation of speech enhancement algorithms
Speech Commun.
(2007)
S. Haykin
Adaptive Filter Theory
(1996)
D.I. Kim et al.
Performance analysis of signed self-orthogonalizing adaptive lattice filter
IEEE Trans. Circuits Syst. II: Analog Digit. Signal Process.
(2000)
S. Gazor et al.
Adaptive filtering with decorrelation for coloured AR environments
IEE Proc. – Vis. Image Signal Process.
(2005)
S.C. Douglas et al.
Self-whitening algorithms for adaptive equalization and deconvolution
IEEE Trans. Signal Process.
(1999)

M. Mboup et al.

LMS coupled adaptive prediction and system identification: a statistical model and transient mean analysis

IEEE Trans. Signal Process.

(1994)

S. Narayan et al.

Transform domain LMS algorithm

IEEE Trans. Acoust. Speech Signal Process.

(1983)

F. Beaufays

Transform-domain adaptive filters: an analytical approach

IEEE Trans. Signal Process.

(1995)

Cited by (16)

On-line harmonic signal denoising from the measurement with non-stationary and non-Gaussian noise
2022, Signal Processing
Citation Excerpt :
Harmonic signals exist in many scenarios, such as mechanical condition monitoring [1,2], helicopter detection [3,4], speech acquisition [5–7], biomedical diagnosis [8], etc.
Harmonic denoising is one of the important preprocessing steps before extracting harmonic signal characteristics. Several signal processing techniques have been developed and applied for denoising the harmonics, which assume that the noise follows a Gaussian distribution and is stationary. However, the noise is often not so simple as a Gaussian distribution, and it could be non-Gaussian and non-stationary in most practical scenarios. A novel online denoising method for the harmonic signal with non-stationary complex noises based on the Bayesian Maximum a Posteriori (MAP) framework is proposed in this paper. The measured signal is divided equally into several frames. Then these frames are transformed into the time-frequency domain by the Short-Time Fourier transform (STFT) and are assumed to be the sum of a low-rank matrix and a noise matrix. The online model of the low-rank matrix and the noise matrix is then constructed between the frames. The online Gaussian mixture model (GMM) and low-rank matrix factorization are performed on the measurement matrix in the complex number domain to reconstruct the harmonic signal. The performance of the proposed method is validated in the simulations. The non-Gaussian and non-stationary noise can be removed more effectively, and the proposed algorithm can improve the frequency estimation accuracy.
Peak-aware guided filtering for spectrum signal denoising
2022, Chemometrics and Intelligent Laboratory Systems
Citation Excerpt :
The analysis of spectrum signals includes detecting and measuring peak value, reducing noise, improving the resolution of overlapping peak value, compensating instrument pseudo-factor, and decomposing complex signal into its component parts [1]. It is particularly important to identify and process the peak of a signal in a wide range of applications, including shape and the maxima-minima (QRS and ST-T level) in electrocardiogram (ECG) processing [2], seismological research [3], speech signal processing [4,5], and so on. However, signals are usually polluted by noise during data acquisition, storage and transmission, and the noise is larger at the peak [1].
In the analysis of spectrum signals, it is especially important to identify the peaks of a spectrum signal. Due to various noise, spectrum signals must be smoothed in advance, while preserving peaks information. In this paper, we propose a peak-aware guided filtering (PAGF) for peak-preserving smoothing of a noisy spectrum signal. The content of a guidance signal is taken into account in the filtering process, which can provide smooth structure information and peak position of the input signal. The guidance signal is generated from the input signal by a nonlinear diffusion filtering (NDF) since the output of NDF provides sufficiently smooth structure and accurate peak position for the input signal. The proposed PAGF is validated by experiments on commonly tested signals, in comparison to previous peak-preserving smoothing methods. The results show that the PAGF can better preserve the peak height, width and position of the input spectrum signal while removing the noise.
Applying segmentation and classification to improve performance of smoothing
2021, Digital Signal Processing: A Review Journal
Citation Excerpt :
For example, in seismological research, the resolution of seismic waveforms will directly affect the reliability of seismic data [1]. In speech signal processing, high-quality speech signals can help improve speech recognition [2,3]. The image restoration is to restore a clear scene from an image degraded by optical system aberration, atmospheric disturbance, motion, defocus, and other noise [4].
Combining segmentation, classification, and time-fractional diffusion filtering, an excellent smoothing method of peak-preserving is proposed. First, the signal is divided into equal-length segments. Second, these segments are classified according to the similarity. Third, similar segments in the same class are stacked into a two-dimensional array, and then they are filtered by the two-dimensional discrete cosine transform. As a result, a preliminary smoothed signal is obtained. Finally, the preliminary smoothed signal is filtered in the time domain using the time-fractional diffusion filtering for peak-preserving. Thus, the final smoothed signal can be obtained. As a validation of the proposed method, comparisons are performed with different commonly-used methods such as the time-fractional diffusion method (TFDM), regularized method (RegM), Savitzky-Golay method (SGM) and wavelet method (WM). The results show that the proposed method has a better signal-to-noise ratio (SNR) and root-mean-square error (RMSE) than other classical filtering methods.
Wireless signal enhancement based on generative adversarial networks
2020, Ad Hoc Networks
Citation Excerpt :
Linear methods of signal enhancement have been widely used for noise removal due to their relative simplicity. Typical techniques used for signal enhancement in received signals are least mean square (LMS) [2,3] and its variants [4]. However, these algorithms have limited performance on nonlinear signals and cannot achieve a global optimum solution for noise and interference elimination.
Compared to traditional signal enhancement strategies in wireless communication, the emerging route based on deep learning has been showing better potential adaptivity to dynamic effects of noise and interference conditions. In this paper, we design and establish a signal enhancement network based on the specialized Generative Adversarial Networks, which can adaptively learn the characteristics of signals and achieve a signal enhancement in time-varying systems. We design a customized object function, and the raw time-domain signal is added to the network as a condition to achieve the state of the art enhancement effect with the effect that the symbol information remains unchanged. Besides its robust learning ability to dynamic channel effects on the signal, it also has the excellently adversarial ability for signal jitter and skews, the network can still track the signal cognitively. Experiments show that our proposed network’s wireless signal enhancement effect is state of the art of all methods.
A New Method for Evaluating Natural Gas Pipelines Based on ICEEMDAN-LMS: A View of Noise Reduction in Defective Pipelines
2023, Applied Sciences (Switzerland)
Enhancing Biomedical ReQA With Adversarial Hard In-Batch Negative Samples
2023, IEEE/ACM Transactions on Computational Biology and Bioinformatics

View all citing articles on Scopus

View full text

A new pre-whitening transform domain LMS algorithm and its application to speech denoising

Highlights

Abstract

Introduction

Section snippets

Parametric discrete Fourier and Hartley transforms

Transform domain LMS adaptive filter analysis

Proposed pre-whitening transform domain LMS adaptive filter

Computational complexity

Simulation results

Conclusion

Digit. Signal Process.

Signal Process.

Digit. Signal Process.

Signal Process.

Signal Process.

Speech Commun.

Adaptive Filter Theory

Performance analysis of signed self-orthogonalizing adaptive lattice filter

IEEE Trans. Circuits Syst. II: Analog Digit. Signal Process.

Adaptive filtering with decorrelation for coloured AR environments

IEE Proc. – Vis. Image Signal Process.

Self-whitening algorithms for adaptive equalization and deconvolution

IEEE Trans. Signal Process.

LMS coupled adaptive prediction and system identification: a statistical model and transient mean analysis

IEEE Trans. Signal Process.

Transform domain LMS algorithm

IEEE Trans. Acoust. Speech Signal Process.

Transform-domain adaptive filters: an analytical approach

IEEE Trans. Signal Process.