Skip to main content

About this book

This book constitutes the proceedings of the 9th International Conference on Latent Variable Analysis and Signal Separation, LVA/ICA 2010, held in St. Malo, France, in September 2010.

The 25 papers presented were carefully reviewed and selected from over hundred submissions. The papers collected in this volume demonstrate that the research activity in the field continues to gather theoreticians and practitioners, with contributions ranging range from abstract concepts to the most concrete and applicable questions and considerations. Speech and audio, as well as biomedical applications, continue to carry the mass of the considered applications. Unsurprisingly the concepts of sparsity and non-negativity, as well as tensor decompositions, have become predominant, reflecting the strongactivity on these themes in signal and image processing at large.

Table of Contents


Speech and Audio Applications

Blind Source Separation Based on Time-Frequency Sparseness in the Presence of Spatial Aliasing

In this paper, we propose a novel method for blind source separation (BSS) based on time-frequency sparseness (TF) that can estimate the number of sources and time-frequency masks, even if the spatial aliasing problem exists. Many previous approaches, such as degenerate unmixing estimation technique (DUET) or observation vector clustering (OVC), are limited to microphone arrays of small spatial extent to avoid spatial aliasing. We develop an offline and an online algorithm that can both deal with spatial aliasing by directly comparing observed and model phase differences using a distance metric that incorporates the phase indeterminacy of 2


and considering all frequency bins simultaneously. Separation is achieved using a linear blind beamformer approach, hence musical noise common to binary masking is avoided. Furthermore, the offline algorithm can estimate the number of sources. Both algorithms are evaluated in simulations and real-world scenarios and show good separation performance.

Benedikt Loesch, Bin Yang

Adaptive Time-Domain Blind Separation of Speech Signals

We present an adaptive algorithm for blind audio source separation (BASS) of moving sources via Independent Component Analysis (ICA) in time-domain. The method is shown to achieve good separation quality even with a short demixing filter length (L = 30). Our experiments show that the proposed adaptive algorithm can outperform the off-line version of the method (in terms of the average output SIR), even in the case in which the sources do not move, because it is capable of better adaptation to the nonstationarity of the speech.

Jiří Málek, Zbyněk Koldovský, Petr Tichavský

Time-Domain Blind Audio Source Separation Method Producing Separating Filters of Generalized Feedforward Structure

Time-domain methods for blind separation of audio signals are preferred due to their lower demand for available data and the avoidance of the permutation problem. However, their computational demands increase rapidly with the length of separating filters due to the simultaneous growth of the dimension of an

observation space

. We propose, in this paper, a general framework that allows the time-domain methods to compute separating filters of theoretically infinite length without increasing the dimension. Based on this framework, we derive a generalized version of the time-domain method of Koldovský and Tichavský (2008). For instance, it is demonstrated that its performance might be improved by 4dB of SIR using the Laguerre filter bank.

Zbyněk Koldovský, Petr Tichavský, Jiří Málek

Subband Blind Audio Source Separation Using a Time-Domain Algorithm and Tree-Structured QMF Filter Bank

T-ABCD is a time-domain method for blind linear separation of audio sources proposed by Koldovský and Tichavský (2008). The method produces short separating filters (5-40 taps) and works well with signals recorded at the sampling frequency of 8-16 kHz. In this paper, we propose a novel subband-based variant of T-ABCD, in which the input signals are decomposed into subbands using a tree-structured QMF filter bank. T-ABCD is then applied to each subband in parallel, and the separated subbands are re-ordered and synthesized to yield the final separated signals. The analysis filter of the filter bank is carefully designed to enable maximal decimation of signals without aliasing. Short filters applied within subbands then result in sufficiently long filters in fullband. Using a reasonable number of subbands, the method yields improved speed, stability and performance at an arbitrary sampling frequency.

Zbyněk Koldovský, Petr Tichavský, Jiří Málek

A General Modular Framework for Audio Source Separation

Most of audio source separation methods are developed for a particular scenario characterized by the number of sources and channels and the characteristics of the sources and the mixing process. In this paper we introduce a general modular audio source separation framework based on a library of flexible source models that enable the incorporation of prior knowledge about the characteristics of each source. First, this framework generalizes several existing audio source separation methods, while bringing a common formulation for them. Second, it allows to imagine and implement new efficient methods that were not yet reported in the literature. We first introduce the framework by describing the flexible model, explaining its generality, and summarizing our modular implementation using a Generalized Expectation-Maximization algorithm. Finally, we illustrate the above-mentioned capabilities of the framework by applying it in several new and existing configurations to different source separation scenarios.

Alexey Ozerov, Emmanuel Vincent, Frédéric Bimbot

Adaptive Segmentation and Separation of Determined Convolutive Mixtures under Dynamic Conditions

In this paper, we propose a method for blind source separation (BSS) of convolutive audio recordings with short blocks of stationary sources, i.e. dynamically changing source activity but no source movements. It consists of a time-frequency sparseness based localization step to identify segments with stationary sources whose number is equal to the number of microphones. We then use a frequency domain independent component analysis (ICA) algorithm that is robust to short data segments to separate each identified segment. In each segment we solve the permutation problem using the state coherence transform (SCT). Experimental results using real room impulse responses show a good separation performance.

Benedikt Loesch, Bin Yang

Blind Speech Extraction Combining Generalized MMSE STSA Estimator and ICA-Based Noise and Speech Probability Density Function Estimations

In this paper, we propose a new blind speech extraction method combining ICA-based dynamic noise estimation and a generalized minimum mean-square-error short-time spectral amplitude estimator of the target speech. To deal with various types of speech signals with different probability density functions (p.d.f.), we also introduce a spectral-subtraction-based speech p.d.f. estimation and provide a theoretical justification of the proposed approach. We conduct an experiment in an actual railway-station environment, and show the improved noise reduction of the proposed method by objective and subjective evaluations.

Hiroshi Saruwatari, Ryoi Okamoto, Yu Takahashi, Kiyohiro Shikano

Blind Estimation of Locations and Time Offsets for Distributed Recording Devices

This paper presents a blind technique to estimate locations and recording time offsets of distributed recording devices from asynchronously recorded signals. In our method, locations of sound sources and recording devices, and the recording time offsets are estimated from observed time differences of arrivals (TDOAs) by decreasing the mean squared errors. The auxiliary-function-based updates guarantee the monotonic decrease of the objective function at each iteration. The TDOAs are estimated by the generalized cross correlation technique. The validity of our approach is shown by experiments in real environment, where locations of seven sound sources and eight microphones and eight time offsets were estimated from signals recorded by four stereo IC recorders in reverberant rooms.

Keisuke Hasegawa, Nobutaka Ono, Shigeki Miyabe, Shigeki Sagayama

Speech Separation via Parallel Factor Analysis of Cross-Frequency Covariance Tensor

This paper considers separation of convolutive speech mixtures in frequency-domain within a tensorial framework. By assuming that components associated with neighboring frequency bins of the same source are still correlated, a set of cross-frequency covariance tensors with trilinear structure are established, and an algorithm consisting of consecutive parallel factor (PARAFAC) decompositions is developed. Each PARAFAC decompositon used in the proposed method can simultaneously estimate two neighboring frequency responses, one of which is a common factor with the subsequent cross-frequency covariance tensor, and thus could be used to align the permutations of the estimates in all the PARAFAC decompositions. In addition, the issue of identifiability is addressed, and simulations with synthetic speech signals are provided to verify the efficacy of the proposed method.

Xiao-Feng Gong, Qiu-Hua Lin

Under-Determined Reverberant Audio Source Separation Using Local Observed Covariance and Auditory-Motivated Time-Frequency Representation

We consider the local Gaussian modeling framework for under-determined convolutive audio source separation, where the spatial image of each source is modeled as a zero-mean Gaussian variable with full-rank time- and frequency-dependent covariance. We investigate two methods to improve the accuracy of parameter estimation, based on the use of local observed covariance and auditory-motivated time-frequency representation. We derive an iterative expectation-maximization (EM) algorithm with a suitable initialization scheme. Experimental results over stereo synthetic reverberant mixtures of speech show the effectiveness of the proposed methods.

Ngoc Q. K. Duong, Emmanuel Vincent, Rémi Gribonval

Crystal-MUSIC: Accurate Localization of Multiple Sources in Diffuse Noise Environments Using Crystal-Shaped Microphone Arrays

This paper presents crystal-MUSIC, a method for DOA estimation of multiple sources in the presence of diffuse noise. MUSIC is well known as a method for the estimation of the DOAs of multiple sources but

is not very robust to diffuse noise from many directions, because the covariance structure of such noise is not spherical

. Our method makes it possible for MUSIC to accurately estimate the DOAs by removing the contribution of diffuse noise from the spatial covariance matrix. This denoising is performed in two steps: 1) denoising of the off-diagonal entries via a blind noise decorrelation using crystal-shaped arrays, and 2) denoising of the diagonal entries through a low-rank matrix completion technique.

The denoising process does not require the spatial covariance matrix of diffuse noise to be known, but relies only on an isotropy feature of diffuse noise.

Experimental results with real-world noise show that the DOA estimation accuracy is substantially improved compared to the conventional MUSIC.

Nobutaka Ito, Emmanuel Vincent, Nobutaka Ono, Rémi Gribonval, Shigeki Sagayama

Convolutive Signal Separation

Consistent Wiener Filtering: Generalized Time-Frequency Masking Respecting Spectrogram Consistency

Wiener filtering is one of the most widely used methods in audio source separation. It is often applied on time-frequency representations of signals, such as the short-time Fourier transform (STFT), to exploit their short-term stationarity, but so far the design of the Wiener time-frequency mask did not take into account the necessity for the output spectrograms to be consistent, i.e., to correspond to the STFT of a time-domain signal. In this paper, we generalize the concept of Wiener filtering to time-frequency masks which can involve manipulation of the phase as well by formulating the problem as a consistency-constrained Maximum-Likelihood one. We present two methods to solve the problem, one looking for the optimal time-domain signal, the other promoting consistency through a penalty function directly in the time-frequency domain. We show through experimental evaluation that, both in oracle conditions and combined with spectral subtraction, our method outperforms classical Wiener filtering.

Jonathan Le Roux, Emmanuel Vincent, Yuu Mizuno, Hirokazu Kameoka, Nobutaka Ono, Shigeki Sagayama

Blind Separation of Convolutive Mixtures of Non-stationary Sources Using Joint Block Diagonalization in the Frequency Domain

We recently proposed a new method based on spectral decorrelation for blindly separating linear instantaneous mixtures of non-stationary sources. In this paper, we propose a generalization of this method to FIR convolutive mixtures using a second-order approach based on block-diagonalization of covariance matrices in the frequency domain. Contrary to similar time or time-frequency domain methods, our approach requires neither the piecewise stationarity of the sources nor their sparseness. The simulation results show the better performance of our approach compared to these methods.

Hicham Saylani, Shahram Hosseini, Yannick Deville

Single Microphone Blind Audio Source Separation Using EM-Kalman Filter and Short+Long Term AR Modeling

Blind Source Separation (BSS) arises in a variety of fields in speech processing such as speech enhancement, speakers diarization and identification. Generally, methods for BSS consider several observations of the same recording. Single microphone analysis is the worst underdetermined case, but, it is also the more realistic one. In this article, the autoregressive structure (short term prediction) and the periodic signature (long term prediction) of voiced speech signal are modeled and a linear state space model with unknown parameters is derived. The Expectation Maximization (EM) algorithm is used to estimate these unknown parameters and therefore help source separation.

Siouar Bensaid, Antony Schutz, Dirk T. M. Slock

The 2010 Signal Separation Evaluation Campaign (SiSEC2010)

The 2010 Signal Separation Evaluation Campaign (SiSEC2010): Audio Source Separation

This paper introduces the audio part of the 2010 community-based Signal Separation Evaluation Campaign (SiSEC2010). Seven speech and music datasets were contributed, which include datasets recorded in noisy or dynamic environments, in addition to the SiSEC2008 datasets. The source separation problems were split into five tasks, and the results for each task were evaluated using different objective performance criteria. We provide an overview of the audio datasets, tasks and criteria. We also report the results achieved with the submitted systems, and discuss organization strategies for future campaigns.

Shoko Araki, Alexey Ozerov, Vikrham Gowreesunker, Hiroshi Sawada, Fabian Theis, Guido Nolte, Dominik Lutter, Ngoc Q. K. Duong

The 2010 Signal Separation Evaluation Campaign (SiSEC2010): Biomedical Source Separation

We present an overview of the biomedical part of the 2010 community-based Signal Separation Evaluation Campaign (SiSEC2010), coordinated by the authors. In addition to the audio tasks which have been evaluated in the previous SiSEC, SiSEC2010 considered several biomedical tasks. Here, three biomedical datasets from molecular biology (gene expression profiles) and neuroscience (EEG) were contributed. This paper describes the biomedical datasets, tasks and evaluation criteria. This paper also reports the results of the biomedical part of SiSEC2010 achieved by participants.

Shoko Araki, Fabian Theis, Guido Nolte, Dominik Lutter, Alexey Ozerov, Vikrham Gowreesunker, Hiroshi Sawada, Ngoc Q. K. Duong


Use of Bimodal Coherence to Resolve Spectral Indeterminacy in Convolutive BSS

Recent studies show that visual information contained in visual speech can be helpful for the performance enhancement of audio-only blind source separation (BSS) algorithms. Such information is exploited through the statistical characterisation of the coherence between the audio and visual speech using, e.g. a Gaussian mixture model (GMM). In this paper, we present two new contributions. An adapted expectation maximization (AEM) algorithm is proposed in the training process to model the audio-visual coherence upon the extracted features. The coherence is exploited to solve the permutation problem in the frequency domain using a new sorting scheme. We test our algorithm on the XM2VTS multimodal database. The experimental results show that our proposed algorithm outperforms traditional audio-only BSS.

Qingju Liu, Wenwu Wang, Philip Jackson

Non-negative Hidden Markov Modeling of Audio with Application to Source Separation

In recent years, there has been a great deal of work in modeling audio using non-negative matrix factorization and its probabilistic counterparts as they yield rich models that are very useful for source separation and automatic music transcription. Given a sound source, these algorithms learn a dictionary of spectral vectors to best explain it. This dictionary is however learned in a manner that disregards a very important aspect of sound, its temporal structure. We propose a novel algorithm, the non-negative hidden Markov model (N-HMM), that extends the aforementioned models by jointly learning several small spectral dictionaries as well as a Markov chain that describes the structure of changes between these dictionaries. We also extend this algorithm to the non-negative factorial hidden Markov model (N-FHMM) to model sound mixtures, and demonstrate that it yields superior performance in single channel source separation tasks.

Gautham J. Mysore, Paris Smaragdis, Bhiksha Raj

Nonnegative Matrix Factorization with Markov-Chained Bases for Modeling Time-Varying Patterns in Music Spectrograms

This paper presents a new sparse representation for polyphonic music signals. The goal is to learn the time-varying spectral patterns of musical instruments, such as attack of the piano or vibrato of the violin in polyphonic music signals without any prior information. We model the spectrogram of music signals under the assumption that they are composed of a limited number of components which are composed of Markov-chained spectral patterns. The proposed model is an extension of nonnegative matrix factorization (NMF). An efficient algorithm is derived based on the auxiliary function method.

Masahiro Nakano, Jonathan Le Roux, Hirokazu Kameoka, Yu Kitano, Nobutaka Ono, Shigeki Sagayama

An Experimental Evaluation of Wiener Filter Smoothing Techniques Applied to Under-Determined Audio Source Separation

Multichannel under-determined source separation is often carried out in the time-frequency domain by estimating the source coefficients in each time-frequency bin based on some sparsity assumption. Due to the limited amount of data, this estimation is often inaccurate and results in musical noise artifacts. A number of single- and multichannel smoothing techniques have been introduced to reduce such artifacts in the context of speech denoising but have not yet been systematically applied to under-determined source separation. We present some of these techniques, extend them to multichannel input when needed, and compare them on a set of speech and music mixtures. Many techniques initially designed for diffuse and/or stationary interference appear to fail with directional nonstationary interference. Temporal covariance smoothing provides the best tradeoff between artifacts and interference and increases the overall signal-to-distortion ratio by up to 3 dB.

Emmanuel Vincent

Auxiliary-Function-Based Independent Component Analysis for Super-Gaussian Sources

This paper presents new algorithms of independent component analysis (ICA) for super-Gaussian sources based on auxiliary function technique. The algorithms consist of two alternative updates: 1) update of demixing matrix and 2) update of weighted covariance matrix, which include no tuning parameters such as step size. The monotonic decrease of the objective function at each update is guaranteed. The experimental results show that the derived algorithms are robust to nonstationary data and outliers, and the convergence is faster than natural-gradient-based algorithm.

Nobutaka Ono, Shigeki Miyabe


ICA Separability of Nonlinear Models with References: General Properties and Application to Heisenberg-Coupled Quantum States (Qubits)

Relatively few results were reported about the separability of given classes of nonlinear mixtures by means of the ICA criterion. We here prove the separability of a wide class of nonlinear global (i.e. mixing + separating) models involving ”reference signals”, i.e. unmixed signals. This work therefore concerns a nonlinear extension of linear adaptive noise cancellation (ANC). We then illustrate the usefulness of our general results by applying them to a model of Heisenberg-coupled quantum states. This paper opens the way to practical ICA methods for nonlinear mixtures encountered in various applications.

Yannick Deville

Adaptive Underdetermined ICA for Handling an Unknown Number of Sources

Independent Component Analysis is the best known method for solving blind source separation problems. In general, the number of sources must be known in advance. In many cases, previous assumption is not justified. To overcome difficulties caused by an unknown number of sources, an adaptive algorithm based on a simple geometric approach for Independent Component Analysis is presented. By adding a learning rule for the number of sources, the complete method is a two-step algorithm, adapting alternately the number of sources and the mixing matrix. The independent components are estimated in a separate source inference step as required for underdetermined mixtures.

Andreas Sandmair, Alam Zaib, Fernando Puente León

Independent Phase Analysis: Separating Phase-Locked Subspaces

We present a two-stage algorithm to perform blind source separation of sources organized in subspaces, where sources in different subspaces have zero phase synchrony and sources in the same subspace have full phase synchrony. Typical separation techniques such as ICA are not adequate for such signals, because phase-locked signals are not independent. We demonstrate the usefulness of this algorithm on a simulated dataset. The results show that the algorithm works very well in low-noise situations. We also discuss the necessary improvements to be made before the algorithm is able to deal with real-world signals.

Miguel Almeida, José Bioucas-Dias, Ricardo Vigário

Second and Higher-Order Correlation Analysis of Multiple Multidimensional Variables by Joint Diagonalization

In this paper, we introduce two efficient methods for second and higher-order correlation, or linear and nonlinear dependence, analysis of several multidimensional variables. We show that both the second and higher-order correlation analysis can be cast into a specific joint diagonalization problem. Compared with existing multiset canonical correlation analysis (MCCA) and independent vector analysis (IVA) algorithms, desired features of the new methods are that they can exploit the nonwhiteness of observations, they do not assume a specific density model, and they use simultaneous separation and thus are free of error accumulation arising in deflationary separation. Simulation results are presented to show the performance gain of the new methods over MCCA and IVA approaches.

Xi-Lin Li, Matthew Anderson, Tülay Adalı

Independent Component Analysis of Time/Position Varying Mixtures

Blind Source Separation (BSS) is a well known problem that has been addressed in numerous studies in the last few decades. Most of the studies in this field address the problem of time/position invariant mixtures of multiple sources. Real problems are however usually not time and/or position invariant, and much more complicated. We present an extension of the Maximum Likelihood (ML) Independent Component Analysis (ICA) approach to time variant instantaneous mixtures.

Michael Shamis, Yehoshua Y. Zeevi

Random Pruning of Blockwise Stationary Mixtures for Online BSS

We explore information redundancy of linearly mixed sources in order to accomplish the demixing task (BSS) by ICA techniques in real-time. Assuming piecewise stationarity of the sources, the idea is to prune uniformly and independently most of sample data while preserving the ability of Kurtosis-based algorithms to reconstruct the original sources using pruned mixtures instead of original ones. The mainstay of this method is to control the sub-mixtures size so that the Kurtosis is sharply concentrated about that of the entire mixtures with exponentially small error probabilities. Referring to the FastICA algorithm, it is shown that the dimensionality reduction proposed while assuring high quality of the source estimate yields to a significant reduction of the demixing time. In particular, it is experimentally shown that, in case of online applications, the pruning of blockwise stationary data is not only essential for guarantying the time-constraints keeping, but it is also effective.

Alessandro Adamo, Giuliano Grossi

Use of Prior Knowledge in a Non-Gaussian Method for Learning Linear Structural Equation Models

We discuss causal structure learning based on linear structural equation models. Conventional learning methods most often assume Gaussianity and create many indistinguishable models. Therefore, in many cases it is difficult to obtain much information on the structure. Recently, a non-Gaussian learning method called LiNGAM has been proposed to identify the model structure without using prior knowledge on the structure. However, more efficient learning can be achieved if some prior knowledge on a part of the structure is available. In this paper, we propose to use prior knowledge to improve the performance of a state-of-art non-Gaussian method. Experiments on artificial data show that the accuracy and computational time are significantly improved even if the amount of prior knowledge is not so large.

Takanori Inazumi, Shohei Shimizu, Takashi Washio

A New Performance Index for ICA: Properties, Computation and Asymptotic Analysis

In the independent component (IC) model it is assumed that the components of the observed


-variate random vector


are linear combinations of the components of a latent




such that the


components of


are independent. Then


 = Ω


where Ω is a full-rank




mixing matrix. In the independent component analysis (ICA) the aim is to estimate an unmixing matrix Γ such that Γ


has independent components. The comparison of the performances of different unmixing matrix estimates


in the simulations is then difficult as the estimates are for different population quantities Γ. In this paper we suggest a new natural performance index which finds the shortest distance (using Frobenius norm) between the identity matrix and the set of matrices equivalent to the gain matrix

$\hat{\Gamma} \Omega$

. The index is shown to possess several nice properties, and it is easy and fast to compute. Also, the limiting behavior of the index as the sample size approaches infinity can be easily derived if the limiting behavior of the estimate


is known.

Pauliina Ilmonen, Klaus Nordhausen, Hannu Oja, Esa Ollila

Blind Operation of a Recurrent Neural Network for Linear-Quadratic Source Separation: Fixed Points, Stabilization and Adaptation Scheme

Retrieving unknown sources from


mixtures of them requires one to define a separating structure, before proceeding to methods for estimating mixing or separating parameters in blind configurations. Recurrent neural networks are attractive separating structures for a wide range of nonlinear mixing models. In a previous paper, we proposed such a network for the


version of linear-quadratic separation. We here extend this approach to the more difficult


case. We optimize the fixed points and stability of this structure thanks to its free weights. We define the general architecture of future adaptation algorithms that will be able to take advantage of these free weights. Numerical results illustrate the theoretical properties derived in this paper.

Yannick Deville, Shahram Hosseini

Statistical Model of Speech Signals Based on Composite Autoregressive System with Application to Blind Source Separation

This paper presents a new statistical model for speech signals, which consists of a time-invariant dictionary incorporating a set of the power spectral densities of excitation signals and a set of all-pole filters where the gain of each pair of excitation and filter elements is allowed to vary over time. We use this model to develop a combined blind separation and dereverberation method for speech. Reasonably good separations were obtained under a highly reverberant condition.

Hirokazu Kameoka, Takuya Yoshioka, Mariko Hamamura, Jonathan Le Roux, Kunio Kashino

Information-Theoretic Model Selection for Independent Components

Independent Component Analysis (ICA) is an essential building block for data analysis in many applications. Selecting the truly meaningful components from the result of an ICA algorithm, or comparing the results of different algorithms, however, are non-trivial problems. We introduce a very general technique for evaluating ICA results rooted in information-theoretic model selection. The basic idea is to exploit the natural link between non-Gaussianity and data compression: The better the data transformation represented by one or several ICs improves the effectiveness of data compression, the higher is the relevance of the ICs. In an extensive experimental evaluation we demonstrate that our novel information-theoretic measure robustly selects the most interesting components from data without requiring any assumptions or thresholds.

Claudia Plant, Fabian J. Theis, Anke Meyer-Baese, Christian Böhm

Blind Source Separation of Overdetermined Linear-Quadratic Mixtures

This work deals with the problem of source separation in overdetermined linear-quadratic (LQ) models. Although the mixing model in this situation can be inverted by linear structures, we show that some simple independent component analysis (ICA) strategies that are often employed in the linear case cannot be used with the studied model. Motivated by this fact, we consider the more complex yet more robust ICA framework based on the minimization of the mutual information. Special attention is given to the development of a solution that be as robust as possible to suboptimal convergences. This is achieved by defining a method composed of a global optimization step followed by a local search procedure. Simulations confirm the effectiveness of the proposal.

Leonardo T. Duarte, Ricardo Suyama, Romis Attux, Yannick Deville, João M. T. Romano, Christian Jutten

Constrained Complex-Valued ICA without Permutation Ambiguity Based on Negentropy Maximization

Complex independent component analysis (ICA) has found utility in separation of complex-valued signals such as communications, functional magnetic resonance imaging, and frequency-domain speeches. However, permutation ambiguity is a main problem of complex ICA for order-sensitive applications, e.g., frequency-domain speech separation. This paper proposes a semi-blind complex ICA algorithm based on negentropy maximization. The magnitude correlation of a source signal is utilized to constrain the separation process. As a result, the complex-valued signals are separated without permutation. Experiments with synthetic complex-valued signals, synthetic speech signals, and recorded speech signals are performed. The results demonstrate that the proposed algorithm can not only solve the permutation problem, but also achieve slightly improved separation compared to the standard blind algorithm.

Qiu-Hua Lin, Li-Dan Wang, Jian-Gang Lin, Xiao-Feng Gong

Time Series Causality Inference Using Echo State Networks

One potential strength of recurrent neural networks (RNNs) is their – theoretical – ability to find a connection between cause and consequence in time series in an constraint-free manner, that is without the use of explicit probability theory. In this work we present a solution which uses the echo state approach for this purpose. Our approach learns probabilities explicitly using an online learning procedure and echo state networks. We also demonstrate the approach using a test model.

N. Michael Mayer, Oliver Obst, Chang Yu-Chen

Complex Blind Source Separation via Simultaneous Strong Uncorrelating Transform

In this paper, we address the problem of complex blind source separation (BSS), in particular, separation of nonstationary complex signals. It is known that, under certain conditions, complex BSS can be solved effectively by the so-called Strong Uncorrelating Transform (SUT), which simultaneously diagonalizes one Hermitian positive definite and one complex symmetric matrix. Our current work generalizes SUT to simultaneously diagonalize more than two matrices. A Conjugate Gradient (CG) algorithm for computing simultaneous SUT is developed on an appropriate manifold setting of the problem, namely complex oblique projective manifold. Performance of our method, in terms of separation quality, is investigated by several numerical experiments.

Hao Shen, Martin Kleinsteuber

A General Approach for Robustification of ICA Algorithms

This paper presents a general and robust approach to mitigating impact of outliers in independent component analysis applications. The approach detects and removes outlier samples from the dataset and has minimal impact on the overall performance when the dataset is free of outliers. It also has minimal computational burdens, is simply parameterized, and readily implemented. Significant gains in performance is shown for algorithms when outliers are present.

Matthew Anderson, Tülay Adalı

Strong Sub- and Super-Gaussianity

We introduce the terms

strong sub- and super-Gaussianity

to refer to the previously introduced class of densities log-concave is



and log-convex in



respectively. We derive relationships among the various definitions of sub- and super-Gaussianity, and show that strong sub- and super-Gaussianity are related to the score function being star-shaped upward or downward with respect to the origin. We illustrate the definitions and results by extending a theorem of Benveniste, Goursat, and Ruget on uniqueness of separating local optima in ICA.

Jason A. Palmer, Ken Kreutz-Delgado, Scott Makeig


Hybrid Channel Estimation Strategy for MIMO Systems with Decision Feedback Equalizer

We propose combining supervised and unsupervised algorithms in order to improve the performance of multiple-input multiple-output digital communication systems which make use of decision-feedback equalizers at the receiver. The basic idea is to avoid the periodical transmission of pilot symbols by using a simple criterion to determine the time instants when the performance obtained with an unsupervised algorithm is poor or, equivalently, those instants when pilot symbols must be transmitted. Simulation results show how the novel approach provides an adequate BER with a low overhead produced by the transmission of pilot symbols.

Héctor José Pérez-Iglesias, Adriana Dapena, Paula M. Castro, José A. García-Naya

An Alternating Minimization Method for Sparse Channel Estimation

The problem of estimating a sparse channel, i.e. a channel with a few non-zero taps, appears in many fields of communication including acoustic underwater or wireless transmissions. In this paper, we have developed an algorithm based on Iterative Alternating Minimization technique which iteratively detects the location and the value of the channel taps. In fact, at each iteration we use an approximate Maximum A posteriori Probability (MAP) scheme for detection of the taps, while a least square method is used for estimating the values of the taps at each iteration. For approximate MAP detection, we have proposed three different methods leading to three variants for our algorithm. Finally, we experimentally compared the new algorithms to the Cramér-Rao lower bound of the estimation based on knowing the locations of the taps. We experimentally show that by selecting appropriate preliminaries for our algorithm, one of its variants almost reaches the Cramér-Rao bound for high SNR, while the others always achieve good performance.

Rad Niazadeh, Massoud Babaie-Zadeh, Christian Jutten

A Method for Filter Equalization in Convolutive Blind Source Separation

The separation of convolutive mixed signals can be carried out in the time-frequency domain, where the task is reduced to multiple instantaneous problems. This direct approach leads to the permutation and scaling problems, but it is possible to introduce an objective function in the time-frequency domain and minimize it with respect to the time domain coefficients. While this approach allows for the elimination of the permutation problem, the unmixing filters can be quite distorted due the unsolved scaling problem. In this paper we propose a method for equalization of these filters by using the scaling ambiguity. The resulting filters have a characteristic of a Dirac pulse and introduce less distortion to the separated signals. The results are shown on a real-world example.

Radoslaw Mazur, Alfred Mertins

Cancellation of Nonlinear Inter-Carrier Interference in OFDM Systems with Nonlinear Power-Amplifiers

Due to a high peak-to-average power ratio (PAPR), orthogonal frequency division multiplexing (OFDM) signals are often driven at the nonlinear region of power amplifiers (PAs). As a consequence, the orthogonality between the subcarriers is broken and nonlinear inter-carrier interference (ICI) is introduced. In this paper, we proposed two techniques for canceling nonlinear ICI in wireless OFDM communication systems with nonlinear PAs. The proposed techniques are based on the concept of power diversity, which consists in a transmission scheme that re-transmits the symbols several times with a different transmission power each time. The main advantage of using the power diversity is that the problem of canceling the nonlinear ICI can be viewed as a source separation problem, where the ICI terms correspond to “virtual” sources. The proposed techniques are able to provide a more robust transmission at the cost of a lower transmission rate.

C. Alexandre R. Fernandes, João Cesar M. Mota, Gérard Favier

Tensor Factorizations

Probabilistic Latent Tensor Factorization

We develop a probabilistic framework for multiway analysis of high dimensional datasets. By exploiting a link between graphical models and tensor factorization models we can realize any arbitrary tensor factorization structure, and many popular models such as CP or TUCKER models with Euclidean error and their non-negative variants with KL error appear as special cases. Due to the duality between exponential families and Bregman divergences, we can cast the problem as inference in a model with Gaussian or Poisson components, where tensor factorisation reduces to a parameter estimation problem. We derive the generic form of update equations for multiplicative and alternating least squares. We also propose a straightforward matricisation procedure to convert element-wise equations into the matrix forms to ease implementation and parallelisation.

Y. Kenan Yılmaz, A. Taylan Cemgil

Nonorthogonal Independent Vector Analysis Using Multivariate Gaussian Model

We consider the problem of joint blind source separation of multiple datasets and introduce an effective solution to the problem. We pose the problem in an independent vector analysis (IVA) framework utilizing the multivariate Gaussian source vector distribution. We provide a new general IVA implementation using a decoupled nonorthogonal optimization algorithm and establish the connection between the new approach and another approach using second-order statistics, multiset canonical correlation analysis. Experimental results are given to demonstrate the success of the new algorithm in achieving reliable source separation for both Gaussian and non-Gaussian sources.

Matthew Anderson, Xi-Lin Li, Tülay Adalı

Deterministic Blind Separation of Sources Having Different Symbol Rates Using Tensor-Based Parallel Deflation

In this work, we address the problem of blind separation of non-synchronous statistically independent sources from underdetermined mixtures. A deterministic tensor-based receiver exploiting symbol rate diversity by means of parallel deflation is proposed. By resorting to bank of samplers at each sensor output, a set of third-order tensors is built, each one associated with a different source symbol period. By applying multiple Canonical Decompositions (CanD) on these tensors, we can obtain parallel estimates of the related sources along with an estimate of the mixture matrix. Numerical results illustrate the bit-error-rate performance of the proposed approach for some system configurations.

André L. F. de Almeida, Pierre Comon, Xavier Luciani

Second Order Subspace Analysis and Simple Decompositions

The recovery of the mixture of an


-dimensional signal generated by


independent processes is a well studied problem (see e.g. [1,10]) and robust algorithms that solve this problem by Joint Diagonalization exist. While there is a lot of empirical evidence suggesting that these algorithms are also capable of solving the case where the source signals have block structure (apart from a final permutation recovery step), this claim could not be shown yet - even more, it previously was not known if this model separable at all. We present a precise definition of the subspace model, introducing the notion of simple components, show that the decomposition into simple components is unique and present an algorithm handling the decomposition task.

Harold W. Gutch, Takanori Maehara, Fabian J. Theis

Sensitivity of Joint Approximate Diagonalization in FD BSS

This paper investigates the sensitivity of the joint approximate diagonalization of a set of time varying cross-spectral matrices for blind separation of convolutive mixtures of speech signals. We introduce the multitaper method of cross-spectrum estimation. Based on the work of [1] factors affecting the sensitivity of the joint approximate diagonalization problem were investigated. We studied the effect of the number of matrices in the set, and observed that there exists a link between the uniqueness of the joint diagonalizer measured by modulus of uniqueness parameter and the estimation of demixing system parameters.

Savaskan Bulek, Nurgun Erdol

Sparsity I

Blind Compressed Sensing: Theory

Compressed sensing successfully recovers a signal, which is sparse under some basis representation, from a small number of linear measurements. However, prior knowledge of the sparsity basis is essential for the recovery process. In this work we define the blind compressed sensing problem, which aims to avoid the need for this prior knowledge, and discuss the uniqueness of its solution. We prove that this problem is ill possed in general unless further constraints are imposed. We then suggest three possible constraints on the sparsity basis that can be added to the problem in order to render its solution unique. This allows a general sampling and reconstruction system that does not require prior knowledge of the sparsity basis.

Sivan Gleichman, Yonina C. Eldar

Blind Extraction of the Sparsest Component

In this work, we present a discussion concerning some fundamental aspects of sparse component analysis (SCA), a methodology that has been increasingly employed to solve some challenging signal processing problems. In particular, we present some insights into the use of ℓ


norm as a quantifier of sparseness and its application as a cost function to solve the blind source separation (BSS) problem. We also provide results on experiments in which source extraction was successfully made when we performed a search for sparse components in the mixtures of sparse signals. Finally, we make an analysis of the behavior of this approach on scenarios in which the source signals are not sparse.

Everton Z. Nadalin, André K. Takahata, Leonardo T. Duarte, Ricardo Suyama, Romis Attux

Blind Extraction of Intermittent Sources

In this work, we tackle the problem of blind extraction of intermittent sources. Our approach is based on the generalized eigenvector decomposition of covariance matrices and extends previous works in two aspects: by developing a more precise technique to detect inactive periods and by building a more general yet more precise strategy to estimate the vectors that lead to the separation of the intermittent sources. Simulations are carried out to illustrate the effectiveness of our proposal.

Bertrand Rivet, Leonardo T. Duarte, Christian Jutten

Dictionary Learning for Sparse Representations: A Pareto Curve Root Finding Approach

A new dictionary learning method for exact sparse representation is presented in this paper. As the dictionary learning methods often iteratively update the sparse coefficients and dictionary, when the approximation error is small or zero, algorithm convergence will be slow or non-existent. The proposed framework can be used in such a setting by gradually increasing the fidelity of the approximation. This technique has previously been used for the convex sparse representations. It has been extended here to the non-convex dictionary learning problem by allowing the dictionary be modified.

Mehrdad Yaghoobi, Mike E. Davies

SMALLbox - An Evaluation Framework for Sparse Representations and Dictionary Learning Algorithms

SMALLbox is a new foundational framework for processing signals, using adaptive sparse structured representations. The main aim of SMALLbox is to become a test ground for exploration of new provably good methods to obtain inherently data-driven sparse models, able to cope with large-scale and complicated data. The toolbox provides an easy way to evaluate these methods against state-of-the art alternatives in a variety of standard signal processing problems. This is achieved trough a unifying interface that enables a seamless connection between the three types of modules: problems, dictionary learning algorithms and sparse solvers. In addition, it provides interoperability between existing state-of-the-art toolboxes. As an open source MATLAB toolbox, it can be also seen as a tool for reproducible research in the sparse representations research community.

Ivan Damnjanovic, Matthew E. P. Davies, Mark D. Plumbley

Sparsity; Biomedical Applications

Fast Block-Sparse Decomposition Based on SL0

In this paper we present a new algorithm based on Smoothed ℓ


(SL0), called Block SL0 (BSL0), for Under-determined Systems of Linear Equations (USLE) in which the nonzero elements of the unknown vector occur in clusters. Contrary to the previous algorithms such as Block Orthogonal Matching Pursuit (BOMP) and mixed ℓ




norm, our approach provides a fast algorithm, while providing the same (or better) accuracy. Moreover, we will see experimentally that BSL0 has better performance than SL0, BOMP and mixed ℓ




norm when the number of nonzero elements of the source vector approaches the upper bound of uniqueness theorem.

Sina Hamidi Ghalehjegh, Massoud Babaie-Zadeh, Christian Jutten

Second-Order Source Separation Based on Prior Knowledge Realized in a Graph Model

Matrix factorization techniques provide efficient tools for the detailed analysis of large-scale biological and biomedical data. While underlying algorithms usually work fully blindly, we propose to incorporate prior knowledge encoded in a graph model. This graph introduces a partial ordering in data without intrinsic (e.g. temporal or spatial) structure, which allows the definition of a graph-autocorrelation function. Using this framework as constraint to the matrix factorization task we develop a second-order source separation algorithm called graph-decorrelation algorithm (GraDe). We demonstrate its applicability and robustness by analyzing microarray data from a stem cell differentiation experiment.

Florian Blöchl, Andreas Kowarsch, Fabian J. Theis

Noise Adjusted PCA for Finding the Subspace of Evoked Dependent Signals from MEG Data

Evoked signals that underlie multi-channel magnetoencephalography (MEG) data can be dependent. It follows that ICA can fail to separate the evoked dependent signals. As a first step towards separation, we adress the problem of finding a subspace of possibly mixed evoked signals that are separated from the non-evoked signals. Specifically, a vector basis of the evoked subspace and the associated mixed signals are of interest.

It was conjectured that ICA followed by clustering is suitable for this subspace analysis. As an alternative, we propose the use of noise adjusted PCA (NAPCA). This method uses two covariance matrices obtained from pre- and post-stimulation data in order to find a subspace basis. Subsequently, the associated signals are obtained by linear projection onto the estimated basis. Synthetic and recorded data are analyzed and the performance of NAPCA and the ICA approach is compared.

Our results suggest that ICA followed by clustering is a valid approach. Nevertheless, NAPCA outperforms the ICA approach for synthetic and for real MEG data from a study with simultaneous visual and auditory stimulation. Hence, NAPCA should be considered as a viable alternative for the analysis of evoked MEG data.

Florian Kohl, Gerd Wübbeler, Dorothea Kolossa, Clemens Elster, Markus Bär, Reinhold Orglmeister

Binary Sparse Coding

We study a sparse coding learning algorithm that allows for a simultaneous learning of the data sparseness and the basis functions. The algorithm is derived based on a generative model with binary latent variables instead of continuous-valued latents as used in classical sparse coding. We apply a novel approach to perform maximum likelihood parameter estimation that allows for an efficient estimation of all model parameters. The approach is a new form of variational EM that uses truncated sums instead of factored approximations to the intractable posterior distributions. In contrast to almost all previous versions of sparse coding, the resulting learning algorithm allows for an estimation of the optimal degree of sparseness along with an estimation of the optimal basis functions. We can thus monitor the time-course of the data sparseness during the learning of basis functions. In numerical experiments on artificial data we show that the algorithm reliably extracts the true underlying basis functions along with noise level and data sparseness. In applications to natural images we obtain Gabor-like basis functions along with a sparseness estimate. If large numbers of latent variables are used, the obtained basis functions take on properties of simple cell receptive fields that classical sparse coding or ICA-approaches do not reproduce.

Marc Henniges, Gervasio Puertas, Jörg Bornschein, Julian Eggert, Jörg Lücke

A Multichannel Spatial Compressed Sensing Approach for Direction of Arrival Estimation

In this work, we present a direction-of-arrival (DOA) estimation method for narrowband sources impinging from the far-field on a uniform linear array (ULA) of sensors, based on the multichannel compressed sensing (CS) framework. We discretize the angular space uniformly into a grid of possible locations, which is much larger than the number of sensors, and assume that only a few of them will correspond to the active sources. As long as the DOAs of the sources are located at a few locations on the angular grid, they will share a common spatial support. To exploit this joint sparsity, we take several time snapshots and formulate a multichannel spatial compressed sensing (SM-CS) problem. Simultaneous Orthogonal Matching Pursuit (SOMP) is used for the reconstruction and the estimation of the angular power spectrum. The performance of the proposed method is compared against standard spectral-based approaches and other sparsity based methods.

Aris Gretsistas, Mark D. Plumbley

Robust Second-Order Source Separation Identifies Experimental Responses in Biomedical Imaging

Multidimensional biomedical imaging requires robust statistical analyses. Corresponding experiments such as EEG or FRAP commonly result in multiple time series. These data are classically characterized by recording response patterns to any kind of stimulation mixed with any degree of noise levels. Here, we want to detect the underlying signal sources such as these experimental responses in an unbiased fashion, and therefore extend and employ a source separation technique based on temporal autodecorrelation. Our extension first centers the data using a multivariate median, and then separates the sources based on approximate joint diagonalization of multiple sign autocovariance matrices.

Fabian J. Theis, Nikola S. Müller, Claudia Plant, Christian Böhm

Decomposition of EEG Signals for Multichannel Neural Activity Analysis in Animal Experiments

We describe in this paper some advanced protocols for the discrimination and classification of neuronal spike waveforms within multichannel electrophysiological recordings. Sparse decomposition was used to serarate the linearly independent signals underlying sensory information in cortical spike firing pat- terns. We introduce some modifications in the the IDE algorithm to take into account prior knowledge on the spike waveforms. We have investigated motor cortex responses recorded during movement in freely moving rats to provide ev- idence for the relationship between these patterns and special behavioral task.

Vincent Vigneron, Hsin Chen, Yen-Tai Chen, Hsin-Yi Lai, You-Yin Chen

Non-negativity; Image Processing Applications

Using Non-Negative Matrix Factorization for Removing Show-Through

Scanning process usually degrades digital documents due to the contents of the backside of the scanned manuscript. This is often because of the show-through effect,


the backside image that interferes with the main front side picture mainly due to the intrinsic transparency of the paper used for printing or writing.

In this paper, we first use one of Non-negative Matrix Factorization (NMF) methods for canceling show-through phenomenon. Then, non-linearity of show-through effect is included by changing the cost function used in this method. Simulation results show that this proposed algorithm can remove show-through effectively.

Farnood Merrikh-Bayat, Massoud Babaie-Zadeh, Christian Jutten

Nonlinear Band Expansion and 3D Nonnegative Tensor Factorization for Blind Decomposition of Magnetic Resonance Image of the Brain


- and


-divergence based nonnegative tensor factorization (NTF) is combined with nonlinear band expansion (NBE) for blind decomposition of the magnetic resonance image (MRI) of the brain. Concentrations and 3D tensor of spatial distributions of brain substances are identified from the Tucker3 model of the 3D MRI tensor. NBE enables to account for the presence of more brain substances than number of bands and, more important, to improve conditioning of the expanded matrix of concentrations of brain substances. Unlike matrix factorization methods NTF preserves local spatial structure in the MRI. Unlike ICA-, NTF-based factorization is insensitive to statistical dependence among spatial distributions of brain substances. Efficiency of the NBE-NTF algorithm is demonstrated over NBE-ICA and NTF-only algorithms on blind decomposition of the realistically simulated MRI of the brain.

Ivica Kopriva, Andrzej Cichocki

Informed Source Separation Using Latent Components

We address the issue of source separation in a particular


configuration where both the sources and the mixtures are assumed to be known during a so-called


stage. This knowledge enables the computation of a side information which ought to be small enough to be watermarked in the mixtures. At the


stage, the sources are no longer assumed to be known, only the mixtures and the side information are processed to perform source separation.

The proposed method models the sources jointly using latent variables in a framework close to multichannel nonnegative matrix factorization and models the mixing process as linear filtering. Separation at the decoding stage is done using generalized Wiener filtering of the mixtures. An experimental setup shows that the method gives very satisfying results with mixtures composed of many sources. A study of its performance with respect to the number of latent variables is presented.

Antoine Liutkus, Roland Badeau, Gaël Richard

Non-stationary t-Distribution Prior for Image Source Separation from Blurred Observations

We propose a non-stationary spatial image model for the solution of the image separation problem from blurred observations. Our model is defined on first order image differentials. We model the image differentials using


-distribution with space varying scale parameters. This prior image model has been used in the Bayesian formulation and the image source are estimated using a Langevin sampling method. We have tested the proposed model on astrophysical image mixtures and obtained better results regarding stationary model for the maps which have high intensity changes.

Koray Kayabol, Ercan E. Kuruoglu

Automatic Rank Determination in Projective Nonnegative Matrix Factorization

Projective Nonnegative Matrix Factorization (PNMF) has demonstrated advantages in both sparse feature extraction and clustering. However, PNMF requires users to specify the column rank of the approximative projection matrix, the value of which is unknown beforehand. In this paper, we propose a method called ARDPNMF to automatically determine the column rank in PNMF. Our method is based on automatic relevance determination (ARD) with Jeffrey’s prior. After deriving the multiplicative update rule using the expectation-maximization technique for ARDPNMF, we test it on various synthetic and real-world datasets for feature extraction and clustering applications to show the effectiveness of our algorithm. For FERET faces and the Swimmer dataset, interpretable number of features are obtained correctly via our algorithm. Several UCI datasets for clustering are also tested, in which we find that ARDPNMF can estimate the number of clusters quite accurately with low deviation and good cluster purity.

Zhirong Yang, Zhanxing Zhu, Erkki Oja

Non-negative Independent Component Analysis Algorithm Based on 2D Givens Rotations and a Newton Optimization

In this paper, we consider the Independent Component Analysis problem when the hidden sources are non-negative (Non-negative ICA). This problem is formulated as a non-linear cost function optimization over the special orthogonal matrix group




). Using Givens rotations and Newton optimization, we developed an effective axis pair rotation method for Non-negative ICA. The performance of the proposed method is compared to those designed by Plumbley and simulations on synthetic data show the efficiency of the proposed algorithm.

Wendyam Serge Boris Ouedraogo, Antoine Souloumiac, Christian Jutten

A New Geometrical BSS Approach for Non Negative Sources

A new blind source separation method for non-negative sources based on geometrical evidences of the linear mixing model is presented. We show that the proposed method is able to find the mixing matrix as well as the original sources from an observation matrix under the assumption that for every source there is at least one instance where the underlined source is active and all the others are not. One major advantage of our proposal is that the number of sources is found automatically as being the number of extreme data in a set of points. Under the assumption mentioned above, our approach outperforms two well known implementations for NNMF BSS (ALS and multiplicative update algorithms).

Cosmin Lazar, Danielle Nuzillard, Ann Nowé

Dependent Component Analysis for Cosmology: A Case Study

In this paper, we discuss various dependent component analysis approaches available in the literature and study their performances on the problem of separation of dependent cosmological sources from multichannel microwave radiation maps of the sky. Realisticaly simulated cosmological radiation maps are utilised in the simulations which demonstrate the superior performance obtained by tree-dependent component analysis and correlated component analysis methods when compared to classical ICA.

Ercan E. Kuruoglu

Tensors; Joint Diagonalization

A Time-Frequency Technique for Blind Separation and Localization of Pure Delayed Sources

In this paper we address the problem of overdetermined blind separation and localization of several sources, given that an unknown scaled and delayed version of each source contributes to each sensor recording. The separation is performed in the time-frequency domain via an Alternating Least Squares (ALS) algorithm coupled with a Vandermonde structure enforcing strategy across the frequency mode. The latter allows to update the delays and scaling factors of each source with respect to all sensors, up to the ambiguities inherent to the mixing model. After convergence, a reference sensor can be chosen to remove these ambiguities and the Time Difference of Arrival (TDOA) estimates can be exploited to localize the sources individually.

Dimitri Nion, Bart Vandewoestyne, Siegfried Vanaverbeke, Koen Van Den Abeele, Herbert De Gersem, Lieven De Lathauwer

Joint Eigenvalue Decomposition Using Polar Matrix Factorization

In this paper we propose a new algorithm for the joint eigenvalue decomposition of a set of real non-defective matrices. Our approach resorts to a Jacobi-like procedure based on polar matrix decomposition. We introduce a new criterion in this context for the optimization of the hyperbolic matrices, giving birth to an original algorithm called JDTM. This algorithm is described in detail and a comparison study with reference algorithms is performed. Comparison results show that our approach provides quicker and more accurate results in all the considered situations.

Xavier Luciani, Laurent Albera

Joint SVD and Its Application to Factorization Method

This paper introduces an application of joint SVD to the factorization method which is a standard method in computer vision for the estimation of the camera motion and the object shape from an image stream taken by a camera that moves around the object. For computer vision systems with several cameras installed for the same direction, we implement a new algorithm for estimation of camera motion matrix that utilizes the measurement matrices from all the cameras based on joint SVD.

Gen Hori

Sparsity II

Double Sparsity: Towards Blind Estimation of Multiple Channels

We propose a framework for blind multiple filter estimation from convolutive mixtures, exploiting the time-domain sparsity of the mixing filters and the disjointness of the sources in the time-frequency domain. The proposed framework includes two steps: (a) a clustering step, to determine the frequencies where each source is active alone; (b) a filter estimation step, to recover the filter associated to each source from the corresponding incomplete frequency information. We show how to solve the filter estimation step (b) using convex programming, and we explore numerically the factors that drive its performance. Step (a) remains challenging, and we discuss possible strategies that will be studied in future work.

Prasad Sudhakar, Simon Arberet, Rémi Gribonval

Adaptive and Non-adaptive ISI Sparse Channel Estimation Based on SL0 and Its Application in ML Sequence-by-Sequence Equalization

In this paper, we firstly propose an adaptive method based on the idea of Least Mean Square (LMS) algorithm and the concept of smoothed



(SL0) norm presented in [1] for estimation of sparse Inter Symbol Interface (ISI) channels which will appear in wireless and acoustic underwater transmissions. Afterwards, a new non-adaptive fast channel estimation method based on SL0 sparse signal representation is proposed. ISI channel estimation will have a direct effect on the performance of the ISI equalizer at the receiver. So, in this paper we investigate this effect in the case of optimal Maximum Likelihood Sequence-by-sequence Equalizer (MLSE) [2]. In order to implement this equalizer, we propose a new method called pre-filtered Parallel Viterbi Algorithm (or pre-filtered PVA) for

general ISI sparse channels

which has much less complexity than ordinary Viterbi Algorithm (VA) and also with no considerable loss of optimality, which we have examined by doing some experiments. Indeed, Simulation results clearly show that the proposed concatenated estimation-equalization methods have much better performance than the usual equalization methods such as Linear Mean Square Equalization (LMSE) for ISI sparse channels, while preserving simplicity at the receiver with the use of PVA.

Rad Niazadeh, Sina Hamidi Ghalehjegh, Massoud Babaie-Zadeh, Christian Jutten

Biomedical Applications

Extraction of Foetal Contribution to ECG Recordings Using Cyclostationarity-Based Source Separation Method

In this paper we propose a cyclostationary approach to the problem of the foetal electrocardiogram (FECG) extraction from a set of cutaneous potential recordings of an expectant mother. We adopted a semi-blind source separation (BSS) method for which the only necessary prior knowledge is that of the fundamental cyclic frequency of the cyclostationary process to be estimated. Using this technique, the estimated cyclostationary FECG source of interest is found to be free from any interferences with the mother’s ECG (MECG) signal. Experimental results and perspectives for future research conclude this paper.

Michel Haritopoulos, Cécile Capdessus, Asoke K. Nandi

Common SpatioTemporal Pattern Analysis

In this work we present a method for the estimation of a rank-one pattern living in two heterogeneous spaces, when observed through a mixture in multiple observation sets. Using a well chosen representation for an observed set of second order tensors (matrices), a singular value decomposition of the set structure yields an accurate estimate under some widely acceptable conditions. The method performs a completely algebraic estimation in both heterogeneous spaces without the need for heuristic parameters. Contrary to existing methods, neither independence in one of the spaces, nor joint decorrelation in both of the heterogeneous spaces is required. In addition, because the method is not variance based in the input space, it has the critical advantage of being applicable with low signal-to-noise ratios. This makes this method an excellent candidate ,e.g., for the direct estimation of the spatio-temporal P300 pattern in passive exogenous brain computer interface paradigms. For these applications it is often sufficient to consider quasi-decorrelation in the temporal space only, while we do not want to impose a similar constraint in the spatial domain.

Ronald Phlypo, Nisrine Jrad, Bertrand Rivet, Marco Congedo

Recovering Spikes from Noisy Neuronal Calcium Signals via Structured Sparse Approximation

Two-photon calcium imaging is an emerging experimental technique that enables the study of information processing within neural circuits

in vivo

. While the spatial resolution of this technique permits the calcium activity of individual cells within the field of view to be monitored, inferring the precise times at which a neuron emits a spike is challenging because spikes are hidden within noisy observations of the neuron’s calcium activity. To tackle this problem, we introduce the use of sparse approximation methods for recovering spikes from the time-varying calcium activity of neurons. We derive sufficient conditions for exact recovery of spikes with respect to (


) the decay rate of the spike-evoked calcium event and (


) the maximum firing rate of the cell under test. We find—both in theory and in practice—that standard sparse recovery methods are not sufficient to recover spikes from noisy calcium signals when the firing rate of the cell is high, suggesting that in order to guarantee exact recovery of spike times, additional constraints must be incorporated into the recovery procedure. Hence, we introduce an iterative framework for

structured sparse approximation

that is capable of achieving superior performance over standard sparse recovery methods by taking into account knowledge that spikes are non-negative and also separated in time. We demonstrate the utility of our approach on simulated calcium signals in various amounts of additive Gaussian noise and under different degrees of model mismatch.

Eva L. Dyer, Marco F. Duarte, Don H. Johnson, Richard G. Baraniuk

Semi-nonnegative Independent Component Analysis: The (3,4)-SENICAexp Method

To solve the Independent Component Analysis (ICA) problem under the constraint of nonnegative mixture, we propose an iterative algorithm, called (3,4)-SENICA


. This method profits from some interesting properties enjoyed by third and fourth order statistics in the presence of mixed independent processes, imposing the nonnegativity of the mixture by means of an exponential change of variable. This process allows us to obtain an unconstrained problem, optimized using an ELSALS-like procedure. Our approach is tested on synthetic magnetic resonance spectroscopic imaging data and compared to two existing ICA methods, namely SOBI and CoM2.

Julie Coloigner, Laurent Albera, Ahmad Karfoul, Amar Kachenoura, Pierre Comon, Lotfi Senhadji

Classifying Healthy Children and Children with Attention Deficit through Features Derived from Sparse and Nonnegative Tensor Factorization Using Event-Related Potential

In this study, we use features extracted by Nonnegative Tensor Factorization (NTF) from event-related potentials (ERPs) to discriminate healthy children and children with attention deficit (AD). The peak amplitude of an ERP has been extensively used to discriminate different groups of subjects for the clinical research. However, such discriminations sometimes fail because the peak amplitude may vary severely with the increased number of subjects and wider range of ages and it can be easily affected by many factors. This study formulates a framework, using NTF to extract features of the evoked brain activities from time-frequency represented ERPs. Through using the estimated features of a negative ERP-mismatch negativity, the correct rate on the recognition between health children and children with AD approaches to about 76%. However, the peak amplitude did not discriminate them. Hence, it is promising to apply NTF for diagnosing clinical children instead of measuring the peak amplitude.

Fengyu Cong, Anh Huy Phan, Heikki Lyytinen, Tapani Ristaniemi, Andrzej Cichocki

Emerging Topics

Riemannian Geometry Applied to BCI Classification

In brain-computer interfaces based on motor imagery, covariance matrices are widely used through spatial filters computation and other signal processing methods. Covariance matrices lie in the space of Symmetric Positives-Definite (SPD) matrices and therefore, fall within the Riemannian geometry domain. Using a differential geometry framework, we propose different algorithms in order to classify covariance matrices in their native space.

Alexandre Barachant, Stéphane Bonnet, Marco Congedo, Christian Jutten

Separating Reflections from a Single Image Using Spatial Smoothness and Structure Information

We adopt two priors to realize reflection separation from a single image, namely spatial smoothness, which is based on pixels’ color dependency, and structure difference, which is got from different source images (transmitted image and reflected image) and different color channels of the same image. By analysing the optical model of reflection, we simplify the mixing matrix further and realize the method for getting spatially varying mixing coefficients. Based on the priors and using Gibbs sampling and appropriate probability density with Bayesian framework, our approach can achieve impressive results for many real world images that corrupted with reflections.

Qing Yan, Ercan E. Kuruoglu, Xiaokang Yang, Yi Xu, Koray Kayabol

ICA over Finite Fields

Independent Component Analysis is usually performed over the fields of reals or complex numbers and the only other field where some insight has been gained so far is GF(2), the finite field with two elements. We extend this to arbitrary finite fields, proving separability of the model if the sources are non-uniform and non-degenerate and present algorithms performing this task.

Harold W. Gutch, Peter Gruber, Fabian J. Theis


Additional information

Premium Partner

    Image Credits