Top

Advances in Data Analysis and Classification

Published in:

26-11-2014 | Regular Article

NMF versus ICA for blind source separation

Author: Andri Mirzal

Published in: Advances in Data Analysis and Classification | Issue 1/2017

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Blind source separation (BSS) is a problem of recovering source signals from signal mixtures without or very limited information about the sources and the mixing process. From literatures, nonnegative matrix factorization (NMF) and independent component analysis (ICA) seem to be the mainstream techniques for solving the BSS problems. Even though the using of NMF and ICA for BSS is well studied, there is still a lack of works that compare the performances of these techniques. Moreover, the nonuniqueness property of NMF is rarely mentioned even though this property actually can make the reconstructed signals vary significantly, and thus introduces the difficulty on how to choose the representative reconstructions from several possible outcomes. In this paper, we compare the performances of NMF and ICA as BSS methods using some standard NMF and ICA algorithms, and point out the difficulty in choosing the representative reconstructions originated from the nonuniqueness property of NMF.

previous article On visual distances for spectrum-type functional data

next article Dichotomic lattices and local discretization for Galois lattices

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

There is actually a study that considers the effect of the nonuniqueness property of NMF to the reconstruction results as it displays the average signal-to-noise values over some trials (Plaza et al. 2012). However the authors do not mention the difficulty in choosing the representative reconstructions.

The auxiliary function for proving the nonincreasing property in any MUR based NMF algorithm is also the Lyapunov function, so that the stability of any MUR based NMF algorithm can be shown directly by using the result presented in Badeau (2010).

MVCNMF has a justification as a BSS method as it looks for estimate source vectors that span a simplex that circumscribes the observed data.

Anttila P et al (1995) Source identification of bulk wet deposition in Finland by positive matrix factorization. Atmos Environ 29(14):1705–1718CrossRef

Arngren M, Schmidt MN, Larsen J (2011) Unmixing of hyperspectral images using Bayesian non-negative matrix factorization. J Signal Process Syst 65:479–496CrossRef

Badeau R (2010) Stability analysis of multiplicative update algorithms and application to nonnegative matrix factorization. IEEE Trans Neural Netw 21(12):1869–1881CrossRef

Berry M, Brown M, Langville A, Pauca P, Plemmons RJ (2007) Algorithms and applications for approximate nonnegative matrix factorization. Comput Stat Data Anal 52(1):155–173MathSciNetCrossRefMATH

Bertin N et al (2009) A tempering approach for Itakura–Saito non-negative matrix factorization. With application to music transcription. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, pp 1545–1548

Bertin N et al (2010) Enforcing harmonicity and smoothness in Bayesian non-negative matrix factorization applied to polyphonic music transcription. IEEE Trans Audio Speech Lang Process 18(3):538–549CrossRef

Bertrand A, Moonen M (2010) Blind separation of non-negative source signals using multiplicative updates and subspace projection. Signal Process 90(10):2877–2890CrossRefMATH

Brunet JP et al (2003) Metagenes and molecular pattern discovery using matrix factorization. Proc Natl Acad Sci USA 101(12):4164–4169CrossRef

Carmona-Saez P et al (2006) Biclustering of gene expression data by non-smooth non-negative matrix factorization. BMC Bioinform 7(78)

Cauquy MA, Roggemann M, Schultz T (2004) Approaches for processing spectral measurements of reflected sunlight for space situational awareness. In: Proceedings of SPIE conference on defense and security, vol 5428, pp 48–57

Chen Z, Nowrouzian B, Zarowski CJ (2005) An investigation of SNR estimation techniques based on uniform Cramer-Rao lower bound. In: 48th midwest symposium on circuits and systems, pp 215–218

Choi S (2008) Algorithms for orthogonal nonnegative matrix factorization. In: Proceedings of IEEE int’l joint conf. on neural networks, pp 1828–1832

Cichocki A, Amari S, Zdunek R, Kompass R, Hori G, He Z (2006) Extended SMART algorithms for non-negative matrix factorization. Lect Notes Comput Sci 4029:548–562CrossRef

Craig MD (1994) Minimum-volume transforms for remotely sensed data. IEEE Trans Geosci Remote Sens 32(3):542–552CrossRef

Devarajan K (2008) Nonnegative matrix factorization: an analytical and interpretive tool in computational biology. PLoS Comput Biol 4(7):e1000029CrossRef

Ding C, Li T, Peng W, Park H (2006) Orthogonal nonnegative matrix t-factorizations for clustering. In: Proceedings of 12th ACM SIGKDD int’l conf. on knowledge discovery and data mining, pp 126–135

Févotte C et al (2009) Nonnegative matrix factorization with the Itakura–Saito divergence. With application to music analysis. Neural Comput 21(3):793–830CrossRefMATH

Févotte C, Idier J (2011) Algorithms for nonnegative matrix factorization with the \(\beta \)-divergence. Neural Comput 23(9):2421–2456MathSciNetCrossRefMATH

FitzGerald D et al (2009) On the use of the beta divergence for musical source separation. In: Proceedings of the Irish signals and systems conference

Fogel P et al (2007) Inferential, robust non-negative matrix factorization analysis of microarray data. Bioinformatics 23(1):44–49CrossRef

Gao Y, Church G (2005) Improving molecular cancer class discovery through sparse non-negative matrix factorization. Bioinformatics 21(21):3970–3975CrossRef

Gillis N, Glineur F (2012) A multilevel approach for nonnegative matrix factorization. J Comput Appl Math 236(7):1708–1723MathSciNetCrossRefMATH

Grady PD (2007) Sparse separation of under-determined speech mixtures. Ph.D. thesis, National University of Ireland, Maynooth

Grady PD, Pearlmutter BA (2008) Discovering speech phones using convolutive non-negative matrix factorisation with a sparseness constraint. Neurocomputing 72(1–3):88–101

Grippo L, Sciandrone M (2000) On the convergence of the block nonlinear Gauss–Seidel method under convex constraints. Oper Res Lett 26(3):127–136MathSciNetCrossRefMATH

Hennequin R et al (2010) NMF with time-frequency activations to model non stationary audio events. In: Proceedings of IEEE international conference on acoustics speech and signal processing, pp 445–448

Hindi H (2004) A tutorial on convex optimization. In: Proceedings of American control conference, pp 3252–3265

Hoyer PO (2004) Non-negative matrix factorization with sparseness constraints. J Mach Learn Res 5:1457–1469MathSciNetMATH

Hyvrinen A (1999) Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans Neural Netw 10(3):626–634CrossRef

Hyvrinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural Netw 13:411–430CrossRef

Inamura K et al (2005) Two subclasses of lung squamous cell carcinoma with different gene expression profiles and prognosis identified by hierarchical clustering and non-negative matrix factorization. Oncogene 24:7105–7113CrossRef

Jia S, Qian Y (2009) Constrained nonnegative matrix factorization for hyperspectral unmixing. IEEE Trans Geosci Remote Sens 47(1):161–173CrossRefMATH

Keshava N, Mustard J (2002) Spectral unmixing. IEEE Signal Process Mag 8:44–57CrossRef

Kim H, Park H (2007) Sparse non-negative matrix factorizations via alternating non-negativity constrained least squares for microarray data analysis. Bioinformatics 23(12):1495–1502CrossRef

Kim H, Park H (2008) Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method. SIAM J Matrix Anal Appl 30(2):713–730MathSciNetCrossRefMATH

Kim J, Park H (2008) Sparse nonnegative matrix factorization for clustering. CSE Technical Reports; GT-CSE-08-01, Georgia Institute of Technology

Kim J, Park H (2008) Toward faster nonnegative matrix factorization: a new algorithm and comparisons. In: Proceedings of the eighth IEEE international conference on data mining, pp 353–362

Lee D, Seung H (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791CrossRef

Lee D, Seung H (2000) Algorithms for non-negative matrix factorization. In: Proceedings of advances in neural processing information systems, pp 556–562

Li SZ et al (2001) Learning spatially localized, parts-based representation. In: Proceedings of IEEE comp. soc. conf. on computer vision and pattern recognition, pp 207–212

Li H, Adali T, Wang W, Emge D (2005) Non-negative matrix factorization with orthogonality constraints for chemical agent detection in Raman spectra. In: Proceedings of IEEE workshop on machine learning for signal processing, pp 253–258

Lin CJ (2005) Projected gradient methods for non-negative matrix factorization. Technical Report ISSTECH-95-013. Department of CS, National Taiwan University

Luu L et al (2003) Object characterization from spectral data. In: Proceedings of AMOS technical conference

Masalmah YM (2007) Unsupervised unmixing of hyperspectral imagery using the constrained positive matrix factorization. Ph.D. thesis, Computing and Information Science and Engineering, University of Puerto Rico

Miao L, Qi H (2007) Endmember extraction from highly mixed data using minimum volume constrained nonnegative matrix factorization. IEEE Trans Geosci Remote Sens 45(3):765–777CrossRef

Mirzal A (2014) A convergent algorithm for orthogonal nonnegative matrix factorization. J Comput Appl Math 260:149–166MathSciNetCrossRefMATH

Nascimento JMP, Dias JMB (2005) Does independent component analysis play a role in unmixing hyperspectral data? IEEE Trans Geosci Remote Sens 43(1):175–187CrossRef

Oja E, Plumbley MD (2004) Blind separation of positive sources by globally convergent gradient search. Neural Comput 16(9):1811–1825CrossRefMATH

Oja E, Plumbley MD (2003) Blind separation of positive sources using non-negative PCA. In: Proceedings of the 4th international symposium on independent component analysis and blind signal separation, warning for areas of moderate seismicity, pp 11–16

Paatero P, Tapper U (1994) Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5:111–126CrossRef

Pascual-Montano A et al (2006) Nonsmooth nonnegative matrix factorization. IEEE Trans Pattern Anal Mach Intell 28(3):403–415CrossRef

Pauca VP, Piper J, Plemmons RJ (2006) Nonnegative matrix factorization for spectral data analysis. Linear Algebra Appl 416(1):29–47MathSciNetCrossRefMATH

Piper J et al (2004) Object characterization from spectral data using nonnegative matrix factorization. In: Proceedings of AMOS technical conference

Plaza J, Hendrix EMT, Garcia I, Martin G, Plaza A (2012) On endmember identification in hyperspectral images without pure pixels: a comparison of algorithms. J Math Imaging Vis 42:163–175MathSciNetCrossRefMATH

Plumbey MD (2002) Conditions for nonnegative independent component analysis. IEEE Signal Process Lett 9(6):177–180CrossRef

Plumbey MD (2003) Algorithms for nonnegative independent component analysis. IEEE Trans Neural Netw 14(3):534–543CrossRef

Plumbey MD, Oja E (2004) A nonnegative PCA algorithm for independent component analysis. IEEE Trans Neural Netw 15(1):66–76CrossRef

Ren G (2009) SNR estimation algorithm based on the preamble for OFDM systems in frequency selective channels. IEEE Trans Commun 57(8):2230–2234CrossRef

Shahnaz F et al (2006) Document clustering using nonnegative matrix factorization. Inf Process Manag 42(2):373–386CrossRefMATH

Sinha P (2002a) Identifying perceptually significant features for recognizing faces. In: Proceedings of the SPIE electronic imaging symposium

Sinha P (2002b) Recognizing complex patterns. Nat Neurosci 5(suppl.):1093–1097

Sinha P et al (2006) Face recognition by humans: nineteen results all computer vision researchers should know about. Proc IEEE 94(11):1948–1962

Stögbauer H, Kraskov A, Astakhov SA, Grassberger P (2004) Least dependent component analysis based on mutual information. Phys Rev E 70(6):066123CrossRef

Vincent E et al (2010) Adaptive harmonic spectral decomposition for multiple pitch estimation. IEEE Trans Audio Speech Lang Process 18:528–537CrossRef

Virtanen T et al (2008) Bayesian extensions to non-negative matrix factorisation for audio signal modelling. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, pp 1825–1828

Wang G et al (2006) LS-NMF: a modified non-negative matrix factorization algorithm utilizing uncertainty estimates. BMC Bioinform 7(175)

Wang JJY et al (2013) Non-negative matrix factorization by maximizing correntropy for cancer clustering. BMC Bioinform 14(107)

Wang D, Lu H (2013) On-line learning parts-based representation via incremental orthogonal projective non-negative matrix factorization. Signal Process 93(6):1608–1623CrossRef

Xu W et al (2003) Document clustering based on non-negative matrix factorization. In: Proceedings of ACM SIGIR, pp 267–273

Xu X et al (2006) Subspace-based noise variance and SNR estimation for MIMO OFDM systems. J Electron (China) 23(2):176–180CrossRef

Yoo J, Choi S (2010) Orthogonal nonnegative matrix tri-factorization for co-clustering: multiplicative updates on Stiefel manifolds. Inf Process Manag 46(5):559–570CrossRef

Yoo J, Choi S (2008) Orthogonal nonnegative matrix factorization: multiplicative updates on Stiefel manifolds. In: Proceedings of the 9th int’l conf. intelligent data engineering and automated learning, pp 140–147

Yuvaraj N, Vivekanandan P (2013) An efficient SVM based tumor classification with symmetry non-negative matrix factorization using gene expression data. In: Int’l conf. on information communication and embedded systems, pp 761–768

Zarowski CJ (2002) Limitations on SNR estimator accuracy. IEEE Trans Signal Process 50(9):2368–2372CrossRef

Zheng CH et al (2009) Tumor clustering using nonnegative matrix factorization with gene selection. IEEE Trans Inf Technol Biomed 13(4):599–607CrossRef

Zhou G et al (2011) Online blind source separation using incremental nonnegative matrix factorization with volume constraint. IEEE Trans Neural Netw 22(4):550–560CrossRef

Title: NMF versus ICA for blind source separation
Author: Andri Mirzal
Publication date: 26-11-2014
Publisher: Springer Berlin Heidelberg
Published in: Advances in Data Analysis and Classification / Issue 1/2017
Print ISSN: 1862-5347
Electronic ISSN: 1862-5355
DOI: https://doi.org/10.1007/s11634-014-0192-4

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 1/2017

A generalized maximum entropy estimator to simple linear measurement error model with a composite indicator

Dichotomic lattices and local discretization for Galois lattices

Multiple straight-line fitting using a Bayes factor

Editorial for issue 1/2017

Advances in credit scoring: combining performance and interpretation in kernel discriminant analysis

A uniform framework for the combination of penalties in generalized structured models

Premium Partner