Skip to main content
Top
Published in: The Journal of Supercomputing 2/2014

01-11-2014

Parallel approach to NNMF on multicore architecture

Authors: P. Alonso, V. M. García, F. J. Martínez-Zaldívar, A. Salazar, L. Vergara, A. M. Vidal

Published in: The Journal of Supercomputing | Issue 2/2014

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

We tackle the parallelization of Non-Negative Matrix Factorization (NNMF), using the Alternating Least Squares and Lee and Seung algorithms, motivated by its use in audio source separation. For the first algorithm, a very suitable technique is the use of active set algorithms for solving several non-negative inequality constraints least squares problems. We have addressed the NNMF for dense matrix on multicore architectures, by organizing these optimization problems for independent columns. Although in the sequential case, the method is not as efficient as the block pivoting variant used by other authors, they are very effective in the parallel case, producing satisfactory results for the type of applications where is to be used. For the Lee and Seung method, we propose a reorganization of the algorithm steps that increases the convergence speed and a parallelization of the solution. The article also includes a theoretical and experimental study of the performance obtained with similar matrices to that which arise in applications that have motivated this work.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788–791CrossRef Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788–791CrossRef
2.
go back to reference Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. Advances in neural information processing systems, vol 13. MIT Press, Cambridge, pp 556–562 Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. Advances in neural information processing systems, vol 13. MIT Press, Cambridge, pp 556–562
3.
go back to reference Paatero P, Tapper U (1994) Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5:111–126CrossRef Paatero P, Tapper U (1994) Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5:111–126CrossRef
4.
go back to reference Paatero P (1997) Least squares formulation of robust non-negative factor analysis. Chemometr Intell. Lab. Syst. 37:23–35CrossRef Paatero P (1997) Least squares formulation of robust non-negative factor analysis. Chemometr Intell. Lab. Syst. 37:23–35CrossRef
5.
go back to reference Battenberg E, Freed A, Wessel D (2010) Advances in the parallelization of music and audio applications. International computer music conference. Stony Brook, New York Battenberg E, Freed A, Wessel D (2010) Advances in the parallelization of music and audio applications. International computer music conference. Stony Brook, New York
6.
go back to reference Wnag J, Zhong W, Zhang J (2006) NNMF-based factorization techniques for high-accuracy privacy protection on non-negative-valued datasets, data mining workshops, 2006. ICDM Workshops 2006. Sixth IEEE International Conference on Computing and Processing, pp 513–517 Wnag J, Zhong W, Zhang J (2006) NNMF-based factorization techniques for high-accuracy privacy protection on non-negative-valued datasets, data mining workshops, 2006. ICDM Workshops 2006. Sixth IEEE International Conference on Computing and Processing, pp 513–517
7.
go back to reference Rodriguez-Serrano FJ, Carabias-Orti JJ, Vera-Candeas P, Virtanen T, Ruiz-Reyes N (2012) Multiple instrument mixtures source separation evaluation using instrument-dependent NMF models. In: Theis F (ed) LVA/ICA 2012. Springer-Verlag, Berlin, pp 380–387 Rodriguez-Serrano FJ, Carabias-Orti JJ, Vera-Candeas P, Virtanen T, Ruiz-Reyes N (2012) Multiple instrument mixtures source separation evaluation using instrument-dependent NMF models. In: Theis F (ed) LVA/ICA 2012. Springer-Verlag, Berlin, pp 380–387
8.
go back to reference Xu W, Liu X, Gong Y (2003) Document clustering based on non-negative matrix factorization, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in information retrieval. SIGIR03, Toronto, pp 267–273. Xu W, Liu X, Gong Y (2003) Document clustering based on non-negative matrix factorization, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in information retrieval. SIGIR03, Toronto, pp 267–273.
9.
go back to reference Berry MW, Browne M, Langville A, Pauca V, Plemmons R (2007) Algorithms and applications for aproximate nonnegative matrix factorization. Comput Stat Data Anal 52:155–173MathSciNetCrossRefMATH Berry MW, Browne M, Langville A, Pauca V, Plemmons R (2007) Algorithms and applications for aproximate nonnegative matrix factorization. Comput Stat Data Anal 52:155–173MathSciNetCrossRefMATH
10.
go back to reference Kim J, Park H (2007) Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics 23:1495–1502CrossRef Kim J, Park H (2007) Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics 23:1495–1502CrossRef
11.
go back to reference Devajaran K (2008) Nonnegative matrix factorization: an analytical and interpretative tool in computational biology. PLoS Comput Biol 4(7):e1000029CrossRef Devajaran K (2008) Nonnegative matrix factorization: an analytical and interpretative tool in computational biology. PLoS Comput Biol 4(7):e1000029CrossRef
12.
go back to reference Guan N, Tao D, Luo Z, Yuan B (2012) NeNMF: an optimal gradient method for nonnegative matrix factorization. IEEE Trans Signal Proc 60(6):2882–2898MathSciNetCrossRef Guan N, Tao D, Luo Z, Yuan B (2012) NeNMF: an optimal gradient method for nonnegative matrix factorization. IEEE Trans Signal Proc 60(6):2882–2898MathSciNetCrossRef
13.
go back to reference Cichocki A, Phan A-H (2009) Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Trans Fund Electron Commun Comput Sci E92– A:708–721CrossRef Cichocki A, Phan A-H (2009) Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Trans Fund Electron Commun Comput Sci E92– A:708–721CrossRef
14.
go back to reference Cichocki A, Zdunek R, Amari S-I (2007) Hierarchical ALS algorithms for nonnegative matrix and 3d tensor factorization, independent component analysis and signal separation, lecture notes in computer science, vol 4666. Springer, NewYork, pp 169–176 Cichocki A, Zdunek R, Amari S-I (2007) Hierarchical ALS algorithms for nonnegative matrix and 3d tensor factorization, independent component analysis and signal separation, lecture notes in computer science, vol 4666. Springer, NewYork, pp 169–176
15.
go back to reference Cichocki A, Cruces S, Amari S-I (2011) Generalized alpha–beta divergences and their application to robust nonnegative matrix factorization. Entropy 13:134–170CrossRef Cichocki A, Cruces S, Amari S-I (2011) Generalized alpha–beta divergences and their application to robust nonnegative matrix factorization. Entropy 13:134–170CrossRef
16.
go back to reference Kim J, Park H (2011) Fast nonnegative matrix factorization: an active-set-like method and comparisons. SIAM J Sci Comput 33(6):3261–3281MathSciNetCrossRefMATH Kim J, Park H (2011) Fast nonnegative matrix factorization: an active-set-like method and comparisons. SIAM J Sci Comput 33(6):3261–3281MathSciNetCrossRefMATH
17.
go back to reference Kim D, Sra S, Dhillon IS (2007) Fast newton-type methods for the least squares nonnegative matrix approximation problem, Proceedings of the seventh SIAM international conference on data mining. SIAM, Philadelphia, pp 343–354 Kim D, Sra S, Dhillon IS (2007) Fast newton-type methods for the least squares nonnegative matrix approximation problem, Proceedings of the seventh SIAM international conference on data mining. SIAM, Philadelphia, pp 343–354
18.
go back to reference Kim J, Park H (2008) Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method. SIAM J Matrix Anal Appl 30:713–730MathSciNetCrossRefMATH Kim J, Park H (2008) Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method. SIAM J Matrix Anal Appl 30:713–730MathSciNetCrossRefMATH
19.
go back to reference Carabias-Orti JJ, Virtanen T, Vera-Candeas P, Ruiz-Reyes N, Caadas-Quesada FJ (2011) Musical instrument sound multi-excitation model for non-negative spectrogram factorization. IEEE J Select Top Signal Proc 5(6):1144–1158CrossRef Carabias-Orti JJ, Virtanen T, Vera-Candeas P, Ruiz-Reyes N, Caadas-Quesada FJ (2011) Musical instrument sound multi-excitation model for non-negative spectrogram factorization. IEEE J Select Top Signal Proc 5(6):1144–1158CrossRef
20.
go back to reference Mejia-Roa E, Garcia C, Gomez JI, Prieto M, Tenllado C, Pascual-Montano A, Tirado F (2012) Parallelism on the non-negative matrix factorization, applications, tools and techniques on the road to exascale computing, vol 22. IOS Press, Netherlands Mejia-Roa E, Garcia C, Gomez JI, Prieto M, Tenllado C, Pascual-Montano A, Tirado F (2012) Parallelism on the non-negative matrix factorization, applications, tools and techniques on the road to exascale computing, vol 22. IOS Press, Netherlands
21.
go back to reference Dong C, Zhoa H, Wang W (2010) Parallel nonnegative matrix factorization algorithm on the distributed memory platform. Int J Par Prog 38:117–137CrossRefMATH Dong C, Zhoa H, Wang W (2010) Parallel nonnegative matrix factorization algorithm on the distributed memory platform. Int J Par Prog 38:117–137CrossRefMATH
22.
go back to reference Grippo L, Sciandrone M (2000) On the convergence of the block nonlinear Gauss–Seidel method under convex constraints. Oper Res Lett 26:127–136MathSciNetCrossRefMATH Grippo L, Sciandrone M (2000) On the convergence of the block nonlinear Gauss–Seidel method under convex constraints. Oper Res Lett 26:127–136MathSciNetCrossRefMATH
23.
go back to reference Lawson CL, Hanson RJ (1995) Solving least squares problems. SIAM, Philadelphia Lawson CL, Hanson RJ (1995) Solving least squares problems. SIAM, Philadelphia
24.
go back to reference Bjorck A (1996) Numerical methods for least squares problems. SIAM, Philadelphia Bjorck A (1996) Numerical methods for least squares problems. SIAM, Philadelphia
25.
go back to reference MATLAB version 8.0 (R2012b) (2012) The MathWorks Inc, Natick, Massachusetts. MATLAB version 8.0 (R2012b) (2012) The MathWorks Inc, Natick, Massachusetts.
Metadata
Title
Parallel approach to NNMF on multicore architecture
Authors
P. Alonso
V. M. García
F. J. Martínez-Zaldívar
A. Salazar
L. Vergara
A. M. Vidal
Publication date
01-11-2014
Publisher
Springer US
Published in
The Journal of Supercomputing / Issue 2/2014
Print ISSN: 0920-8542
Electronic ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-013-1083-8

Other articles of this Issue 2/2014

The Journal of Supercomputing 2/2014 Go to the issue

Premium Partner