nach oben

Erschienen in:

2019 | OriginalPaper | Buchkapitel

14. Nonnegative Matrix Factorization

verfasst von : Ke-Lin Du, M. N. S. Swamy

Erschienen in: Neural Networks and Statistical Learning

Verlag: Springer London

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Low-rank matrix factorization or factor analysis is an important task that is helpful in the analysis of high-dimensional real-world data such as dimension reduction, data compression, feature extraction, and information retrieval. Nonnegative matrix factorization is a special low-rank factorization technique for nonnegative data. This chapter is dedicated to nonnegative matrix factorization. Other matrix decomposition methods, such as Nystrom method and CUR matrix decomposition, are also introduced in this chapter.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Principal Component Analysis

Nächstes Kapitel Independent Component Analysis

Banerjee, A., Merugu, S., Dhillon, I. S., & Ghosh, J. (2005). Clustering with Bregman divergences. Journal of Machine Learning Research, 6, 1705–1749.MathSciNetMATH

Berry, M. W., Browne, M., Langville, A. N., Pauca, V. P., & Plemmons, R. J. (2007). Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics and Data Analysis, 52(1), 155–173.MathSciNetMATHCrossRef

Blumensath, T. (2016). Directional clustering through matrix factorization. IEEE Transactions on Neural Networks and Learning Systems, 27(10), 2095–2107.MathSciNetCrossRef

Cai, D., He, X., & Han, J. (2011). Locally consistent concept factorization for document clustering. IEEE Transactions on Knowledge and Data Engineering, 23(6), 902–913.CrossRef

Cai, D., He, X., Han, J., & Huang, T. S. (2011). Graph regularized nonnegative matrix factorization for data representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(8), 1548–1560.CrossRef

Chen, Y., Rege, M., Dong, M., & Hua, J. (2008). Non-negative matrix factorization for semi-supervised data clustering. Knowledge and Information Systems, 17(3), 355–379.CrossRef

Cichocki, A., & Anh-Huy, P. (2009). Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, 92(3), 708–721.CrossRef

Cichocki, A., & Zdunek, R. (2006). NMFLAB for signal and image processing. Technical Report, Laboratory for Advanced Brain Signal Processing, BSI RIKEN, Saitama, Japan.

Cichocki, A., & Zdunek, R. (2007). Multilayer nonnegative matrix factorization using projected gradient approaches. International Journal of Neural Systems, 17(6), 431–446.CrossRef

10.

Cichocki, A., Zdunek, R., & Amari, S. (2006). New algorithms for non-negative matrix factorization in applications to blind source separation. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (Vol. 5, pp. 621–624). Toulouse, France.

11.

Deerwester, S. C., Dumais, S. T., Landauer, T. K., Furnas, G. W., & Harshman, R. A., (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 416, 391–407.

12.

Dhillon, I. S., & Sra, S. (2006). Generalized nonnegative matrix approximations with Bregman divergences. In Advances in neural information processing systems (Vol. 18, pp. 283–290).

13.

Ding, C., Li, T., Peng, W., & Park, H. (2006). Orthogonal nonnegative matrix tri-factorizations for clustering. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’06) (pp. 126–135). Philadelphia, PA.

14.

Ding, C., Li, T., & Jordan, M. I. (2010). Convex and semi-nonnegative matrix factorizations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1), 45–55.CrossRef

15.

Ding, C. H., He, X., & Simon, H. D. (2005). On the equivalence of nonnegative matrix factorization and spectral clustering. In Proceedings of the SIAM International Conference on Data Mining (pp. 606–610). Newport Beach, CA.

16.

Donoho, D., & Stodden, V. (2003). When does nonnegative matrix factorization give a correct decomposition into parts? In Advances in neural information processing systems (Vol. 16, pp. 1141–1148). Vancouver, Canada; Cambridge, MA: MIT Press.

17.

Drineas, P., & Mahoney, M. W. (2005). On the Nystrom method for approximating a gram matrix for improved kernel-based learning. Journal of Machine Learning Research, 6, 2153–2175.MathSciNetMATH

18.

Drineas, P., Kannan, R., & Mahoney, M. W. (2006). Fast Monte Carlo algorithms for matrices. III: Computing a compressed approximate matrix decomposition. SIAM Journal on Computing, 36(1), 184–206.

19.

Drineas, P., Mahoney, M. W., & Muthukrishnan, S. (2006). Subspace sampling and relative-error matrix approximation: Column-based methods. In Proceedings of the 10th Annual International Workshop on Randomization and Computation (RANDOM), LNCS (Vol. 4110, pp. 316–326). Berlin: Springer.

20.

Drineas, P., Mahoney, M. W., & Muthukrishnan, S. (2007). Relative-error CUR matrix decompositions. Technical Report, Department of Mathematics, Yale University.

21.

Drineas, P., Mahoney, M. W., & Muthukrishnan, S. (2008). Relative-error CUR matrix decompositions. SIAM Journal on Matrix Analysis and Applications, 30(2), 844–881.MathSciNetMATHCrossRef

22.

Drineas, P., Magdon-Ismail, M., Mahoney, M. W., & Woodruff, D. P. (2012). Fast approximation of matrix coherence and statistical leverage. Journal of Machine Learning Research, 13, 3441–3472.MathSciNetMATH

23.

Gillis, N. (2012). Sparse and unique nonnegative matrix factorization through data preprocessing. Journal of Machine Learning Research, 13, 3349–3386.MathSciNetMATH

24.

Gillis, N., & Luce, R. (2014). Robust near-separable nonnegative matrix factorization using linear optimization. Journal of Machine Learning Research, 15, 1249–1280.MathSciNetMATH

25.

Goreinov, S. A., Tyrtyshnikov, E. E., & Zamarashkin, N. L. (1997). A theory of pseudoskeleton approximations. Linear Algebra and Its Applications, 261, 1–21.MathSciNetMATHCrossRef

26.

Guan, N., Tao, D., Luo, Z., & Yuan, B. (2012). Online nonnegative matrix factorization with robust stochastic approximation. IEEE Transactions on Neural Networks and Learning Systems, 23(7), 1087–1099.CrossRef

27.

Guan, N., Liu, T., Zhang, Y., Tao, D., & Davis, L. S. (2019). Truncated Cauchy non-negative matrix factorization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(1), 246–259.CrossRef

28.

Halko, N., Martinsson, P. G., & Tropp, J. A. (2011). Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM Review, 53(2), 217–288.MathSciNetMATHCrossRef

29.

He, Y., Lu, H., Huang, L., & Xie, S. (2014). Pairwise constrained concept factorization for data representation. Neural Networks, 52, 1–17.MATHCrossRef

30.

He, Z., Xie, S., Zdunek, R., Zhou, G., & Cichocki, A. (2011). Symmetric nonnegative matrix factorization: Algorithms and applications to probabilistic clustering. IEEE Transactions on Neural Networks, 22(12), 2117–2131.CrossRef

31.

Hosoda, K., Watanabe, M., Wersing, H., Korner, E., Tsujino, H., Tamura, H., et al. (2009). A model for learning topographically organized parts-based representations of objects in visual cortex: Topographic nonnegative matrix factorization. Neural Computation, 21, 2605–2633.MathSciNetMATHCrossRef

32.

Hoyer, P. O. (2004). Nonnegative matrix factorization with sparseness constraints. Journal of Machine Learning Research, 5, 1457–1469.MATH

33.

Hsieh, C.-J., & Dhillon, I. S. (2011). Fast coordinate descent methods with variable selection for non-negative matrix factorization. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1064–1072).

34.

Keprt, A., & Snasel, V. (2005). Binary factor analysis with genetic algorithms. In Proceedings of the 4th IEEE International Workshop on Soft Computing as Transdisciplinary Science and Technology (WSTST), AINSC (Vol. 29, pp. 1259–1268). Muroran, Japan; Berlin: Springer.

35.

Kim, D., Sra, S., & Dhillon, I. S. (2008). Fast projection-based methods for the least squares nonnegative matrix approximation problem. Statistical Analysis and Data Mining, 1, 38–51.MathSciNetCrossRef

36.

Kimura, K., Kudo, M., & Tanaka, Y. (2016). A column-wise update algorithm for nonnegative matrix factorization in Bregman divergence with an orthogonal constraint. Machine Learning, 103, 285–306.MathSciNetMATHCrossRef

37.

Kompass, R. (2007). A generalized divergence measure for nonnegative matrix factorization. Neural Computation, 19(3), 780–791.MathSciNetMATHCrossRef

38.

Kong, D., Ding, C., & Huang, H. (2011). Robust nonnegative matrix factorization using \(l_{2,1}\)-norm. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management (pp. 673–682). Glasgow, UK.

39.

Kuang, D., Ding, C., & Park, H. (2012). Symmetric nonnegative matrix factorization for graph clustering. In Proceedings of the 12th SIAM International Conference on Data Mining (pp. 106–117). Anaheim, CA.

40.

Laurberg, H., Christensen, M. G., Plumbley, M. D., Hansen, L. K., & Jensen, S. H. (2008). Theorems on positive data: on the uniqueness of NMF. Computational Intelligence and Neuroscience, 2008, Article ID 764206.

41.

Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by nonnegative matrix factorization. Nature, 401(6755), 788–791.MATHCrossRef

42.

Lee, H., Yoo, J., & Choi, S. (2010). Semi-supervised nonnegative matrix factorization. IEEE Signal Processing Letters, 17(1), 4–7.CrossRef

43.

Li, L., Lebanon, G., & Park, H. (2012). Fast Bregman divergence NMF using Taylor expansion and coordinate descent. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 307–315).

44.

Li, L.-X., Wu, L., Zhang, H.-S., & Wu, F.-X. (2014). A fast algorithm for nonnegative matrix factorization and its convergence. IEEE Transactions on Neural Networks and Learning Systems, 25(10), 1855–1863.CrossRef

45.

Li, M., Bi, W., Kwok, J. T., & Lu, B.-L. (2015). Large-scale Nystrom kernel matrix approximation using randomized SVD. IEEE Transactions on Neural Networks and Learning Systems, 26(1), 152–164.MathSciNetCrossRef

46.

Li, X., & Pang, Y. (2010). Deterministic column-based matrix decomposition. IEEE Transactions on Knowledge and Data Engineering, 22(1), 145–149.MathSciNetCrossRef

47.

Lin, C.-J. (2007). Projected gradients for non-negative matrix factorization. Neural Computation, 19(10), 2756–2779.MathSciNetMATHCrossRef

48.

Lin, C.-J. (2007). On the convergence of multiplicative update algorithms for non-negative matrix factorization. IEEE Transactions on Neural Networks, 18(6), 1589–1596.CrossRef

49.

Liu, H., Wu, Z., Li, X., Cai, D., & Huang, T. S. (2012). Constrained nonnegative matrix factorization for image representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(7), 1299–1311.CrossRef

50.

Liu, H., Li, X., & Zheng, X. (2013). Solving non-negative matrix factorization by alternating least squares with a modified strategy. Data Mining and Knowledge Discovery, 26(3), 435–451.MathSciNetMATHCrossRef

51.

Liu, T., Gong, M., & Tao, D. (2017). Large-cone nonnegative matrix factorization. IEEE Transactions on Neural Networks and Learning Systems, 28(9), 2129–2142.MathSciNet

52.

Mirzal, A. (2014). A convergent algorithm for orthogonal nonnegative matrix factorization. Journal of Computational and Applied Mathematics, 260, 149–166.MathSciNetMATHCrossRef

53.

Paatero, P., & Tapper, U. (1994). Positive matrix factorization: A nonnegative factor model with optimal utilization of error estimates of data values. Environmetrics, 5(2), 111–126.CrossRef

54.

Pascual-Montano, A., Carazo, J. M., Kochi, K., Lehmann, D., & Pascual-Marqui, R. D. (2006). Nonsmooth nonnegative matrix factorization (nsNMF). EEE Transactions on Pattern Analysis and Machine Intelligence, 28(3), 403–415.CrossRef

55.

Peharz, R., & Pernkopf, F. (2012). Sparse nonnegative matrix factorization with \(l^0\)-constraints. Neurocomputing, 80, 38–46.CrossRef

56.

Sajda, P., Du, S., Brown, T. R., Stoyanova, R., Shungu, D. C., Mao, X., et al. (2004). Nonnegative matrix factorization for rapid recovery of constituent spectra in magnetic resonance chemical shift imaging of the brain. IEEE Transactions on Medical Imaging, 23(12), 1453–1465.CrossRef

57.

Vavasis, S. A. (2009). On the complexity of nonnegative matrix factorization. SIAM Journal on Optimization, 20(3), 1364–1377.MathSciNetMATHCrossRef

58.

Wang, D., & Lu, H. (2013). On-line learning parts-based representation via incremental orthogonal projective non-negative matrix factorization. Signal Processing, 93, 1608–1623.CrossRef

59.

Wang, F., Li, T., & Zhang, C. (2008). Semi-supervised clustering via matrix factorization. In Proceedings of the SIAM International Conference on Data Mining (pp. 1–12). Atlanta, GA.

60.

Wang, J. J.-Y., & Gao, X. (2015). Max–min distance nonnegative matrix factorization. Neural Networks, 61, 75–84.MATHCrossRef

61.

Wang, S., & Zhang, Z. (2013). Improving CUR matrix decomposition and the Nystrom approximation via adaptive sampling. Journal of Machine Learning Research, 14, 2729–2769.MathSciNetMATH

62.

Wang, S., Zhang, Z., & Zhang, T. (2016). Towards more efficient SPSD matrix approximation and CUR matrix decomposition. Journal of Machine Learning Research, 17, 1–49.MathSciNetMATH

63.

Williams, C., & Seeger, M. (2001). Using the Nystrom method to speedup kernel machines. In T. Leen, T. Dietterich, & V. Tresp (Eds.), Advances in neural information processing systems (Vol. 13, pp. 682–690). Cambridge, MA, USA: MIT Press.

64.

Wu, X., Kumar, V., Quinlan, J. R., Ghosh, J., Yang, Q., Motoda, H., et al. (2008). Top 10 algorithms in data mining. Knowledge and Information Systems, 14(1), 1–37.CrossRef

65.

Xu, W., & Gong, Y. (2004). Document clustering by concept factorization. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 202–209).

66.

Yang, S., & Yi, Z. (2010). Convergence analysis of non-negative matrix factorization for BSS algorithm. Neural Processing Letters, 31, 45–64.CrossRef

67.

Yang, S., Yi, Z., Ye, M., & He, X. (2014). Convergence analysis of graph regularized non-negative matrix factorization. IEEE Transactions on Knowledge and Data Engineering, 26(9), 2151–2165.CrossRef

68.

Yang, Z., & Laaksonen, J. (2007). Multiplicative updates for non-negative projections. Neurocomputing, 71, 363–373.CrossRef

69.

Yang, Z., & Oja, E. (2010). Linear and nonlinear projective nonnegative matrix factorization. IEEE Transactions on Neural Networks, 21(5), 734–749.CrossRef

70.

Yokota, T., Zdunek, R., Cichocki, A., & Yamashita, Y. (2015). Smooth nonnegative matrix and tensor factorizations for robust multi-way data analysis. Signal Processing, 113, 234–249.CrossRef

71.

Zafeiriou, S., & Petrou, M. (2010). Nonlinear non-negative component analysis algorithms. IEEE Transactions on Image Processing, 19(4), 1050–1066.MathSciNetMATHCrossRef

72.

Zafeiriou, S., Tefas, A., Buciu, I., & Pitas, I. (2006). Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification. IEEE Transactions on Neural Networks, 17(3), 683–695.CrossRef

73.

Zdunek, R., & Cichocki, A. (2007). Nonnegative matrix factorization with constrained second-order optimization. Signal Processing, 87, 1904–1916.MATHCrossRef

74.

Zdunek, R., & Cichocki, A. (2008). Fast nonnegative matrix factorization algorithms using projected gradient approaches for large-scale problems. Computational Intelligence and Neuroscience, 2008, Article ID 939567.

75.

Zhang, J., Wei, L., Feng, X., Ma, Z., & Wang, Y. (2008). Pattern expression nonnegative matrix factorization: Algorithm and applications to blind source separation. Computational Intelligence and Neuroscience, 2008, Article ID 168769.

76.

Zhang, X., Zong, L., Liu, X., & Luo, J. (2016). Constrained clustering with nonnegative matrix factorization. IEEE Transactions on Neural Networks and Learning Systems, 27(7), 1514–1526.MathSciNetCrossRef

77.

Zhang, Z., & Zhao, K. (2013). Low-rank matrix approximation with manifold regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(7), 1717–1729.CrossRef

78.

Zhao, R., & Tan, V. Y. F. (2017). Online nonnegative matrix factorization with outliers. IEEE Transactions on Signal Processing, 65(3), 555–570.MathSciNetMATHCrossRef

79.

Zhou, G., Xie, S., Yang, Z., Yang, J.-M., & He, Z. (2011). Minimum-volume-constrained nonnegative matrix factorization: Enhanced ability of learning parts. IEEE Transactions on Neural Networks, 22(10), 1626–1637.CrossRef

Titel: Nonnegative Matrix Factorization
verfasst von: Ke-Lin Du
M. N. S. Swamy
Verlag: Springer London
Buch: Neural Networks and Statistical Learning
Print ISBN: 978-1-4471-7451-6

Electronic ISBN: 978-1-4471-7452-3

Copyright-Jahr: 2019
DOI: https://doi.org/10.1007/978-1-4471-7452-3_14

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"