Abstract
Nonnegative matrix factorization (NMF) is one widely used feature extraction technology in the tasks of image clustering and image classification. For the former task, various unsupervised NMF methods based on the data distribution structure information have been proposed. While for the latter task, the label information of the dataset is one very important guiding. However, most previous proposed supervised NMF methods emphasis on imposing the discriminant constraints on the coefficient matrix. When dealing with new coming samples, the transpose or the pseudoinverse of the basis matrix is used to project these samples to the low dimension space. In this way, the label influence to the basis matrix is indirect. Although, there are also some methods trying to constrain the basis matrix in NMF framework, either they only restrict within-class samples or impose improper constraint on the basis matrix. To address these problems, in this article a novel NMF framework named discriminative and orthogonal subspace constraints-based nonnegative matrix factorization (DOSNMF) is proposed. In DOSNMF, the discriminative constraints are imposed on the projected subspace instead of the directly learned representation. In this manner, the discriminative information is directly connected with the projected subspace. At the same time, an orthogonal term is incorporated in DOSNMF to adjust the orthogonality of the learned basis matrix, which can ensure the orthogonality of the learned subspace and improve the sparseness of the basis matrix at the same time. This framework can be implemented in two ways. The first way is based on the manifold learning theory. In this way, two graphs, i.e., the intrinsic graph and the penalty graph, are constructed to capture the intra-class structure and the inter-class distinctness. With this design, both the manifold structure information and the discriminative information of the dataset are utilized. For convenience, we name this method as the name of the framework, i.e., DOSNMF. The second way is based on the Fisher’s criterion, we name it Fisher’s criterion-based DOSNMF (FDOSNMF). The objective functions of DOSNMF and FDOSNMF can be easily optimized using multiplicative update (MU) rules. The new methods are tested on five datasets and compared with several supervised and unsupervised variants of NMF. The experimental results reveal the effectiveness of the proposed methods.
- Shounan An, Jiho Yoo, and Seungjin Choi. 2011. Manifold-respecting discriminant nonnegative matrix factorization. Pattern Recogn. Lett. 32, 6 (2011), 832--837. Google ScholarDigital Library
- Alain Baccini, Ph Besse, and A. Falguerolles. 1996. A L1-norm PCA and a heuristic approach. Ord. Symbol. Data Anal. 1, 1 (1996), 359--368.Google ScholarCross Ref
- Stephen Boyd and Lieven Vandenberghe. 2004. Convex Optimization. Cambridge University Press. Google ScholarDigital Library
- Deng Cai, Xiaofei He, Jiawei Han, and Thomas S. Huang. 2011. Graph regularized nonnegative matrix factorization for data representation. IEEE Trans. Pattern Anal. Mach. Intell. 33 (2011), 1548--1560. Google ScholarDigital Library
- Yongsheng Dong, Dacheng Tao, and Xuelong Li. 2015a. Nonnegative multiresolution representation-based texture image classification. ACM Trans. Intell. Syst. Technol. 7, 1 (2015), 4:1--4:21. Google ScholarDigital Library
- Yongsheng Dong, Dacheng Tao, Xuelong Li, Jinwen Ma, and Jiexin Pu. 2015b. Texture classification and retrieval using shearlets and linear regression. IEEE Trans. Cybernet. 45, 3 (2015), 358--369.Google ScholarCross Ref
- David Donoho and Victoria Stodden. 2004. When does non-negative matrix factorization give a correct decomposition into parts? In Proceedings of the Conference on Advances in Neural Information Processing Systems. 1141--1148. Google ScholarDigital Library
- Hongchang Gao, Feiping Nie, Weidong Cai, and Heng Huang. 2015. Robust capped norm nonnegative matrix factorization: Capped norm NMF. In Proceedings of ACM International on Conference on Information and Knowledge Management. 871--880. Google ScholarDigital Library
- Naiyang Guan, Tongliang Liu, Yangmuzi Zhang, Dacheng Tao, and Larry Steven Davis. 2017. Truncated cauchy non-negative matrix factorization for robust subspace learning. IEEE Trans. Pattern Anal. Mach. Intell. (2017).Google Scholar
- Naiyang Guan, Dacheng Tao, Zhigang Luo, and Bo Yuan. 2011. Manifold regularized discriminative nonnegative matrix factorization with fast gradient descent. IEEE Trans. Image Process. 20, 7 (2011), 2030--2048. Google ScholarDigital Library
- Lihua Guo, Chenggang Guo, Lei Li, Qinghua Huang, Yanshan Li, and Xuelong Li. 2018. Two-stage local constrained sparse coding for fine-grained visual categorization. Sci. China Info. Sci. 61, 1 (2018), 018104.Google ScholarCross Ref
- Yuchen Guo, Guiguang Ding, Li Liu, Jungong Han, and Ling Shao. 2017. Learning to hash with optimized anchor embedding for scalable retrieval. IEEE Trans. Image Process. 26, 3 (2017), 1344--1354. Google ScholarDigital Library
- Patrik O. Hoyer. 2002. Non-negative sparse coding. In Proceedings of the IEEE Workshop Neural Networks for Signal Processing. 557--565.Google ScholarCross Ref
- Jin Huang, Feiping Nie, Heng Huang, and Chris Ding. 2014. Robust manifold nonnegative matrix factorization. ACM Trans. Knowl. Discov. Data 8, 3 (2014), 11. Google ScholarDigital Library
- Aapo Hyvärinen, Juha Karhunen, and Erkki Oja. 2004. Independent Component Analysis. Vol. 46. John Wiley 8 Sons.Google Scholar
- Alan Julian Izenman. 2013. Linear discriminant analysis. In Modern Multivariate Statistical Techniques. Springer, 237--280.Google Scholar
- Ian Jolliffe. 2005. Principal Component Analysis. Wiley Online Library.Google Scholar
- Daniel D. Lee and H. Sebastian Seung. 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401, 6755 (1999), 788--791.Google Scholar
- Daniel D. Lee and H. Sebastian Seung. 2000. Algorithms for non-negative matrix factorization. In Proceedings of the Advances in Neural Information Processing Systems. 556--562.Google Scholar
- Stan Z. Li, Xin Wen Hou, Hong Jiang Zhang, and Qian Sheng Cheng. 2001. Learning spatially localized, parts-based representation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1. 1--207.Google ScholarCross Ref
- Xuelong Li, Guosheng Cui, and Yongsheng Dong. 2017a. Graph regularized non-negative low-rank matrix factorization for image clustering. IEEE Trans. Cybernet. 47, 11 (2017), 3840--3853.Google ScholarCross Ref
- Xuelong Li, Guosheng Cui, and Yongsheng Dong. 2017b. Refined-graph regularization-based nonnegative matrix factorization. ACM Trans. Intell. Syst. Technol. 9, 1 (2017), 1:1--1:21. Google ScholarDigital Library
- Xuelong Li, Quanmao Lu, Yongsheng Dong, and Dacheng Tao. 2017. SCE: A manifold regularized set-covering method for data partitioning. IEEE Trans. Neural Netw. Learn. Syst. PP, 99 (2017), 1--14.Google Scholar
- Zhao Li, Xindong Wu, and Hong Peng. 2010. Nonnegative matrix factorization on orthogonal subspace. Pattern Recogn. Lett. 31, 9 (2010), 905--911. Google ScholarDigital Library
- Haifeng Liu, Zhaohui Wu, Xuelong Li, Deng Cai, and Thomas S. Huang. 2012. Constrained nonnegative matrix factorization for image representation. IEEE Trans. Pattern Anal. Mach. Intell. 34, 7 (2012), 1299--1311. Google ScholarDigital Library
- Tongliang Liu, Mingming Gong, and Dacheng Tao. 2017. Large-cone nonnegative matrix factorization. IEEE Trans. Neural Netw. Learn. Syst. 28, 9 (2017), 2129--2142.Google Scholar
- Tongliang Liu and Dacheng Tao. 2016. On the performance of Manhattan nonnegative matrix factorization. IEEE Trans. Neural Netw. Learn. Syst. 27, 9 (2016), 1851--1863.Google ScholarCross Ref
- Xianzhong Long, Hongtao Lu, Yong Peng, and Wenbin Li. 2014. Graph regularized discriminative non-negative matrix factorization for face recognition. Multimedia Tools Appl. 72, 3 (2014), 2679--2699. Google ScholarDigital Library
- Yuwu Lu, Zhihui Lai, Yong Xu, Xuelong Li, David Zhang, and Chun Yuan. 2017. Nonnegative discriminant matrix factorization. IEEE Trans. Circ. Syst. Video Technol. 27, 7 (2017), 1392--1405.Google ScholarDigital Library
- John Makhoul, Salim Roucos, and Herbert Gish. 1985. Vector quantization in speech coding. In Proc. IEEE, Vol. 73. IEEE, 1551--1588.Google ScholarCross Ref
- Feiping Nie, Rui Zhang, and Xuelong Li. 2017. A generalized power iteration method for solving quadratic problem on the stiefel manifold. Sci. China Info. Sci. 60, 11 (2017), 112101:1--112101:10.Google Scholar
- Weiya Ren, Guohui Li, Dan Tu, and Li Jia. 2014. Nonnegative matrix factorization with regularizations. IEEE J. Emerg. Select. Topics Circ. Syst. 4, 1 (2014), 153--164.Google ScholarCross Ref
- Dapeng Tao, Dacheng Tao, Xuelong Li, and Xinbo Gao. 2017. Large sparse cone non-negative matrix factorization for image annotation. ACM Trans. Intell. Syst. Technol. 8, 3 (2017), 37:1--37:21. Google ScholarDigital Library
- George Trigeorgis, Konstantinos Bousmalis, Stefanos Zafeiriou, and Bjoern Schuller. 2014. A deep semi-NMF model for learning hidden representations. In Proceedings of the International Conference on Machine Learning. 1692--1700. Google ScholarDigital Library
- Jing Wang, Feng Tian, Xiao Wang, H. C. Yu, Changhong Liu, and Liang Yang. 2017. Multi-component nonnegative matrix factorization. In Proceedings of the International Joint Conferences on Artificial Intelligence. Google ScholarDigital Library
- Yanhui Xiao, Zhenfeng Zhu, Yao Zhao, Yunchao Wei, Shikui Wei, and Xuelong Li. 2014. Topographic NMF for data representation. IEEE Trans. Cybernet. 44, 10 (2014), 1762--1771.Google ScholarCross Ref
- Xing Xu, Li He, Huimin Lu, Atsushi Shimada, and Rin-Ichiro Taniguchi. 2017a. Non-linear matrix completion for social image tagging. IEEE Access 5 (2017), 6688--6696.Google ScholarCross Ref
- Xing Xu, Fumin Shen, Yang Yang, Jie Shao, and Zi Huang. 2017b. Transductive visual-semantic embedding for zero-shot learning. In Proceedings of the ACM on International Conference on Multimedia Retrieval. 41--49. Google ScholarDigital Library
- Xing Xu, Fumin Shen, Yang Yang, Dongxiang Zhang, Heng Tao Shen, and Jingkuan Song. 2017c. Matrix tri-factorization with manifold regularizations for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3798--3807.Google ScholarCross Ref
- Jianchao Yang, Shuicheng Yang, Yun Fu, Xuelong Li, and Thomas Huang. 2008. Non-negative graph embedding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--8.Google Scholar
- Chao Yao, Ya-Feng Liu, Bo Jiang, Jungong Han, and Junwei Han. 2017. LLE score: A new filter-based unsupervised feature selection method based on nonlinear manifold embedding and its application to image recognition. IEEE Trans. Image Process. 26, 11 (2017), 5257--5269.Google ScholarDigital Library
- Chao Yao, Zhaoyang Lu, Jing Li, Yamei Xu, and Jungong Han. 2014. A subset method for improving linear discriminant analysis. Neurocomputing 138 (2014), 310--315.Google ScholarCross Ref
- Yuan Yuan, Xuelong Li, Yanwei Pang, Xin Lu, and Dacheng Tao. 2009. Binary sparse nonnegative matrix factorization. IEEE Trans. Circ. Syst. Video Technol. 19, 5 (2009), 772--777. Google ScholarDigital Library
- Stefanos Zafeiriou, Anastasios Tefas, Ioan Buciu, and Ioannis Pitas. 2006. Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification. IEEE Trans. Neural Netw. 17, 3 (2006), 683--695. Google ScholarDigital Library
- Xueyi Zhao, Xi Li, Zhongfei Zhang, Chunhua Shen, Yueting Zhuang, Lixin Gao, and Xuelong Li. 2016. Scalable linear visual feature learning via online parallel nonnegative matrix factorization. IEEE Trans. Neural Netw. Learn. Syst. 27, 12 (2016), 2628--2642.Google ScholarCross Ref
- Ruicong Zhi, Markus Flierl, Qiuqi Ruan, and W. Bastiaan Kleijn. 2011. Graph-preserving sparse nonnegative matrix factorization with application to facial expression recognition. IEEE Trans. Syst., Man, Cybernet., Part B (Cybernet.) 41, 1 (2011), 38--52. Google ScholarDigital Library
- Xiaofeng Zhu, Xuelong Li, and Shichao Zhang. 2016. Block-row sparse multiview multilabel learning for image classification. IEEE Trans. Cybernet. 46, 2 (2016), 450--461.Google ScholarCross Ref
Index Terms
- Discriminative and Orthogonal Subspace Constraints-Based Nonnegative Matrix Factorization
Recommendations
Refined-Graph Regularization-Based Nonnegative Matrix Factorization
Regular Papers and Special Issue: Data-driven Intelligence for Wireless NetworkingNonnegative matrix factorization (NMF) is one of the most popular data representation methods in the field of computer vision and pattern recognition. High-dimension data are usually assumed to be sampled from the submanifold embedded in the original ...
Orthogonal nonnegative matrix t-factorizations for clustering
KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data miningCurrently, most research on nonnegative matrix factorization (NMF)focus on 2-factor $X=FG^T$ factorization. We provide a systematicanalysis of 3-factor $X=FSG^T$ NMF. While it unconstrained 3-factor NMF is equivalent to it unconstrained 2-factor NMF, ...
Max-min distance nonnegative matrix factorization
Nonnegative Matrix Factorization (NMF) has been a popular representation method for pattern classification problems. It tries to decompose a nonnegative matrix of data samples as the product of a nonnegative basis matrix and a nonnegative coefficient ...
Comments