Skip to main content
Top
Published in: Neural Processing Letters 2/2019

03-10-2018

Calibrated Multi-label Classification with Label Correlations

Authors: Zhi-Fen He, Ming Yang, Hui-Dong Liu, Lei Wang

Published in: Neural Processing Letters | Issue 2/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Multi-label classification is a special learning task where each instance may be associated with multiple labels simultaneously. There are two main challenges: (a) discovering and exploiting the label correlations automatically, and (b) separating the relevant labels from the irrelevant labels of each instance effectively. Nevertheless, many existing multi-label classification algorithms fail to deal with both challenges at the same time. In this paper, we integrate multi-label classification, label correlations and threshold calibration into a unified learning framework, and propose calibrated multi-label classification with label correlations, named CMLLC. Specifically, we firstly introduce a label covariance matrix to characterize the label correlations and a virtual label to calibrate label decision threshold of each instance. Secondly, the framework of our CMLLC model is constructed for joint learning of the label correlations and model parameters corresponding to each label and the virtual label. Lastly, the optimization problem is jointly convex and solved by an alternating iterative method. Experimental results on sixteen multi-label benchmark datasets in terms of five evaluation criteria demonstrate that CMLLC outperforms the state-of-the-art multi-label classification algorithms.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Schapire RE, Singer Y (2000) Boostexter: a boosting-based system for text categorization. Mach Learn 39(2/3):135–168CrossRef Schapire RE, Singer Y (2000) Boostexter: a boosting-based system for text categorization. Mach Learn 39(2/3):135–168CrossRef
2.
go back to reference Zhang ML, Zhou ZH (2006) Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351CrossRef Zhang ML, Zhou ZH (2006) Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351CrossRef
3.
go back to reference Yan Y, Wang Y, Gao WC (2018) LSTM: multi-label ranking for document classification. Neural Process Lett 47(1):117–138CrossRef Yan Y, Wang Y, Gao WC (2018) LSTM: multi-label ranking for document classification. Neural Process Lett 47(1):117–138CrossRef
4.
go back to reference Boutell MR, Luo J, Luo JB, Shen XP, Brown CM (2004) Learning multi-label scene classification. Pattern Recognit 37(9):1757–1771CrossRef Boutell MR, Luo J, Luo JB, Shen XP, Brown CM (2004) Learning multi-label scene classification. Pattern Recognit 37(9):1757–1771CrossRef
5.
go back to reference Jiang A, Wang C, Zhu Y (2008) Calibrated rank-SVM for multi-label image categorization. In: Proceedings of the international joint conference on neural networks, Hong Kong, China, pp 1450–1455 Jiang A, Wang C, Zhu Y (2008) Calibrated rank-SVM for multi-label image categorization. In: Proceedings of the international joint conference on neural networks, Hong Kong, China, pp 1450–1455
6.
go back to reference Liu W, Yang X, Tao D (2018) Multiview dimension reduction via Hessian multiset canonical correlations. Inf Fusion 41:119–128CrossRef Liu W, Yang X, Tao D (2018) Multiview dimension reduction via Hessian multiset canonical correlations. Inf Fusion 41:119–128CrossRef
7.
go back to reference Yu J, Zhang B, Kuang Z (2017) iPrivacy: image privacy protection by identifying sensitive objects via deep multi-task learning. IEEE Trans Inf Forensics Secur 12(5):1005–1016CrossRef Yu J, Zhang B, Kuang Z (2017) iPrivacy: image privacy protection by identifying sensitive objects via deep multi-task learning. IEEE Trans Inf Forensics Secur 12(5):1005–1016CrossRef
8.
go back to reference Yu J, Yang X, Gao F (2017) Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans Cybern 47(12):4014–4024CrossRef Yu J, Yang X, Gao F (2017) Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans Cybern 47(12):4014–4024CrossRef
9.
go back to reference Tao D, Hong C, Yu J (2015) Multimodal deep autoencoder for human pose recovery. IEEE Trans Image Process 24(12):5659–5670MathSciNetCrossRef Tao D, Hong C, Yu J (2015) Multimodal deep autoencoder for human pose recovery. IEEE Trans Image Process 24(12):5659–5670MathSciNetCrossRef
10.
go back to reference Trohidis K, Tsoumakas G, Kalliris G, Vlahavas IP (2008) Multilabel classification of music into emotions. In: Proceedings of the 9th international conference on music information retrieval, Philadephia, PA, USA, pp 325–330 Trohidis K, Tsoumakas G, Kalliris G, Vlahavas IP (2008) Multilabel classification of music into emotions. In: Proceedings of the 9th international conference on music information retrieval, Philadephia, PA, USA, pp 325–330
11.
go back to reference Zhang ML, Zhou ZH (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837CrossRef Zhang ML, Zhou ZH (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837CrossRef
12.
go back to reference Huang SJ, Yu Y, Zhou ZH (2012) Multi-label hypothesis reuse. In: Proceedings of the 18th ACM SIGKDD international conference on knowledge discovery and data mining, Beijing, China, pp 525–533 Huang SJ, Yu Y, Zhou ZH (2012) Multi-label hypothesis reuse. In: Proceedings of the 18th ACM SIGKDD international conference on knowledge discovery and data mining, Beijing, China, pp 525–533
13.
go back to reference Elisseeff A, Weston J (2001) A kernel method for multi-labelled classification. In: Proceedings of the 14th conference on neural information processing systems (NIPS2001), Vancouver, British Columbia, Canada, pp 681–687 Elisseeff A, Weston J (2001) A kernel method for multi-labelled classification. In: Proceedings of the 14th conference on neural information processing systems (NIPS2001), Vancouver, British Columbia, Canada, pp 681–687
14.
go back to reference Zhang ML, Pena JM, Robles V (2009) Feature selection for multi-label naive Bayes classification. Inf Sci 179(19):3218–3229CrossRef Zhang ML, Pena JM, Robles V (2009) Feature selection for multi-label naive Bayes classification. Inf Sci 179(19):3218–3229CrossRef
15.
go back to reference Zhang ML (2009) ML-RBF: RBF neural networks for multi-label learning. Neural Process Lett 29(2):61–74CrossRef Zhang ML (2009) ML-RBF: RBF neural networks for multi-label learning. Neural Process Lett 29(2):61–74CrossRef
16.
go back to reference Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333–359 Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333–359
17.
go back to reference Tsoumakas G, Katakis I, Vlahavas I (2008) Effective and efficient multilabel classification in domains with large number of labels. In: Proceedings of ECML/PKDD 2008 workshop on mining multidimensional data, Antwerp, Belgium, pp 30–44 Tsoumakas G, Katakis I, Vlahavas I (2008) Effective and efficient multilabel classification in domains with large number of labels. In: Proceedings of ECML/PKDD 2008 workshop on mining multidimensional data, Antwerp, Belgium, pp 30–44
18.
go back to reference Ghamrawi N, Mccallum A (2005) Collective multilabel classification. In: Proceedings of the 14th ACM international conference on information and knowledge management, Bremen, Germany, pp 195–200 Ghamrawi N, Mccallum A (2005) Collective multilabel classification. In: Proceedings of the 14th ACM international conference on information and knowledge management, Bremen, Germany, pp 195–200
19.
go back to reference Godbole S, Sarawagi S (2004) Discriminative methods for multi-labeled classification. In: Dai H, Srikant R, Zhang C (eds) Lecture Notes in Artificial Intelligence, vol 3056. Springer, Berlin, pp 22–30 Godbole S, Sarawagi S (2004) Discriminative methods for multi-labeled classification. In: Dai H, Srikant R, Zhang C (eds) Lecture Notes in Artificial Intelligence, vol 3056. Springer, Berlin, pp 22–30
20.
go back to reference Chen G, Song YQ, Wang F, et al (2008) Semi-supervised multi-label learning by solving a Sylvester equation. In: SIAM conference on data mining, Atlanta, Georgia, pp 410–419 Chen G, Song YQ, Wang F, et al (2008) Semi-supervised multi-label learning by solving a Sylvester equation. In: SIAM conference on data mining, Atlanta, Georgia, pp 410–419
21.
go back to reference Gu Q, Li Z, Han J (2011) Correlated multi-label feature selection. In: Proceedings of the 20th ACM international conference on information and knowledge management, Glasgow, Scotland, UK, pp 1087–1096 Gu Q, Li Z, Han J (2011) Correlated multi-label feature selection. In: Proceedings of the 20th ACM international conference on information and knowledge management, Glasgow, Scotland, UK, pp 1087–1096
22.
go back to reference Zhang Y, Yeung DY (2013) Multilabel relationship learning. ACM Trans Knowl Discov Data 7(2):1–30CrossRef Zhang Y, Yeung DY (2013) Multilabel relationship learning. ACM Trans Knowl Discov Data 7(2):1–30CrossRef
23.
go back to reference Zhu Y, Kwok JT, Zhou ZH (2017) Multi-Label Learning with Global and Local Label Correlation. IEEE Trans Knowl Data Eng, arXiv preprint, arXiv:1704:01415 Zhu Y, Kwok JT, Zhou ZH (2017) Multi-Label Learning with Global and Local Label Correlation. IEEE Trans Knowl Data Eng, arXiv preprint, arXiv:​1704:​01415
24.
go back to reference He ZF, Yang M, Liu HD (2014) Joint learning of multi-label classification and label correlations. J Softw 25(9):1967–1981 (in Chinese)MATH He ZF, Yang M, Liu HD (2014) Joint learning of multi-label classification and label correlations. J Softw 25(9):1967–1981 (in Chinese)MATH
25.
go back to reference Hullermeier E, Furnkranz J, Cheng W, Brinker K (2008) Label ranking by learning pairwise preferences. Artif Intell 172(16):1897–1916MathSciNetCrossRef Hullermeier E, Furnkranz J, Cheng W, Brinker K (2008) Label ranking by learning pairwise preferences. Artif Intell 172(16):1897–1916MathSciNetCrossRef
26.
go back to reference Furnkranz F, Hullermeier E, Mencia EL, Brinker K (2008) Multilabel classification via calibrated label ranking. Mach Learn 73(2):133–153CrossRef Furnkranz F, Hullermeier E, Mencia EL, Brinker K (2008) Multilabel classification via calibrated label ranking. Mach Learn 73(2):133–153CrossRef
27.
go back to reference Read J (2008) A pruned problem transformation method for multi-label classification. In: Proceedings of New Zealand computer science research student conference, Christchurch, New Zealand, pp 143–150 Read J (2008) A pruned problem transformation method for multi-label classification. In: Proceedings of New Zealand computer science research student conference, Christchurch, New Zealand, pp 143–150
28.
go back to reference Tsoumakas G, Katakis I, Vlahavas I (2011) Random k-labelsets for multilabel classification. IEEE Trans Knowl Data Eng 23(7):1079–1089CrossRef Tsoumakas G, Katakis I, Vlahavas I (2011) Random k-labelsets for multilabel classification. IEEE Trans Knowl Data Eng 23(7):1079–1089CrossRef
29.
go back to reference Gharroudi O, Elghazel H, Aussem A (2015) Calibrated k-labelsets for ensemble multi-label classification. In: Proceedings of international conference on neural information processing, pp 573–582 Gharroudi O, Elghazel H, Aussem A (2015) Calibrated k-labelsets for ensemble multi-label classification. In: Proceedings of international conference on neural information processing, pp 573–582
30.
go back to reference He ZF, Yang M, Liu HD (2015) Multi-task joint feature selection for multi-label classification. Chin J Electron 24(CJE–2):281–287CrossRef He ZF, Yang M, Liu HD (2015) Multi-task joint feature selection for multi-label classification. Chin J Electron 24(CJE–2):281–287CrossRef
31.
go back to reference Sun Z, Zhao Y, Cao D, Hao H (2017) Hierarchical multilabel classification with optimal path prediction. Neural Process Lett 45(1):263–277CrossRef Sun Z, Zhao Y, Cao D, Hao H (2017) Hierarchical multilabel classification with optimal path prediction. Neural Process Lett 45(1):263–277CrossRef
32.
go back to reference Xu J (2012) An efficient multi-label support vector machine with a zero label. Expert Syst Appl 39(5):4796–4804CrossRef Xu J (2012) An efficient multi-label support vector machine with a zero label. Expert Syst Appl 39(5):4796–4804CrossRef
33.
go back to reference Xu J (2014) Multi-label core vector machine with a zero label. Pattern Recognit 47(7):2542–2557CrossRef Xu J (2014) Multi-label core vector machine with a zero label. Pattern Recognit 47(7):2542–2557CrossRef
34.
go back to reference Clare A, King RD (2001) Knowledge discovery in multi-label phenotypedata. In: Raedt LD, Siebes A (eds) Lecture Notes in Computer Science. Springer, Berlin, pp 42–53MATH Clare A, King RD (2001) Knowledge discovery in multi-label phenotypedata. In: Raedt LD, Siebes A (eds) Lecture Notes in Computer Science. Springer, Berlin, pp 42–53MATH
35.
go back to reference Zhang ML, Zhou ZH (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recognit 40(7):2038–2048CrossRef Zhang ML, Zhou ZH (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recognit 40(7):2038–2048CrossRef
36.
go back to reference Kwok JT (1999) Moderating the outputs of support vector machine classifiers. IEEE Trans Neural Netw 10(5):1018–1031CrossRef Kwok JT (1999) Moderating the outputs of support vector machine classifiers. IEEE Trans Neural Netw 10(5):1018–1031CrossRef
37.
go back to reference Xu J (2013) Fast multi-label core vector machine. Pattern Recognit 46(3):885–898CrossRef Xu J (2013) Fast multi-label core vector machine. Pattern Recognit 46(3):885–898CrossRef
38.
go back to reference Zhang Y, Yeung DY (2010) A convex formulation for learning task relationships in multi-task learning. In: Proceedings of the 26th conference on uncertainty in artificial intelligence, Catalina Island, California, pp 733–742 Zhang Y, Yeung DY (2010) A convex formulation for learning task relationships in multi-task learning. In: Proceedings of the 26th conference on uncertainty in artificial intelligence, Catalina Island, California, pp 733–742
39.
go back to reference Chao G, Sun S (2016) Consensus and complementarity based maximum entropy discrimination for multi-view classification. Inf Sci 367–368:296–310CrossRef Chao G, Sun S (2016) Consensus and complementarity based maximum entropy discrimination for multi-view classification. Inf Sci 367–368:296–310CrossRef
40.
go back to reference Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, New YorkCrossRef Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, New YorkCrossRef
41.
go back to reference Chen J, Ye J (2008) Training SVM with indefinite kernels. In: Proceedings of the 25th international conference on machine learning, Helsinki, Finland, pp 136–143 Chen J, Ye J (2008) Training SVM with indefinite kernels. In: Proceedings of the 25th international conference on machine learning, Helsinki, Finland, pp 136–143
42.
go back to reference Tsoumakas G, Xioufis ES, Vilcek J (2011) Mulan: a java library for multi-label learning. J Mach Learn Res 12(7):2411–2414MathSciNetMATH Tsoumakas G, Xioufis ES, Vilcek J (2011) Mulan: a java library for multi-label learning. J Mach Learn Res 12(7):2411–2414MathSciNetMATH
43.
go back to reference Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetMATH Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetMATH
Metadata
Title
Calibrated Multi-label Classification with Label Correlations
Authors
Zhi-Fen He
Ming Yang
Hui-Dong Liu
Lei Wang
Publication date
03-10-2018
Publisher
Springer US
Published in
Neural Processing Letters / Issue 2/2019
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-018-9925-2

Other articles of this Issue 2/2019

Neural Processing Letters 2/2019 Go to the issue