Skip to main content
Top

31-07-2024

Partial Label Learning with Noisy Labels

Authors: Pan Zhao, Long Tang, Zhigeng Pan

Published in: Annals of Data Science

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Partial label learning (PLL) is a particular problem setting within weakly supervised learning. In PLL, each sample corresponds to a candidate label set in which only one label is true. However, in some practical application scenarios, the emergence of label noise can make some candidate sets lose their true labels, leading to a decline in model performance. In this work, a robust training strategy for PLL, derived from the joint training with co-regularization (JoCoR), is proposed to address this issue in PLL. Specifically, the proposed approach constructs two separate PLL models and a joint loss. The joint loss consists of not only two PLL losses but also a co-regularization term measuring the disagreement of the two models. By automatically selecting samples with small joint loss and using them to update the two models, our proposed approach is able to filter more and more suspected samples with noise candidate label sets. Gradually, the robustness of the PLL models to label noise strengthens due to the reduced disagreement of the two models. Experiments are conducted on two state-of-the-art PLL models using benchmark datasets under various noise levels. The results show that the proposed method can effectively stabilize the training process and reduce the model's overfitting to noisy candidate label sets.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Shi Y (2022) Advances in big data analytics: theory, algorithm and practice. Springer, SingaporeCrossRef Shi Y (2022) Advances in big data analytics: theory, algorithm and practice. Springer, SingaporeCrossRef
2.
go back to reference Shi Y, Tian Y, Kou G et al (2011) Optimization based data mining: theory and applications. Springer, BerlinCrossRef Shi Y, Tian Y, Kou G et al (2011) Optimization based data mining: theory and applications. Springer, BerlinCrossRef
3.
go back to reference Olson DL, Shi Y (2007) Introduction to business data mining. McGraw-Hill/Irwin, New York Olson DL, Shi Y (2007) Introduction to business data mining. McGraw-Hill/Irwin, New York
4.
go back to reference Tien JM (2017) Internet of things, real-time decision making, and artificial intelligence. Ann Data Sci 4(2):149–178CrossRef Tien JM (2017) Internet of things, real-time decision making, and artificial intelligence. Ann Data Sci 4(2):149–178CrossRef
5.
go back to reference Zhou Z-H (2018) A brief introduction to weakly supervised learning. Natl Sci Rev 5:44–53CrossRef Zhou Z-H (2018) A brief introduction to weakly supervised learning. Natl Sci Rev 5:44–53CrossRef
6.
go back to reference Krishnakumar A (2007) Active learning literature survey. Technical reports, University of California, Santa Cruz, p 42 Krishnakumar A (2007) Active learning literature survey. Technical reports, University of California, Santa Cruz, p 42
7.
go back to reference Xiaojin Z (2008) Semi-supervised learning literature survey. Comput Sci TR 1530:60 Xiaojin Z (2008) Semi-supervised learning literature survey. Comput Sci TR 1530:60
8.
go back to reference Dietterich TG, Lathrop RH, Lozano-Pérez T (1997) Solving the multiple instance problem with axis-parallel rectangles. Artif Intell 89:31–71CrossRef Dietterich TG, Lathrop RH, Lozano-Pérez T (1997) Solving the multiple instance problem with axis-parallel rectangles. Artif Intell 89:31–71CrossRef
9.
go back to reference Frénay B, Verleysen M (2013) Classification in the presence of label noise: a survey. IEEE Trans Neural Netw Learn Syst 25:845–869CrossRef Frénay B, Verleysen M (2013) Classification in the presence of label noise: a survey. IEEE Trans Neural Netw Learn Syst 25:845–869CrossRef
10.
go back to reference He S, Feng L, Lv F, et al (2022) Partial label learning with semantic label representations. In: Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. pp 545–553 He S, Feng L, Lv F, et al (2022) Partial label learning with semantic label representations. In: Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. pp 545–553
11.
go back to reference Zhang F, Feng L, Han B, et al (2021) Exploiting class activation value for partial-label learning. In: International conference on learning representations Zhang F, Feng L, Han B, et al (2021) Exploiting class activation value for partial-label learning. In: International conference on learning representations
12.
go back to reference Tian Y, Yu X, Fu S (2023) Partial label learning: taxonomy, analysis and outlook. Neural Netw 161:708–734CrossRef Tian Y, Yu X, Fu S (2023) Partial label learning: taxonomy, analysis and outlook. Neural Netw 161:708–734CrossRef
13.
go back to reference Zhang C, Bengio S, Hardt M et al (2021) Understanding deep learning (still) requires rethinking generalization. Commun ACM 64:107–115CrossRef Zhang C, Bengio S, Hardt M et al (2021) Understanding deep learning (still) requires rethinking generalization. Commun ACM 64:107–115CrossRef
14.
go back to reference Sain SR (1996) The nature of statistical learning theory. Technometrics 38:409CrossRef Sain SR (1996) The nature of statistical learning theory. Technometrics 38:409CrossRef
15.
go back to reference Liu T, Tao D (2015) Classification with noisy labels by importance reweighting. IEEE Trans Pattern Anal Mach Intell 38:447–461CrossRef Liu T, Tao D (2015) Classification with noisy labels by importance reweighting. IEEE Trans Pattern Anal Mach Intell 38:447–461CrossRef
16.
go back to reference Patrini G, Rozza A, Krishna Menon A, et al (2017) Making deep neural networks robust to label noise: a loss correction approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1944–1952 Patrini G, Rozza A, Krishna Menon A, et al (2017) Making deep neural networks robust to label noise: a loss correction approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1944–1952
17.
go back to reference Jiang L, Zhou Z, Leung T, et al (2018) Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In: International conference on machine learning. PMLR, pp 2304–2313 Jiang L, Zhou Z, Leung T, et al (2018) Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In: International conference on machine learning. PMLR, pp 2304–2313
18.
go back to reference Ren M, Zeng W, Yang B, Urtasun R (2018) Learning to reweight examples for robust deep learning. In: International conference on machine learning. PMLR, pp 4334–4343 Ren M, Zeng W, Yang B, Urtasun R (2018) Learning to reweight examples for robust deep learning. In: International conference on machine learning. PMLR, pp 4334–4343
19.
go back to reference Han B, Yao Q, Yu X, et al (2018) Co-teaching: robust training of deep neural networks with extremely noisy labels. In: Proceedings of the 32nd International conference on neural information processing systems. pp 8536–8546 Han B, Yao Q, Yu X, et al (2018) Co-teaching: robust training of deep neural networks with extremely noisy labels. In: Proceedings of the 32nd International conference on neural information processing systems. pp 8536–8546
20.
go back to reference Yu X, Han B, Yao J, et al (2019) How does disagreement help generalization against label corruption? In: International conference on machine learning. PMLR, pp 7164–7173 Yu X, Han B, Yao J, et al (2019) How does disagreement help generalization against label corruption? In: International conference on machine learning. PMLR, pp 7164–7173
21.
go back to reference Malach E, Shalev-Shwartz S (2017) Decoupling" when to update" from" how to update". In: Proceedings of the 31st international conference on neural information processing systems. pp 961–971 Malach E, Shalev-Shwartz S (2017) Decoupling" when to update" from" how to update". In: Proceedings of the 31st international conference on neural information processing systems. pp 961–971
22.
go back to reference Wei H, Feng L, Chen X, An B (2020) Combating noisy labels by agreement: a joint training method with co-regularization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 13726–13735 Wei H, Feng L, Chen X, An B (2020) Combating noisy labels by agreement: a joint training method with co-regularization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 13726–13735
23.
go back to reference Cour T, Sapp B, Taskar B (2011) Learning from partial labels. J Mach Learn Res 12:1501–1536 Cour T, Sapp B, Taskar B (2011) Learning from partial labels. J Mach Learn Res 12:1501–1536
24.
go back to reference Hüllermeier E, Beringer J (2006) Learning from ambiguously labeled examples. Intell Data Anal 10:419–439CrossRef Hüllermeier E, Beringer J (2006) Learning from ambiguously labeled examples. Intell Data Anal 10:419–439CrossRef
25.
go back to reference Zhang M-L, Yu F (2015) Solving the partial label learning problem: an instance-based approach. In: IJCAI. pp 4048–4054 Zhang M-L, Yu F (2015) Solving the partial label learning problem: an instance-based approach. In: IJCAI. pp 4048–4054
26.
go back to reference Jin R, Ghahramani Z (2002) Learning with multiple labels. In: Proceedings of the 15th international conference on neural information processing systems. pp 921–928 Jin R, Ghahramani Z (2002) Learning with multiple labels. In: Proceedings of the 15th international conference on neural information processing systems. pp 921–928
27.
go back to reference Liu L-P, Dietterich TG (2012) A conditional multinomial mixture model for superset label learning. In: Proceedings of the 25th international conference on neural information processing systems-Volume 1. pp 548–556 Liu L-P, Dietterich TG (2012) A conditional multinomial mixture model for superset label learning. In: Proceedings of the 25th international conference on neural information processing systems-Volume 1. pp 548–556
28.
go back to reference Yu F, Zhang M-L (2016) Maximum margin partial label learning. In: Asian conference on machine learning. PMLR, pp 96–111 Yu F, Zhang M-L (2016) Maximum margin partial label learning. In: Asian conference on machine learning. PMLR, pp 96–111
29.
go back to reference Zhang M-L, Yu F, Tang C-Z (2017) Disambiguation-free partial label learning. IEEE Trans Knowl Data Eng 29:2155–2167CrossRef Zhang M-L, Yu F, Tang C-Z (2017) Disambiguation-free partial label learning. IEEE Trans Knowl Data Eng 29:2155–2167CrossRef
30.
go back to reference Wu X, Zhang M-L (2018) Towards enabling binary decomposition for partial label learning. In: IJCAI. pp 2868–2874 Wu X, Zhang M-L (2018) Towards enabling binary decomposition for partial label learning. In: IJCAI. pp 2868–2874
31.
go back to reference Yuan B, Chen J, Zhang W, et al (2018) Iterative cross learning on noisy labels. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 757–765 Yuan B, Chen J, Zhang W, et al (2018) Iterative cross learning on noisy labels. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 757–765
33.
go back to reference Yan Y, Rosales R, Fung G et al (2014) Learning from multiple annotators with varying expertise. Mach Learn 95:291–327CrossRef Yan Y, Rosales R, Fung G et al (2014) Learning from multiple annotators with varying expertise. Mach Learn 95:291–327CrossRef
34.
go back to reference Collobert R, Sinz F, Weston J, Bottou L (2006) Trading convexity for scalability. In: Proceedings of the 23rd international conference on machine learning. pp 201–208 Collobert R, Sinz F, Weston J, Bottou L (2006) Trading convexity for scalability. In: Proceedings of the 23rd international conference on machine learning. pp 201–208
35.
go back to reference Tian Y, Mirzabagheri M, Bamakan SMH et al (2018) Ramp loss one-class support vector machine; a robust and effective approach to anomaly detection problems. Neurocomputing 310:223–235CrossRef Tian Y, Mirzabagheri M, Bamakan SMH et al (2018) Ramp loss one-class support vector machine; a robust and effective approach to anomaly detection problems. Neurocomputing 310:223–235CrossRef
36.
go back to reference Tang L, Tian Y, Li W, Pardalos PM (2021) Valley-loss regular simplex support vector machine for robust multiclass classification. Knowl Based Syst 216:106801CrossRef Tang L, Tian Y, Li W, Pardalos PM (2021) Valley-loss regular simplex support vector machine for robust multiclass classification. Knowl Based Syst 216:106801CrossRef
37.
go back to reference Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Proceedings of the eleventh annual conference on computational learning theory. pp 92–100 Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Proceedings of the eleventh annual conference on computational learning theory. pp 92–100
38.
go back to reference Sindhwani V, Niyogi P, Belkin M (2005) A co-regularization approach to semi-supervised learning with multiple views. In: Proceedings of ICML workshop on learning with multiple views. Citeseer, pp 74–79 Sindhwani V, Niyogi P, Belkin M (2005) A co-regularization approach to semi-supervised learning with multiple views. In: Proceedings of ICML workshop on learning with multiple views. Citeseer, pp 74–79
39.
go back to reference Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases 1(4) Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases 1(4)
40.
go back to reference LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86:2278–2324CrossRef LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86:2278–2324CrossRef
43.
go back to reference Zhou B, Khosla A, Lapedriza A, et al (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2921–2929 Zhou B, Khosla A, Lapedriza A, et al (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2921–2929
Metadata
Title
Partial Label Learning with Noisy Labels
Authors
Pan Zhao
Long Tang
Zhigeng Pan
Publication date
31-07-2024
Publisher
Springer Berlin Heidelberg
Published in
Annals of Data Science
Print ISSN: 2198-5804
Electronic ISSN: 2198-5812
DOI
https://doi.org/10.1007/s40745-024-00552-1

Premium Partner