Skip to main content
Erschienen in:

31.07.2024

Partial Label Learning with Noisy Labels

verfasst von: Pan Zhao, Long Tang, Zhigeng Pan

Erschienen in: Annals of Data Science | Ausgabe 1/2025

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Partial label learning (PLL) is a particular problem setting within weakly supervised learning. In PLL, each sample corresponds to a candidate label set in which only one label is true. However, in some practical application scenarios, the emergence of label noise can make some candidate sets lose their true labels, leading to a decline in model performance. In this work, a robust training strategy for PLL, derived from the joint training with co-regularization (JoCoR), is proposed to address this issue in PLL. Specifically, the proposed approach constructs two separate PLL models and a joint loss. The joint loss consists of not only two PLL losses but also a co-regularization term measuring the disagreement of the two models. By automatically selecting samples with small joint loss and using them to update the two models, our proposed approach is able to filter more and more suspected samples with noise candidate label sets. Gradually, the robustness of the PLL models to label noise strengthens due to the reduced disagreement of the two models. Experiments are conducted on two state-of-the-art PLL models using benchmark datasets under various noise levels. The results show that the proposed method can effectively stabilize the training process and reduce the model's overfitting to noisy candidate label sets.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Shi Y (2022) Advances in big data analytics: theory, algorithm and practice. Springer, SingaporeCrossRef Shi Y (2022) Advances in big data analytics: theory, algorithm and practice. Springer, SingaporeCrossRef
2.
Zurück zum Zitat Shi Y, Tian Y, Kou G et al (2011) Optimization based data mining: theory and applications. Springer, BerlinCrossRef Shi Y, Tian Y, Kou G et al (2011) Optimization based data mining: theory and applications. Springer, BerlinCrossRef
3.
Zurück zum Zitat Olson DL, Shi Y (2007) Introduction to business data mining. McGraw-Hill/Irwin, New York Olson DL, Shi Y (2007) Introduction to business data mining. McGraw-Hill/Irwin, New York
4.
Zurück zum Zitat Tien JM (2017) Internet of things, real-time decision making, and artificial intelligence. Ann Data Sci 4(2):149–178CrossRef Tien JM (2017) Internet of things, real-time decision making, and artificial intelligence. Ann Data Sci 4(2):149–178CrossRef
5.
Zurück zum Zitat Zhou Z-H (2018) A brief introduction to weakly supervised learning. Natl Sci Rev 5:44–53CrossRef Zhou Z-H (2018) A brief introduction to weakly supervised learning. Natl Sci Rev 5:44–53CrossRef
6.
Zurück zum Zitat Krishnakumar A (2007) Active learning literature survey. Technical reports, University of California, Santa Cruz, p 42 Krishnakumar A (2007) Active learning literature survey. Technical reports, University of California, Santa Cruz, p 42
7.
Zurück zum Zitat Xiaojin Z (2008) Semi-supervised learning literature survey. Comput Sci TR 1530:60 Xiaojin Z (2008) Semi-supervised learning literature survey. Comput Sci TR 1530:60
8.
Zurück zum Zitat Dietterich TG, Lathrop RH, Lozano-Pérez T (1997) Solving the multiple instance problem with axis-parallel rectangles. Artif Intell 89:31–71CrossRef Dietterich TG, Lathrop RH, Lozano-Pérez T (1997) Solving the multiple instance problem with axis-parallel rectangles. Artif Intell 89:31–71CrossRef
9.
Zurück zum Zitat Frénay B, Verleysen M (2013) Classification in the presence of label noise: a survey. IEEE Trans Neural Netw Learn Syst 25:845–869CrossRef Frénay B, Verleysen M (2013) Classification in the presence of label noise: a survey. IEEE Trans Neural Netw Learn Syst 25:845–869CrossRef
10.
Zurück zum Zitat He S, Feng L, Lv F, et al (2022) Partial label learning with semantic label representations. In: Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. pp 545–553 He S, Feng L, Lv F, et al (2022) Partial label learning with semantic label representations. In: Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. pp 545–553
11.
Zurück zum Zitat Zhang F, Feng L, Han B, et al (2021) Exploiting class activation value for partial-label learning. In: International conference on learning representations Zhang F, Feng L, Han B, et al (2021) Exploiting class activation value for partial-label learning. In: International conference on learning representations
12.
Zurück zum Zitat Tian Y, Yu X, Fu S (2023) Partial label learning: taxonomy, analysis and outlook. Neural Netw 161:708–734CrossRef Tian Y, Yu X, Fu S (2023) Partial label learning: taxonomy, analysis and outlook. Neural Netw 161:708–734CrossRef
13.
Zurück zum Zitat Zhang C, Bengio S, Hardt M et al (2021) Understanding deep learning (still) requires rethinking generalization. Commun ACM 64:107–115CrossRef Zhang C, Bengio S, Hardt M et al (2021) Understanding deep learning (still) requires rethinking generalization. Commun ACM 64:107–115CrossRef
14.
Zurück zum Zitat Sain SR (1996) The nature of statistical learning theory. Technometrics 38:409CrossRef Sain SR (1996) The nature of statistical learning theory. Technometrics 38:409CrossRef
15.
Zurück zum Zitat Liu T, Tao D (2015) Classification with noisy labels by importance reweighting. IEEE Trans Pattern Anal Mach Intell 38:447–461CrossRef Liu T, Tao D (2015) Classification with noisy labels by importance reweighting. IEEE Trans Pattern Anal Mach Intell 38:447–461CrossRef
16.
Zurück zum Zitat Patrini G, Rozza A, Krishna Menon A, et al (2017) Making deep neural networks robust to label noise: a loss correction approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1944–1952 Patrini G, Rozza A, Krishna Menon A, et al (2017) Making deep neural networks robust to label noise: a loss correction approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1944–1952
17.
Zurück zum Zitat Jiang L, Zhou Z, Leung T, et al (2018) Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In: International conference on machine learning. PMLR, pp 2304–2313 Jiang L, Zhou Z, Leung T, et al (2018) Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In: International conference on machine learning. PMLR, pp 2304–2313
18.
Zurück zum Zitat Ren M, Zeng W, Yang B, Urtasun R (2018) Learning to reweight examples for robust deep learning. In: International conference on machine learning. PMLR, pp 4334–4343 Ren M, Zeng W, Yang B, Urtasun R (2018) Learning to reweight examples for robust deep learning. In: International conference on machine learning. PMLR, pp 4334–4343
19.
Zurück zum Zitat Han B, Yao Q, Yu X, et al (2018) Co-teaching: robust training of deep neural networks with extremely noisy labels. In: Proceedings of the 32nd International conference on neural information processing systems. pp 8536–8546 Han B, Yao Q, Yu X, et al (2018) Co-teaching: robust training of deep neural networks with extremely noisy labels. In: Proceedings of the 32nd International conference on neural information processing systems. pp 8536–8546
20.
Zurück zum Zitat Yu X, Han B, Yao J, et al (2019) How does disagreement help generalization against label corruption? In: International conference on machine learning. PMLR, pp 7164–7173 Yu X, Han B, Yao J, et al (2019) How does disagreement help generalization against label corruption? In: International conference on machine learning. PMLR, pp 7164–7173
21.
Zurück zum Zitat Malach E, Shalev-Shwartz S (2017) Decoupling" when to update" from" how to update". In: Proceedings of the 31st international conference on neural information processing systems. pp 961–971 Malach E, Shalev-Shwartz S (2017) Decoupling" when to update" from" how to update". In: Proceedings of the 31st international conference on neural information processing systems. pp 961–971
22.
Zurück zum Zitat Wei H, Feng L, Chen X, An B (2020) Combating noisy labels by agreement: a joint training method with co-regularization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 13726–13735 Wei H, Feng L, Chen X, An B (2020) Combating noisy labels by agreement: a joint training method with co-regularization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 13726–13735
23.
Zurück zum Zitat Cour T, Sapp B, Taskar B (2011) Learning from partial labels. J Mach Learn Res 12:1501–1536 Cour T, Sapp B, Taskar B (2011) Learning from partial labels. J Mach Learn Res 12:1501–1536
24.
Zurück zum Zitat Hüllermeier E, Beringer J (2006) Learning from ambiguously labeled examples. Intell Data Anal 10:419–439CrossRef Hüllermeier E, Beringer J (2006) Learning from ambiguously labeled examples. Intell Data Anal 10:419–439CrossRef
25.
Zurück zum Zitat Zhang M-L, Yu F (2015) Solving the partial label learning problem: an instance-based approach. In: IJCAI. pp 4048–4054 Zhang M-L, Yu F (2015) Solving the partial label learning problem: an instance-based approach. In: IJCAI. pp 4048–4054
26.
Zurück zum Zitat Jin R, Ghahramani Z (2002) Learning with multiple labels. In: Proceedings of the 15th international conference on neural information processing systems. pp 921–928 Jin R, Ghahramani Z (2002) Learning with multiple labels. In: Proceedings of the 15th international conference on neural information processing systems. pp 921–928
27.
Zurück zum Zitat Liu L-P, Dietterich TG (2012) A conditional multinomial mixture model for superset label learning. In: Proceedings of the 25th international conference on neural information processing systems-Volume 1. pp 548–556 Liu L-P, Dietterich TG (2012) A conditional multinomial mixture model for superset label learning. In: Proceedings of the 25th international conference on neural information processing systems-Volume 1. pp 548–556
28.
Zurück zum Zitat Yu F, Zhang M-L (2016) Maximum margin partial label learning. In: Asian conference on machine learning. PMLR, pp 96–111 Yu F, Zhang M-L (2016) Maximum margin partial label learning. In: Asian conference on machine learning. PMLR, pp 96–111
29.
Zurück zum Zitat Zhang M-L, Yu F, Tang C-Z (2017) Disambiguation-free partial label learning. IEEE Trans Knowl Data Eng 29:2155–2167CrossRef Zhang M-L, Yu F, Tang C-Z (2017) Disambiguation-free partial label learning. IEEE Trans Knowl Data Eng 29:2155–2167CrossRef
30.
Zurück zum Zitat Wu X, Zhang M-L (2018) Towards enabling binary decomposition for partial label learning. In: IJCAI. pp 2868–2874 Wu X, Zhang M-L (2018) Towards enabling binary decomposition for partial label learning. In: IJCAI. pp 2868–2874
31.
Zurück zum Zitat Yuan B, Chen J, Zhang W, et al (2018) Iterative cross learning on noisy labels. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 757–765 Yuan B, Chen J, Zhang W, et al (2018) Iterative cross learning on noisy labels. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 757–765
33.
Zurück zum Zitat Yan Y, Rosales R, Fung G et al (2014) Learning from multiple annotators with varying expertise. Mach Learn 95:291–327CrossRef Yan Y, Rosales R, Fung G et al (2014) Learning from multiple annotators with varying expertise. Mach Learn 95:291–327CrossRef
34.
Zurück zum Zitat Collobert R, Sinz F, Weston J, Bottou L (2006) Trading convexity for scalability. In: Proceedings of the 23rd international conference on machine learning. pp 201–208 Collobert R, Sinz F, Weston J, Bottou L (2006) Trading convexity for scalability. In: Proceedings of the 23rd international conference on machine learning. pp 201–208
35.
Zurück zum Zitat Tian Y, Mirzabagheri M, Bamakan SMH et al (2018) Ramp loss one-class support vector machine; a robust and effective approach to anomaly detection problems. Neurocomputing 310:223–235CrossRef Tian Y, Mirzabagheri M, Bamakan SMH et al (2018) Ramp loss one-class support vector machine; a robust and effective approach to anomaly detection problems. Neurocomputing 310:223–235CrossRef
36.
Zurück zum Zitat Tang L, Tian Y, Li W, Pardalos PM (2021) Valley-loss regular simplex support vector machine for robust multiclass classification. Knowl Based Syst 216:106801CrossRef Tang L, Tian Y, Li W, Pardalos PM (2021) Valley-loss regular simplex support vector machine for robust multiclass classification. Knowl Based Syst 216:106801CrossRef
37.
Zurück zum Zitat Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Proceedings of the eleventh annual conference on computational learning theory. pp 92–100 Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Proceedings of the eleventh annual conference on computational learning theory. pp 92–100
38.
Zurück zum Zitat Sindhwani V, Niyogi P, Belkin M (2005) A co-regularization approach to semi-supervised learning with multiple views. In: Proceedings of ICML workshop on learning with multiple views. Citeseer, pp 74–79 Sindhwani V, Niyogi P, Belkin M (2005) A co-regularization approach to semi-supervised learning with multiple views. In: Proceedings of ICML workshop on learning with multiple views. Citeseer, pp 74–79
39.
Zurück zum Zitat Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases 1(4) Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases 1(4)
40.
Zurück zum Zitat LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86:2278–2324CrossRef LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86:2278–2324CrossRef
43.
Zurück zum Zitat Zhou B, Khosla A, Lapedriza A, et al (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2921–2929 Zhou B, Khosla A, Lapedriza A, et al (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2921–2929
Metadaten
Titel
Partial Label Learning with Noisy Labels
verfasst von
Pan Zhao
Long Tang
Zhigeng Pan
Publikationsdatum
31.07.2024
Verlag
Springer Berlin Heidelberg
Erschienen in
Annals of Data Science / Ausgabe 1/2025
Print ISSN: 2198-5804
Elektronische ISSN: 2198-5812
DOI
https://doi.org/10.1007/s40745-024-00552-1