Skip to main content
Erschienen in: Neural Processing Letters 6/2022

30.05.2022

Weakly Supervised Object Detection Based on Active Learning

verfasst von: Xiao Wang, Xiang Xiang, Baochang Zhang, Xuhui Liu, Jianying Zheng, QingLei Hu

Erschienen in: Neural Processing Letters | Ausgabe 6/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Weakly supervised object detection which reduces the need for strong supersivison during training has recently made significant achievements. However, it remains a challenging issue due to the time-consuming and labor-intensive problems in application. To further reduce the label cost, we introduce a new fusion method of weakly supervised learning and active learning in a unied framework for object detection. Weakly supervised learning based on min-entropy latent model is used to weaken the labels by image-label, while active learning is used to reduce the quantity of labeled images. The fusion method proposed can effectively reduce the dependency of object detection on manual annotation. In this paper, we introduce three strategies of active learning, including least confidence sampling, margining sampling and weighted classification sampling. To validate the effectiveness of each strategy and different sample compositions in weakly supervised learning object detection, we conducted lots of experiments. Extensive experiments show that the combination of image-level labeling and active learning can achieve comparable results with the previous state-of-the-art methods with much lower label cost.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
2.
3.
4.
Zurück zum Zitat Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection, pp 779–788 Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection, pp 779–788
5.
Zurück zum Zitat Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: IEEE conference on computer vision and pattern recognition, pp 7263–7271 Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: IEEE conference on computer vision and pattern recognition, pp 7263–7271
6.
7.
Zurück zum Zitat Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587 Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587
8.
Zurück zum Zitat Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards realtime object detection with region proposal networks. Adv Neural Inf Process Syst 28:91–99 Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards realtime object detection with region proposal networks. Adv Neural Inf Process Syst 28:91–99
9.
Zurück zum Zitat Andrews S, Tsochantaridis I, Hofmann T (2002) Support vector machines for multiple-instance learning. 15:7 Andrews S, Tsochantaridis I, Hofmann T (2002) Support vector machines for multiple-instance learning. 15:7
10.
Zurück zum Zitat Gokberk Cinbis R, Verbeek J, Schmid C (2014) Multi-fold mil training for weakly supervised object localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2409–2416 Gokberk Cinbis R, Verbeek J, Schmid C (2014) Multi-fold mil training for weakly supervised object localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2409–2416
11.
Zurück zum Zitat Pandey M, Lazebnik S (2011) Scene recognition and weakly supervised object localization with deformable part-based models. In: International conference on computer vision, pp 1307–1314 Pandey M, Lazebnik S (2011) Scene recognition and weakly supervised object localization with deformable part-based models. In: International conference on computer vision, pp 1307–1314
12.
Zurück zum Zitat Diba A, Sharma V, Pazandeh A, Pirsiavash H, Van Gool L (2017) Weakly supervised cascaded convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 914–922 Diba A, Sharma V, Pazandeh A, Pirsiavash H, Van Gool L (2017) Weakly supervised cascaded convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 914–922
13.
Zurück zum Zitat Li Y, Liu L, Shen C, van den Hengel A (2016) Image co-localization by mimicking a good detector’s confidence score distribution. In: European Conference on Computer Vision, pp. 19–34 Li Y, Liu L, Shen C, van den Hengel A (2016) Image co-localization by mimicking a good detector’s confidence score distribution. In: European Conference on Computer Vision, pp. 19–34
14.
Zurück zum Zitat Wang X, Zhu Z, Yao C, Bai X (2015) Relaxed multiple-instance svm with application to object discovery. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 1224–1232 Wang X, Zhu Z, Yao C, Bai X (2015) Relaxed multiple-instance svm with application to object discovery. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 1224–1232
15.
Zurück zum Zitat Cinbis RG, Verbeek J, Schmid C (2016) Weakly supervised object localization with multi-fold multiple instance learning. IEEE Trans Pattern Anal Mach Intell 39(1):189–203CrossRef Cinbis RG, Verbeek J, Schmid C (2016) Weakly supervised object localization with multi-fold multiple instance learning. IEEE Trans Pattern Anal Mach Intell 39(1):189–203CrossRef
16.
Zurück zum Zitat Bilen H, Pedersoli M, Tuytelaars T (2014) Weakly supervised object detection with posterior regularization. In: Proceedings BMVC 2014, pp 1–12 Bilen H, Pedersoli M, Tuytelaars T (2014) Weakly supervised object detection with posterior regularization. In: Proceedings BMVC 2014, pp 1–12
17.
Zurück zum Zitat Bilen H, Pedersoli M, Tuytelaars T (2015) Weakly supervised object detection with convex clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1081–1089 Bilen H, Pedersoli M, Tuytelaars T (2015) Weakly supervised object detection with convex clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1081–1089
18.
Zurück zum Zitat Wang C, Ren W, Huang K, Tan T (2014) Weakly supervised object localization with latent category learning. In: Computer vision—ECCV 2014. Springer, Cham, pp 431–445 Wang C, Ren W, Huang K, Tan T (2014) Weakly supervised object localization with latent category learning. In: Computer vision—ECCV 2014. Springer, Cham, pp 431–445
19.
Zurück zum Zitat Song HO, Lee YJ, Jegelka S, Darrell T (2014) Weakly-supervised discovery of visual pattern configurations. Adv Neural Inf Process Syst 2:1637–1645 Song HO, Lee YJ, Jegelka S, Darrell T (2014) Weakly-supervised discovery of visual pattern configurations. Adv Neural Inf Process Syst 2:1637–1645
20.
Zurück zum Zitat Bilen H, Vedaldi A (2016) Weakly supervised deep detection networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2846–2854 Bilen H, Vedaldi A (2016) Weakly supervised deep detection networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2846–2854
21.
Zurück zum Zitat Wan F, Wei P, Jiao J, Han Z, Ye Q (2018) Min-entropy latent model for weakly supervised object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1297–1306 Wan F, Wei P, Jiao J, Han Z, Ye Q (2018) Min-entropy latent model for weakly supervised object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1297–1306
22.
Zurück zum Zitat Aghdam H.H, Gonzalez-Garcia A, Joost V, López A (2019) Active learning for deep detection neural networks. In: International conference on computer vision, pp 3671–3679 Aghdam H.H, Gonzalez-Garcia A, Joost V, López A (2019) Active learning for deep detection neural networks. In: International conference on computer vision, pp 3671–3679
23.
Zurück zum Zitat Zhang B, Li L, Yang S, Wang S, Zha Z-J, Huang Q (2020) State-relabeling adversarial active learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8756–8765 Zhang B, Li L, Yang S, Wang S, Zha Z-J, Huang Q (2020) State-relabeling adversarial active learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8756–8765
24.
Zurück zum Zitat Yoo D, Kweon IS (2019) Learning loss for active learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 93–102 Yoo D, Kweon IS (2019) Learning loss for active learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 93–102
25.
Zurück zum Zitat Gilad-Bachrach R, Navot A, Tishby N (2006) Query by committee made real. In: Advances in neural information processing systems, pp 443–450 Gilad-Bachrach R, Navot A, Tishby N (2006) Query by committee made real. In: Advances in neural information processing systems, pp 443–450
26.
Zurück zum Zitat Lewis DD, Catlett J (1994) Heterogeneous uncertainty sampling for supervised learning. In: Machine learning proceedings 1994, pp 148–156 Lewis DD, Catlett J (1994) Heterogeneous uncertainty sampling for supervised learning. In: Machine learning proceedings 1994, pp 148–156
27.
Zurück zum Zitat Culotta A, McCallum A (2005) Reducing labeling effort for structured prediction tasks. In: AAAI conference on artificial intelligence, pp 2921–2929 Culotta A, McCallum A (2005) Reducing labeling effort for structured prediction tasks. In: AAAI conference on artificial intelligence, pp 2921–2929
28.
Zurück zum Zitat Roth D, Small K (2006) Margin-based active learning for structured output spaces. In: European conference on machine learning. Springer, pp 413–424 Roth D, Small K (2006) Margin-based active learning for structured output spaces. In: European conference on machine learning. Springer, pp 413–424
29.
Zurück zum Zitat Joshi AJ, Porikli F, Papanikolopoulos N (2009) Multi-class active learning for image classification. In: IEEE conference on computer vision and pattern recognition. IEEE, pp 2372–2379 Joshi AJ, Porikli F, Papanikolopoulos N (2009) Multi-class active learning for image classification. In: IEEE conference on computer vision and pattern recognition. IEEE, pp 2372–2379
31.
Zurück zum Zitat Goodfellow IJ, Pouget J, Mirza M, Xu B, Warde D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial networks. Adv Neural Inf Process Syst 3:2672–2680 Goodfellow IJ, Pouget J, Mirza M, Xu B, Warde D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial networks. Adv Neural Inf Process Syst 3:2672–2680
32.
Zurück zum Zitat Saatci Y, Wilson A (2017) Bayesian gans. In: Advances in neural information processing systems, pp 3624–3633 Saatci Y, Wilson A (2017) Bayesian gans. In: Advances in neural information processing systems, pp 3624–3633
33.
Zurück zum Zitat Karras T, Aila T, Laine S, Lehtinen J (2018) Progressive growing of gans for improved quality, stability, and variation. arXiv:1710.10196 Karras T, Aila T, Laine S, Lehtinen J (2018) Progressive growing of gans for improved quality, stability, and variation. arXiv:​1710.​10196
34.
Zurück zum Zitat Jie Z, Wei Y, Jin X, Feng J, Liu W (2017) Deep self-taught learning for weakly supervised object localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1377–1385 Jie Z, Wei Y, Jin X, Feng J, Liu W (2017) Deep self-taught learning for weakly supervised object localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1377–1385
35.
Zurück zum Zitat Dollar P, Appel R, Belongie S, Perona P (2014) Fast feature pyramids for object detection. IEEE Trans Pattern Anal Mach Intell 36(8):1532–1545CrossRef Dollar P, Appel R, Belongie S, Perona P (2014) Fast feature pyramids for object detection. IEEE Trans Pattern Anal Mach Intell 36(8):1532–1545CrossRef
36.
Zurück zum Zitat Zhang X, Feng J, Xiong H, Tian Q (2018) Zigzag learning for weakly supervised object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4262–4270 Zhang X, Feng J, Xiong H, Tian Q (2018) Zigzag learning for weakly supervised object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4262–4270
37.
Zurück zum Zitat Inoue N, Furuta R, Yamasaki T, Aizawa K (2018) Cross-domain weakly-supervised object detection through progressive domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5001–5009 Inoue N, Furuta R, Yamasaki T, Aizawa K (2018) Cross-domain weakly-supervised object detection through progressive domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5001–5009
38.
Zurück zum Zitat Alexe B, Deselaers T, Ferrari V (2010) Classcut for unsupervised class segmentation. In: European conference on computer vision, pp 380–393 Alexe B, Deselaers T, Ferrari V (2010) Classcut for unsupervised class segmentation. In: European conference on computer vision, pp 380–393
39.
Zurück zum Zitat Vicente S, Rother C, Kolmogorov V (2011) Object cosegmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2217–2224 Vicente S, Rother C, Kolmogorov V (2011) Object cosegmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2217–2224
40.
Zurück zum Zitat Zhou Z-H (2018) A brief introduction to weakly supervised learning. Natl Sci Rev 5(1):44–53CrossRef Zhou Z-H (2018) A brief introduction to weakly supervised learning. Natl Sci Rev 5(1):44–53CrossRef
41.
Zurück zum Zitat Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D (2009) Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645CrossRef Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D (2009) Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645CrossRef
42.
Zurück zum Zitat Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2929 Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2929
43.
Zurück zum Zitat Luo W, Schwing A, Urtasun R (2013) Latent structured active learning. Adv Neural Inf Process Syst 26:728–736 Luo W, Schwing A, Urtasun R (2013) Latent structured active learning. Adv Neural Inf Process Syst 26:728–736
44.
Zurück zum Zitat Settles B, Craven M (2008) An analysis of active learning strategies for sequence labeling tasks. In: Proceedings of the 2008 conference on empirical methods in natural language processing, pp 1070–1079 Settles B, Craven M (2008) An analysis of active learning strategies for sequence labeling tasks. In: Proceedings of the 2008 conference on empirical methods in natural language processing, pp 1070–1079
45.
Zurück zum Zitat Li X, Guo Y (2013) Adaptive active learning for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 859–866 Li X, Guo Y (2013) Adaptive active learning for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 859–866
46.
Zurück zum Zitat Zhang D, Lin L, Li Y, Zhang R, Wang K (2017) Cost-effective active learning for deep image classification. IEEE Trans Circuits Syst Video Technol 27(12):2591–2600CrossRef Zhang D, Lin L, Li Y, Zhang R, Wang K (2017) Cost-effective active learning for deep image classification. IEEE Trans Circuits Syst Video Technol 27(12):2591–2600CrossRef
47.
Zurück zum Zitat Zhou Z, Shin J, Zhang L, Gurudu S, Gotway M, Liang J (2017) Fine-tuning convolutional neural networks for biomedical image analysis: actively and incrementally. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7340–7351 Zhou Z, Shin J, Zhang L, Gurudu S, Gotway M, Liang J (2017) Fine-tuning convolutional neural networks for biomedical image analysis: actively and incrementally. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7340–7351
48.
Zurück zum Zitat Ranganathan H, Venkateswara H, Chakraborty S, Panchanathan S (2017) Deep active learning for image classification. In: IEEE international conference on image processing, pp 3934–3938 Ranganathan H, Venkateswara H, Chakraborty S, Panchanathan S (2017) Deep active learning for image classification. In: IEEE international conference on image processing, pp 3934–3938
49.
Zurück zum Zitat Li M, Liu X, van de Weijer J, Raducanu B (2021) Learning to rank for active learning: a listwise approach. In: 2020 25th international conference on pattern recognition, pp 5587–5594 Li M, Liu X, van de Weijer J, Raducanu B (2021) Learning to rank for active learning: a listwise approach. In: 2020 25th international conference on pattern recognition, pp 5587–5594
50.
Zurück zum Zitat Yuan T, Wan F, Fu M, Liu J, Xu S, Ji X, Ye Q (2021) Multiple instance active learning for object detection. arXiv:2104.02324 [cs.CV] Yuan T, Wan F, Fu M, Liu J, Xu S, Ji X, Ye Q (2021) Multiple instance active learning for object detection. arXiv:​2104.​02324 [cs.CV]
51.
Zurück zum Zitat Uijlings JR, Van De Sande KE, Gevers T, Smeulders AW (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171CrossRef Uijlings JR, Van De Sande KE, Gevers T, Smeulders AW (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171CrossRef
52.
Zurück zum Zitat Zitnick CL, Dollár P (2014) Edge boxes: locating object proposals from edges. In: European conference on computer vision, pp 391–405 Zitnick CL, Dollár P (2014) Edge boxes: locating object proposals from edges. In: European conference on computer vision, pp 391–405
Metadaten
Titel
Weakly Supervised Object Detection Based on Active Learning
verfasst von
Xiao Wang
Xiang Xiang
Baochang Zhang
Xuhui Liu
Jianying Zheng
QingLei Hu
Publikationsdatum
30.05.2022
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 6/2022
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-022-10855-0

Weitere Artikel der Ausgabe 6/2022

Neural Processing Letters 6/2022 Zur Ausgabe

Neuer Inhalt