Top

Neural Computing and Applications

Published in:

03-12-2023 | Original Article

Semi-supervised object detection based on single-stage detector for thighbone fracture localization

Authors: Jinman Wei, Jinkun Yao, Guoshan Zhang, Bin Guan, Yueming Zhang, Shaoquan Wang

Published in: Neural Computing and Applications | Issue 7/2024

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The thighbone is the largest bone supporting the lower body. If the thighbone fracture is not treated in time, it will lead to lifelong inability to walk. Correct diagnosis of thighbone disease is very important in orthopedic medicine. Deep learning is promoting the development of fracture detection technology. However, the existing computer-aided diagnosis methods rely on a large number of manually labeled data, and labeling these data costs a lot of time and energy. Therefore, we develop an object detection method with limited labeled image quantity and apply it to the thighbone fracture localization. In this work, we build a semi-supervised object detection framework based on single-stage detector, which includes three modules: adaptive difficult sample oriented (ADSO) module, Fusion Box and deformable expand encoder (Dex encoder). ADSO module takes the classification score as the label reliability evaluation criterion by weighting, Fusion Box is designed to merge similar pseudo boxes into a reliable box for box regression and Dex encoder is proposed to enhance the adaptability of image augmentation. The experiment is conducted on the thighbone fracture dataset, which includes 3484 training thighbone fracture images and 358 testing thighbone fracture images. The experimental results show that the proposed method achieves the state-of-the-art AP in thighbone fracture detection at different labeled data rates, i.e., 1%, 5% and 10%. Besides, we use full data to achieve knowledge distillation, our method achieves 86.2% AP50 and 52.6% AP75. Finally, the effectiveness of our method has also been evaluated using the publicly available datasets COCO and VOC.

previous article Dynamic interactive refinement network for camouflaged object detection

next article Panini: a transformer-based grammatical error correction method for Bangla

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Jones RM, Sharma A, Hotchkiss R, Sperling JW, Lindsey RV (2020) Assessment of a deep-learning system for fracture detection in musculoskeletal radiographs. NPJ Dig Med 3(1):1–6. https://doi.org/10.1038/s41746-020-00352-wCrossRef

Georgalis GL, Scheyer TM (2022) Crushed but not lost: a colubriform snake (serpentes) from the miocene swiss molasse, identified through the use of micro-ct scanning technology. Swiss J Geosci 115(1):1–9CrossRef

Guan B, Yao J, Wang S, Zhang G, Zhang Y, Wang X, Wang M (2022) Automatic detection and localization of thighbone fractures in x-ray based on improved deep learning method. Comput Vis Image Underst 216:103345. https://doi.org/10.1016/j.cviu.2021.103345CrossRef

Hardalaç F, Uysal F, Peker O, Çiçeklidağ M, Tolunay T, Tokgöz N, Kutbay U, Demirciler B, Mert F (2022) Fracture detection in wrist x-ray images using deep learning-based object detection models. Sensors 22(3):1285. https://doi.org/10.3390/s22031285CrossRefPubMedPubMedCentralADS

Sha G, Wu J, Yu B (2020) Detection of spinal fracture lesions based on improved yolov2. In: 2020 IEEE international conference on artificial intelligence and computer applications (ICAICA), pp. 235– 238. https://doi.org/10.1109/ICAICA50127.2020.9182582. IEEE

Thian YL, Li Y, Jagmohan P, Sia D, Chan VEY, Tan RT (2019) Convolutional neural networks for automated fracture detection and localization on wrist radiographs. Radiol Artif Intell 1(1):180001. https://doi.org/10.1148/ryai.2019180001CrossRef

Wu H-Z, Yan L-F, Liu X-Q, Yu Y-Z, Geng Z-J, Wu W-J, Han C-Q, Guo Y-Q, Gao B-L (2021) The feature ambiguity mitigate operator model helps improve bone fracture detection on x-ray radiograph. Sci Rep 11(1):1–10. https://doi.org/10.1038/s41598-021-81236-1CrossRef

Lee D-H et al (2013) Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. In: workshop on challenges in representation learning, ICML vol 3, p 896

Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. Comput Sci 14(7):38–39

10.

Cai Z, Vasconcelos N (2018) Cascade r-cnn: delving into high quality object detection. In: proceedings of the IEEE conference on computer vision and pattern recognition, pp 6154– 6162. https://doi.org/10.1109/CVPR.2018.00644

11.

Cao Y, Xu J, Lin S, Wei F, Hu H (2019) Gcnet: non-local networks meet squeeze-excitation networks and beyond. In: proceedings of the IEEE/CVF international conference on computer vision workshops, pp 1971– 1980. https://doi.org/10.1109/ICCVW.2019.00246

12.

Ding X, Li Q, Cheng Y, Wang J, Bian W, Jie B (2020) Local keypoint-based faster r-cnn. Appl Intell 50(10):3007–3022. https://doi.org/10.1007/s10489-020-01665-9CrossRef

13.

Tian Z, Shen C, Chen H, He T (2019) Fcos: Fully convolutional one-stage object detection. In: proceedings of the IEEE/CVF international conference on computer vision, pp. 9627– 9636 https://doi.org/10.1109/ICCV.2019.00972

14.

Li B, Liu Y, Wang X (2019) Gradient harmonized single-stage detector. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8577– 8584. https://doi.org/10.1609/aaai.v33i01.33018577

15.

Chen Q, Wang Y, Yang T, Zhang X, Cheng J, Sun J (2021) You only look one-level feature. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13039– 13048. https://doi.org/10.1109/CVPR46437.2021.01284

16.

Zhang L, Hu Y, Chen J, Li C, Li K (2022) Mssif-net: an efficient cnn automatic detection method for freight train images. Neural Comput Appl 35(9):6767–6785. https://doi.org/10.1007/s00521-022-08035-1CrossRef

17.

Hurtik P, Molek V, Hula J, Vajgl M, Vlasanek P, Nejezchleba T (2022) Poly-yolo: higher speed, more precise detection and instance segmentation for yolov3. Neural Comput Appl 34(10):8275–8290. https://doi.org/10.1007/s00521-021-05978-9CrossRef

18.

Ren S, He K, Girshick R, Sun J (2016) Faster r-cnn: towards real-time object detection with region proposal networks. In: advances in neural information processing systems, vol. 28

19.

Xu M, Zhang Z, Hu H, Wang J, Wang L, Wei F, Bai X, Liu Z (2021) End-to-end semi-supervised object detection with soft teacher. In: proceedings of the IEEE/CVF international conference on computer vision, pp. 3060–3069. https://doi.org/10.1109/ICCV48922.2021.00305

20.

Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: Common objects in context. In: computer vision–ECCV 2014: 13th European conference, Zurich, Switzerland, pp. 740– 755. https://doi.org/10.1007/978-3-319-10602-1_48. Springer

21.

Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88:303–338CrossRef

22.

Hesamian MH, Jia W, He X, Kennedy P (2019) Deep learning techniques for medical image segmentation: achievements and challenges. J Dig Imaging 32(4):582–596. https://doi.org/10.1007/s10278-019-00227-xCrossRef

23.

Wang W, Huang W, Lu Q, Chen J, Zhang M, Qiao J, Zhang Y (2022) Attention mechanism-based deep learning method for hairline fracture detection in hand x-rays. Neural Comput Appl 34(21):18773–18785CrossRefPubMedPubMedCentral

24.

Khurana Y, Soni U (2022) Leveraging deep learning for covid-19 diagnosis through chest imaging. Neural Comput Appl 34(16):14003–14012. https://doi.org/10.1007/s00521-022-07250-0CrossRefPubMedPubMedCentral

25.

Shaik NS, Cherukuri TK (2022) Hinge attention network: a joint model for diabetic retinopathy severity grading. Appl Intell 52:15105–15121. https://doi.org/10.1007/s10489-021-03043-5.13CrossRef

26.

Fouad H, Soliman AM, Hassanein AS, Al-Feel H (2020) Prediction and diagnosis of vertebral tumors on the internet of medical things platform using geometric rough propagation neural network. Neural Comput Appl 24:1–13

27.

Zhang, X., Wang, Y., Cheng, C.-T., Lu, L., Xiao, J., Liao, C.-H., Miao, S (2020) A new window loss function for bone fracture detection and localization in x-ray images with point-based annotation. arXiv preprint arXiv:2012.04066. https://doi.org/10.48550/arXiv.2012.04066

28.

Wang Y, Zheng K, Cheng C-T, Zhou X-Y, Zheng Z, Xiao J, Lu L, Liao C-H, Miao S (2021) Knowledge distillation with adaptive asymmetric label sharpening for semi-supervised fracture detection in chest x-rays. In: international conference on information processing in medical imaging, pp 599– 610. https://doi.org/10.1007/978-3-030-78191-0_46. Springer

29.

Deng J, Xuan X, Wang W, Li Z, Yao H, Wang Z (2020) A review of research on object detection based on deep learning. J Phys Conf Ser 1684:012028CrossRef

30.

Lee H-L, Kim Y-J, Kim B-G et al (2022) A survey for 3d object detection algorithms from images. J Multim Information Syst 9(3):183–190. https://doi.org/10.33851/JMIS.2022.9.3.183CrossRefADS

31.

Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: single shot multibox detector. Europ Conf Comput Vis 14:21–37

32.

Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2117– 2125

33.

Park H-J, Choi Y-J, Lee Y-W, Kim B-G (2022) ssfpn: scale sequence (s\(\hat{~}\) 2) feature based feature pyramid network for object detection. arXiv preprint arXiv:2208.11533 (2022)

34.

Wu F, Jing X-Y, Liu Q, Wu S-S, He G-L (2017) Large-scale image recognition based on parallel kernel supervised and semi-supervised subspace learning. Neural Comput Appl 28(3):483–498. https://doi.org/10.1007/s00521-015-2081-yCrossRef

35.

Tarvainen A, Valpola H (2017) Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. Adv Neural Inform Process Syst 30:17

36.

Berthelot D, Carlini N, Cubuk ED, Kurakin A, Raffel C (2019) Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. arXiv preprint arXiv:1911.09785. https://doi.org/10.48550/arXiv.1911.09785

37.

Ma Y, Chen D, Wang T, Li G, Yan M (2022) Semi-supervised partial label learning algorithm via reliable label propagation. Appl Intell. https://doi.org/10.1007/s10489-022-04027-9CrossRef

38.

Jeong J, Lee S, Kim J, Kwak N (2019) Consistency-based semi-supervised learning for object detection. Adv Neural Inform Process Syst 32:190

39.

Zoph B, Cubuk ED, Ghiasi G, Lin T-Y, Shlens J, Le QV (2020) Learning data augmentation strategies for object detection. In: European conference on computer vision, Springer: London. pp 566– 583. https://doi.org/10.1007/978-3-030-58583-9_34

40.

Yang Q, Wei X, Wang B, Hua X-S, Zhang L (2021) Interactive self-training with mean teachers for semi-supervised object detection. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5941– 5950. https://doi.org/10.1109/CVPR46437.2021.00588

41.

Wang Z, Li Y, Guo Y, Fang L, Wang S (2021) Data-uncertainty guided multi-phase learning for semi-supervised object detection. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4568– 4577. https://doi.org/10.1109/CVPR46437.2021.00454

42.

Zhang Y, Yao X, Liu C, Chen F, Song X, Xing T, Hu R, Chai H, Xu P, Zhang G (2022) S4od: Semi-supervised learning for single-stage object detection. arXiv preprint arXiv:2204.04492. https://doi.org/10.48550/arXiv.2204.04492

43.

Sohn K, Zhang Z, Li C-L, Zhang H, Lee C-Y, Pfister T (2020) A simple semi-supervised learning framework for object detection. arXiv preprint arXiv:2005.04757. https://doi.org/10.48550/arXiv.2005.04757

44.

Liu YC, Ma CY, He Z, Kuo CW, Vajda P (2021) Unbiased teacher for semi-supervised object detection. arXiv preprint arXiv:2102.09480. https://doi.org/10.48550/arXiv.2102.09480

45.

Lin T-Y, Goyal P, Girshick R, He K (2017) Dollár P Focal loss for dense object detection. In: proceedings of the IEEE international conference on computer vision, pp. 2980– 2988. https://doi.org/10.1109/TPAMI.2018.2858826

46.

Zhou Q, Yu C, Wang Z, Qian Q, Li H (2021) Instant-teaching: An end-to-end semi-supervised object detection framework. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4081– 4090. https://doi.org/10.1109/CVPR46437.2021.00407

47.

Sohn K, Berthelot D, Carlini N, Zhang Z, Zhang H, Raffel CA, Cubuk ED, Kurakin A, Li C-L (2020) Fixmatch: simplifying semi-supervised learning with consistency and confidence. Adv Neural Inform Process Syst 33:596–608

48.

Bochkovskiy A, Wang C-Y, Liao H-YM (2020) Yolov4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934. https://doi.org/10.48550/arXiv.2004.10934

49.

Rothe R, Guillaumin M, Gool LV (2014) Non-maximum suppression for object detection by passing messages between windows. In: Asian conference on computer vision, pp. 290– 306. https://doi.org/10.1007/978-3-319-16865-4_19. Springer

50.

Zhu X, Hu H, Lin S, Dai J (2019) Deformable convnets v2: More deformable, better results. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 9300– 9308. https://doi.org/10.1109/CVPR.2019.00953

51.

Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y (2017) Deformable convolutional networks. In: proceedings of the IEEE international conference on computer vision, pp. 764– 773. https://doi.org/10.1109/ICCV.2017.89

52.

Guan B, Yao J, Zhang G, Wang X (2019) Thigh fracture detection using deep learning method based on new dilated convolutional feature pyramid network. Patt Recogn Lett 125:521–526. https://doi.org/10.1016/j.patrec.2019.06.015CrossRefADS

53.

Wang M, Yao J, Zhang G, Guan B, Wang X, Zhang Y (2021) Parallelnet: multiple backbone network for detection tasks on thigh bone fracture. Multim Syst 27(6):1091–1100. https://doi.org/10.1007/s00530-021-00783-9CrossRef

54.

Hosang J, Benenson R, Dollár P, Schiele B (2015) What makes for effective detection proposals? IEEE Trans Patt Analy Mach Intell 38(4):814–830. https://doi.org/10.1109/TPAMI.2015.2465908CrossRef

55.

He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770– 778

56.

Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248– 255. Ieee

57.

Chen K, Wang J, Pang J, Cao Y, Xiong Y, Li X, Sun S, Feng W, Liu Z, Xu J, et al (2019) Mmdetection: open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155. https://doi.org/10.48550/arXiv.1906.07155

58.

Zhu X, Cheng D, Zhang Z, Lin S, Dai J (2019) An empirical study of spatial attention mechanisms in deep networks. In: proceedings of the IEEE/CVF international conference on computer vision, pp 6688– 6697. https://doi.org/10.1109/ICCV.2019.00679

59.

Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B(2021) Swin transformer: hierarchical vision transformer using shifted windows. In: proceedings of the IEEE/CVF international conference on computer vision, pp. 10012– 10022

Title: Semi-supervised object detection based on single-stage detector for thighbone fracture localization
Authors: Jinman Wei
Jinkun Yao
Guoshan Zhang
Bin Guan
Yueming Zhang
Shaoquan Wang
Publication date: 03-12-2023
Publisher: Springer London
Published in: Neural Computing and Applications / Issue 7/2024
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-023-09277-3

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 7/2024

Multi-delayed impulsive stability for stochastic multi-link complex networks with time-varying coupling structure

Speeding up pattern matching in streaming time-series via block vector and multilevel lower bound

An entity-guided text summarization framework with relational heterogeneous graph neural network

Dynamic interactive refinement network for camouflaged object detection

Enhancement of GWO for solving numerical functions and engineering problems

A train dispatching model in case of segment blockages by integrating the prediction of delay propagation

Premium Partner