Skip to main content
Erschienen in: Intelligent Service Robotics 4/2023

21.06.2023 | Original Research Paper

A novel multiple targets detection method for service robots in the indoor complex scenes

verfasst von: Zongmin Liu, Jirui Wang, Jie Li, Pengda Liu, Kai Ren

Erschienen in: Intelligent Service Robotics | Ausgabe 4/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

With the expansive aging of global population, service robot with living assistance applied in indoor scenes will serve as a crucial role in the field of elderly care in the future. Service robots need to detect multiple targets when completing auxiliary tasks. However, indoor scenes are usually complex and there are many types of interference factors, leading to great challenges in the multiple targets detection. To overcome this technical difficulty, a novel improved Mask RCNN method for multiple targets detection in the indoor complex scenes is proposed in this paper. The improved model utilizes Mask RCNN as the network framework. On this basis, Convolutional Block Attention Module (CBAM) with channel mechanism and space mechanism is integrated, and the influence of different background, distance, angle and interference factors is comprehensively considered. Meanwhile, in order to evaluate the detection and identification effects of the established model, a comprehensive evaluation system based on loss function and Mean Average Precision (mAP) is established. For verification, experiments on the detection and identification effects under different distances, backgrounds, postures and interference factors were conducted. The results demonstrated that designed model improves the accuracy to a higher level and has a better anti-interference ability than other methods while the detection speed was nearly the same. This research will promote the application of intelligent service robots in the field of perception and target grasp.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
2.
Zurück zum Zitat Wilson G, Pereyda C, Raghunath N, Cruz G, Goel S (2019) Robot-enabled support of daily activities in smart home environments. Cognit Syst Res 54:258–272CrossRef Wilson G, Pereyda C, Raghunath N, Cruz G, Goel S (2019) Robot-enabled support of daily activities in smart home environments. Cognit Syst Res 54:258–272CrossRef
3.
Zurück zum Zitat Liu S, Tian G, Zhang Y, Zhang M, Liu S (2022) Service planning oriented efficient object search: a knowledge-based framework for home service robot. Expert Syst Appl 187:115853CrossRef Liu S, Tian G, Zhang Y, Zhang M, Liu S (2022) Service planning oriented efficient object search: a knowledge-based framework for home service robot. Expert Syst Appl 187:115853CrossRef
4.
Zurück zum Zitat Kim M, Suh I (2019) Active object search in an unknown large-scale environment using commonsense knowledge and spatial relations. Intel Serv Robot 12(4):371–380CrossRef Kim M, Suh I (2019) Active object search in an unknown large-scale environment using commonsense knowledge and spatial relations. Intel Serv Robot 12(4):371–380CrossRef
5.
Zurück zum Zitat Li X, Zhong J, Kamruzzaman M (2021) Complicated robot activity recognition by quality-aware deep reinforcement learning. Future Gener Comput Syst 117:480–485CrossRef Li X, Zhong J, Kamruzzaman M (2021) Complicated robot activity recognition by quality-aware deep reinforcement learning. Future Gener Comput Syst 117:480–485CrossRef
6.
Zurück zum Zitat Zhang M, Tian G, Zhang Y, Duan P (2021) Service skill improvement for home robots: autonomous generation of action sequence based on reinforcement learning. Knowl-Based Syst 212:106605CrossRef Zhang M, Tian G, Zhang Y, Duan P (2021) Service skill improvement for home robots: autonomous generation of action sequence based on reinforcement learning. Knowl-Based Syst 212:106605CrossRef
7.
Zurück zum Zitat Wen S, Liu X, Wang Z, Zhang H, Zhang Z, Tian W (2022) An improved multi-object classification algorithm for visual SLAM under dynamic environment. Intell Serv Robot 15:39–55CrossRef Wen S, Liu X, Wang Z, Zhang H, Zhang Z, Tian W (2022) An improved multi-object classification algorithm for visual SLAM under dynamic environment. Intell Serv Robot 15:39–55CrossRef
8.
Zurück zum Zitat Kasaei S, Ghorbani M, Schilperoort J, Rest W (2021) Investigating the importance of shape features, color constancy, color spaces, and similarity measures in open-ended 3D object recognition. Intell Serv Robot 14:329–344CrossRef Kasaei S, Ghorbani M, Schilperoort J, Rest W (2021) Investigating the importance of shape features, color constancy, color spaces, and similarity measures in open-ended 3D object recognition. Intell Serv Robot 14:329–344CrossRef
9.
Zurück zum Zitat Cheong S, Cho B, Lee J, Lee J, Kim D, Nam C, Kim C, Park S (2021) Obstacle rearrangement for robotic manipulation in clutter using a deep Q-network. Intell Serv Robot 14:549–561CrossRef Cheong S, Cho B, Lee J, Lee J, Kim D, Nam C, Kim C, Park S (2021) Obstacle rearrangement for robotic manipulation in clutter using a deep Q-network. Intell Serv Robot 14:549–561CrossRef
10.
Zurück zum Zitat Ercolano G, Rossi S (2021) Combining CNN and LSTM for activity of daily living recognition with a 3D matrix skeleton representation. Intell Serv Robot 14:175–185CrossRef Ercolano G, Rossi S (2021) Combining CNN and LSTM for activity of daily living recognition with a 3D matrix skeleton representation. Intell Serv Robot 14:175–185CrossRef
12.
Zurück zum Zitat Tan M, Pang R, Le Q (2019) Efficientdet: scalable and efficient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10778–10787 Tan M, Pang R, Le Q (2019) Efficientdet: scalable and efficient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10778–10787
14.
Zurück zum Zitat Dai J, Li Y, He K, Sun J (2016) R-FCN: object detection via region-based fully convolutional networks. In: Proceedings of the 30th international conference on neural information processing systems, pp 379–387 Dai J, Li Y, He K, Sun J (2016) R-FCN: object detection via region-based fully convolutional networks. In: Proceedings of the 30th international conference on neural information processing systems, pp 379–387
15.
Zurück zum Zitat Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149CrossRef Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149CrossRef
16.
Zurück zum Zitat Cai Z, Vasconcelos N (2017) Cascade R-CNN: delving into high quality object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6154–6162 Cai Z, Vasconcelos N (2017) Cascade R-CNN: delving into high quality object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6154–6162
17.
Zurück zum Zitat Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587 Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587
18.
Zurück zum Zitat He K, Gkioxari G, Dollár P, Girshick R (2017) Mask R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988 He K, Gkioxari G, Dollár P, Girshick R (2017) Mask R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988
19.
Zurück zum Zitat Wang D, He D (2022) Fusion of Mask RCNN and attention mechanism for instance segmentation of apples under complex background. Comput Electron Agric 196:106864CrossRef Wang D, He D (2022) Fusion of Mask RCNN and attention mechanism for instance segmentation of apples under complex background. Comput Electron Agric 196:106864CrossRef
20.
Zurück zum Zitat Chu P, Li Z, Lammers K, Lu R, Liu X (2021) Deep learning-based apple detection using a suppression Mask R-CNN. Pattern Recogn Lett 147:206–211CrossRef Chu P, Li Z, Lammers K, Lu R, Liu X (2021) Deep learning-based apple detection using a suppression Mask R-CNN. Pattern Recogn Lett 147:206–211CrossRef
21.
Zurück zum Zitat Wang Q, Fan Z, Sheng W, Zhang S, Liu M (2019) Finding misplaced items using a mobile robot in a smart home environment. Front Inf Technol Electron Eng 20(8):1036–1048CrossRef Wang Q, Fan Z, Sheng W, Zhang S, Liu M (2019) Finding misplaced items using a mobile robot in a smart home environment. Front Inf Technol Electron Eng 20(8):1036–1048CrossRef
22.
Zurück zum Zitat Singh K, Kapoor D, Thakur K, Sharma A, Gao X (2022) Computer-vision based object detection and recognition for service robot in indoor environment. Comput Mater Continua 72(1):197–213CrossRef Singh K, Kapoor D, Thakur K, Sharma A, Gao X (2022) Computer-vision based object detection and recognition for service robot in indoor environment. Comput Mater Continua 72(1):197–213CrossRef
23.
Zurück zum Zitat Hameed K, Chai D, Rassau A (2022) Score-based mask edge improvement of Mask-RCNN for segmentation of fruit and vegetables. Expert Syst Appl 190:116205CrossRef Hameed K, Chai D, Rassau A (2022) Score-based mask edge improvement of Mask-RCNN for segmentation of fruit and vegetables. Expert Syst Appl 190:116205CrossRef
24.
Zurück zum Zitat Liu S, Tian G, Cui Y, Shao X (2022) A deep Q-learning network based active object detection model with a novel training algorithm for service robots. Front Inf Technol Electron Eng 23:1673–1683CrossRef Liu S, Tian G, Cui Y, Shao X (2022) A deep Q-learning network based active object detection model with a novel training algorithm for service robots. Front Inf Technol Electron Eng 23:1673–1683CrossRef
25.
Zurück zum Zitat Jaderberg M, Simonyan K, Zisserman A, Kavukcuoglu K (2015) Spatial transformer networks. Adv Neural Inf Process Syst 28:2017–2025 Jaderberg M, Simonyan K, Zisserman A, Kavukcuoglu K (2015) Spatial transformer networks. Adv Neural Inf Process Syst 28:2017–2025
26.
Zurück zum Zitat Hu J, Shen L, Albanie S, Sun G, Wu E (2017) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42:2011–2023CrossRef Hu J, Shen L, Albanie S, Sun G, Wu E (2017) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42:2011–2023CrossRef
27.
Zurück zum Zitat Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q (2020) ECA-Net: efficient channel attention for deep convolutional neural networks. In: Proceedings of the IEEE/ CVF conference on computer vision and pattern recognition, pp 11531–11539 Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q (2020) ECA-Net: efficient channel attention for deep convolutional neural networks. In: Proceedings of the IEEE/ CVF conference on computer vision and pattern recognition, pp 11531–11539
28.
Zurück zum Zitat Woo S, Park J, Lee J, Kweon I (2018) CBAM: convolutional block attention module. In: Proceedings of the European conference on computer vision, pp 3–19 Woo S, Park J, Lee J, Kweon I (2018) CBAM: convolutional block attention module. In: Proceedings of the European conference on computer vision, pp 3–19
29.
Zurück zum Zitat Lin, T Y, Maire M, Belongie S et al (2014) Microsoft COCO: common objects in context. In: Proceedings of the European conference on computer vision, pp 740–755 Lin, T Y, Maire M, Belongie S et al (2014) Microsoft COCO: common objects in context. In: Proceedings of the European conference on computer vision, pp 740–755
30.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
31.
Zurück zum Zitat Lin T, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 936–944 Lin T, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 936–944
32.
Zurück zum Zitat Paszke A, Gross S, Massa F et al (2019) Pytorch: an imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:8026–8037 Paszke A, Gross S, Massa F et al (2019) Pytorch: an imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:8026–8037
Metadaten
Titel
A novel multiple targets detection method for service robots in the indoor complex scenes
verfasst von
Zongmin Liu
Jirui Wang
Jie Li
Pengda Liu
Kai Ren
Publikationsdatum
21.06.2023
Verlag
Springer Berlin Heidelberg
Erschienen in
Intelligent Service Robotics / Ausgabe 4/2023
Print ISSN: 1861-2776
Elektronische ISSN: 1861-2784
DOI
https://doi.org/10.1007/s11370-023-00471-9

Weitere Artikel der Ausgabe 4/2023

Intelligent Service Robotics 4/2023 Zur Ausgabe

Neuer Inhalt