Skip to main content
Erschienen in: Intelligent Service Robotics 4/2017

25.05.2017 | Original Research Paper

A closed-loop approach for tracking a humanoid robot using particle filtering and depth data

verfasst von: Pablo Arturo Martínez, Xiao Lin, Mario Castelán, Josep Casas, Gustavo Arechavaleta

Erschienen in: Intelligent Service Robotics | Ausgabe 4/2017

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Humanoid robots introduce instabilities during biped march that complicate the process of estimating their position and orientation along time. Tracking humanoid robots may be useful not only in typical applications such as navigation, but in tasks that require benchmarking the multiple processes that involve registering measures about the performance of the humanoid during walking. Small robots represent an additional challenge due to their size and mechanic limitations which may generate unstable swinging while walking. This paper presents a strategy for the active localization of a humanoid robot in environments that are monitored by external devices. The problem is faced using a particle filter method over depth images captured by an RGB-D sensor in order to effectively track the position and orientation of the robot during its march. The tracking stage is coupled with a locomotion system controlling the stepping of the robot toward a given oriented target. We present an integral communication framework between the tracking and the locomotion control of the robot based on the robot operating system, which is capable of achieving real-time locomotion tasks using a NAO humanoid robot.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Alcantarilla P, Ni K, Bergasa L, Dellaert F (2011) Visibility learning in largescale urban environment. In: IEEE international conference on robotics and automation. Shanghai, China Alcantarilla P, Ni K, Bergasa L, Dellaert F (2011) Visibility learning in largescale urban environment. In: IEEE international conference on robotics and automation. Shanghai, China
2.
Zurück zum Zitat Alcantarilla PF, Stasse O, Druon S, Bergasa LM, Dellaert F (2013) How to localize humanoids with a single camera? Auton Robot 38(1–2):47–71CrossRef Alcantarilla PF, Stasse O, Druon S, Bergasa LM, Dellaert F (2013) How to localize humanoids with a single camera? Auton Robot 38(1–2):47–71CrossRef
3.
Zurück zum Zitat Arechavaleta G, Laumond JP, Hicheur H, Berthoz A (2008) An optimality principle governing human walking. IEEE Trans Robot 24(1):5–14CrossRef Arechavaleta G, Laumond JP, Hicheur H, Berthoz A (2008) An optimality principle governing human walking. IEEE Trans Robot 24(1):5–14CrossRef
4.
Zurück zum Zitat Castelán M, Arechavaleta G (2009) Approximating the reachable space of human walking paths: a low dimensional linear approach. In: IEEE international conference on humanoid robots. Paris, France, pp 81–86 Castelán M, Arechavaleta G (2009) Approximating the reachable space of human walking paths: a low dimensional linear approach. In: IEEE international conference on humanoid robots. Paris, France, pp 81–86
5.
Zurück zum Zitat Chestnutt J, Takaoka Y, Suga K, Nishiwaki K, Kuffner J, Kagami S (2009) Biped navigation in rough environments using on-board sensing. In: IEEE international conference on intelligent robots and systems. St. Louis, MO, USA Chestnutt J, Takaoka Y, Suga K, Nishiwaki K, Kuffner J, Kagami S (2009) Biped navigation in rough environments using on-board sensing. In: IEEE international conference on intelligent robots and systems. St. Louis, MO, USA
6.
Zurück zum Zitat Clark R, Wang S, Wen H, Trigoni N, Markham A (2016) Increasing the efficiency of 6-DoF visual localization using multi-modal sensory data. In: 2016 IEEE-RAS 16th international conference on humanoid robots (humanoids). IEEE, pp 973–980 Clark R, Wang S, Wen H, Trigoni N, Markham A (2016) Increasing the efficiency of 6-DoF visual localization using multi-modal sensory data. In: 2016 IEEE-RAS 16th international conference on humanoid robots (humanoids). IEEE, pp 973–980
7.
Zurück zum Zitat Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell 25(5):564–577 Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell 25(5):564–577
8.
Zurück zum Zitat Comport AI, Marchand E, Pressigout M, Chaumette F (2006) Real-time markerless tracking for augmented reality: the virtual visual servoing framework. IEEE Trans Vis Comput Graph 12(4):615–628CrossRef Comport AI, Marchand E, Pressigout M, Chaumette F (2006) Real-time markerless tracking for augmented reality: the virtual visual servoing framework. IEEE Trans Vis Comput Graph 12(4):615–628CrossRef
9.
Zurück zum Zitat Davison A, Reid ID, Molton ND, Stasse O (2007) Monoslam: real-time single camera slam. IEEE Trans Pattern Anal Mach Intell 29(6):1052–1067CrossRef Davison A, Reid ID, Molton ND, Stasse O (2007) Monoslam: real-time single camera slam. IEEE Trans Pattern Anal Mach Intell 29(6):1052–1067CrossRef
10.
Zurück zum Zitat Escande A, Mansard N, Wieber PB (2014) Hierarchical quadratic programming: fat online humanoid-robot motion generation. Int J Robot Res 33(7):1006–1028CrossRef Escande A, Mansard N, Wieber PB (2014) Hierarchical quadratic programming: fat online humanoid-robot motion generation. Int J Robot Res 33(7):1006–1028CrossRef
11.
Zurück zum Zitat Fallon MF, Antone M, Roy N, Teller S (2014) Drift-free humanoid state estimation fusing kinematic, inertial and lidar sensing. In: 2014 14th IEEE-RAS international conference on humanoid robots (humanoids). IEEE, pp 112–119 Fallon MF, Antone M, Roy N, Teller S (2014) Drift-free humanoid state estimation fusing kinematic, inertial and lidar sensing. In: 2014 14th IEEE-RAS international conference on humanoid robots (humanoids). IEEE, pp 112–119
12.
Zurück zum Zitat Ganapathi V, Plagemann C, Koller D, Thrun S (2010) Real time motion capture using a single time-of-flight camera. In: IEEE conference on computer vision and pattern recognition, CVPR, pp 755–762 Ganapathi V, Plagemann C, Koller D, Thrun S (2010) Real time motion capture using a single time-of-flight camera. In: IEEE conference on computer vision and pattern recognition, CVPR, pp 755–762
13.
Zurück zum Zitat Ganapathi V, Plagemann C, Koller D, Thrun S (2012) Real-time human pose tracking from range data. In: Fitzgibbon A, Lazebnik S, Perona P, Sato Y, Schmid C (eds) Computer vision – ECCV 2012. Lecture Notes in Computer Science, vol 7577. Springer, Berlin, pp 738–751 Ganapathi V, Plagemann C, Koller D, Thrun S (2012) Real-time human pose tracking from range data. In: Fitzgibbon A, Lazebnik S, Perona P, Sato Y, Schmid C (eds) Computer vision – ECCV 2012. Lecture Notes in Computer Science, vol 7577. Springer, Berlin, pp 738–751
14.
Zurück zum Zitat Gordon NJ, Salmond DJ, Smith AF (1993) Novel approach to nonlinear/non-gaussian bayesian state estimation. In: IEE proceedings F (radar and signal processing), vol 140. IET, pp 107–113 Gordon NJ, Salmond DJ, Smith AF (1993) Novel approach to nonlinear/non-gaussian bayesian state estimation. In: IEE proceedings F (radar and signal processing), vol 140. IET, pp 107–113
15.
Zurück zum Zitat Gratal X, Smith C, Björkman M, Kragic D (2013) Integrating 3D features and virtual visual servoing for hand-eye and humanoid robot pose estimation. In: IEEE/RAS international conference on humanoids robots, pp 240–245 Gratal X, Smith C, Björkman M, Kragic D (2013) Integrating 3D features and virtual visual servoing for hand-eye and humanoid robot pose estimation. In: IEEE/RAS international conference on humanoids robots, pp 240–245
16.
Zurück zum Zitat Hansen DW, Hansen MS, Kirschmeyer M, Larsen R, Silvestre D (2008) Cluster tracking with time-of-flight cameras. In: IEEE computer society conference on computer vision and pattern recognition workshops, CVPRW’08, pp 1–6 Hansen DW, Hansen MS, Kirschmeyer M, Larsen R, Silvestre D (2008) Cluster tracking with time-of-flight cameras. In: IEEE computer society conference on computer vision and pattern recognition workshops, CVPRW’08, pp 1–6
17.
Zurück zum Zitat Herdt A, Diedam H, Wieber PB, Dimitrov D, Mombaur K, Diehl M (2010) Online walking motion generation with automatic footstep placement. Adv Robot 24(5–6):719–737CrossRef Herdt A, Diedam H, Wieber PB, Dimitrov D, Mombaur K, Diehl M (2010) Online walking motion generation with automatic footstep placement. Adv Robot 24(5–6):719–737CrossRef
18.
Zurück zum Zitat Hornung A, Wurm K, Bennewitz M (2010) Humanoid robot localization in complex indoor environments. In: IEEE/RSJ international conference on intelligent robots and systems. Taipei, Taiwan, pp 1690–1695 Hornung A, Wurm K, Bennewitz M (2010) Humanoid robot localization in complex indoor environments. In: IEEE/RSJ international conference on intelligent robots and systems. Taipei, Taiwan, pp 1690–1695
19.
Zurück zum Zitat Klein G, Murray D (2007) Parallel tracking and mapping for small AR workspaces. In: Proceedings 6th IEEE and ACM international symposium on mixed and augmented reality (ISMAR’07). Nara, Japan Klein G, Murray D (2007) Parallel tracking and mapping for small AR workspaces. In: Proceedings 6th IEEE and ACM international symposium on mixed and augmented reality (ISMAR’07). Nara, Japan
20.
Zurück zum Zitat Maier D, Hornung A, Bennewitz M (2012) Real-time navigation in 3D environments based on depth camera data. In: IEEE international conference on humanoid robots. Osaka, Japan, pp 692–697 Maier D, Hornung A, Bennewitz M (2012) Real-time navigation in 3D environments based on depth camera data. In: IEEE international conference on humanoid robots. Osaka, Japan, pp 692–697
21.
Zurück zum Zitat Martínez PA, Castelán M, Arechavaleta G (2016) Vision based persistent localization of a humanoid robot for locomotion tasks. Int J Appl Math Comput Sci 26(3):669MathSciNetCrossRefMATH Martínez PA, Castelán M, Arechavaleta G (2016) Vision based persistent localization of a humanoid robot for locomotion tasks. Int J Appl Math Comput Sci 26(3):669MathSciNetCrossRefMATH
22.
Zurück zum Zitat Martínez PA, Varas D, Castelán M, Camacho M, Marqués F, Arechavaleta G (2014) 3D shape reconstruction from a humanoid generated video sequence. In: 2014 14th IEEE-RAS international conference on humanoid robots (humanoids). IEEE, pp 699–706 Martínez PA, Varas D, Castelán M, Camacho M, Marqués F, Arechavaleta G (2014) 3D shape reconstruction from a humanoid generated video sequence. In: 2014 14th IEEE-RAS international conference on humanoid robots (humanoids). IEEE, pp 699–706
23.
Zurück zum Zitat Matsumoto Y, Wada T, Nishio S, Miyashita T, Hagita N (2010) Scalable and robust multi-people head tracking by combining distributed multiple sensors. J Intell Serv Robot 3(1):29–36CrossRef Matsumoto Y, Wada T, Nishio S, Miyashita T, Hagita N (2010) Scalable and robust multi-people head tracking by combining distributed multiple sensors. J Intell Serv Robot 3(1):29–36CrossRef
24.
Zurück zum Zitat Michel P, Chestnutt J, Kagami S, Nishiwaki K, Kuffner J, Kagami S (2007) GPU-accelerated real-time 3D traking for humanoid locomotion and stair climbing. In: IEEE international conference on intelligent robots and systems. San Diego, USA Michel P, Chestnutt J, Kagami S, Nishiwaki K, Kuffner J, Kagami S (2007) GPU-accelerated real-time 3D traking for humanoid locomotion and stair climbing. In: IEEE international conference on intelligent robots and systems. San Diego, USA
25.
Zurück zum Zitat Michel P, Chestnutt J, Kagami S, Nishiwaki K, Kuffner J, Kanade T (2006) Online environment reconstruction for biped navigation. In: IEEE international conference on robotics and automation. Orlando, FL, USA Michel P, Chestnutt J, Kagami S, Nishiwaki K, Kuffner J, Kanade T (2006) Online environment reconstruction for biped navigation. In: IEEE international conference on robotics and automation. Orlando, FL, USA
26.
Zurück zum Zitat Migniot C, Ababsa F (2013) 3D human tracking in a top view using depth information recorded by the xtion pro-live camera. In: Advances in visual computing. Springer, Berlin, pp 603–612 Migniot C, Ababsa F (2013) 3D human tracking in a top view using depth information recorded by the xtion pro-live camera. In: Advances in visual computing. Springer, Berlin, pp 603–612
27.
Zurück zum Zitat Nummiaro K, Koller-Meier E, Van Gool L (2002) An adaptive color-based particle filter. Image Vis Comput 21(1):99–110CrossRefMATH Nummiaro K, Koller-Meier E, Van Gool L (2002) An adaptive color-based particle filter. Image Vis Comput 21(1):99–110CrossRefMATH
28.
Zurück zum Zitat Obwald S, Hornung A, Bennewitz M (2012) Improved proposals for highly accurate localization using range and vision data. In: IEEE/RSJ international conference on intelligent robots and system. Vilamoura, Portugal Obwald S, Hornung A, Bennewitz M (2012) Improved proposals for highly accurate localization using range and vision data. In: IEEE/RSJ international conference on intelligent robots and system. Vilamoura, Portugal
29.
Zurück zum Zitat Oriolo G, Paolillo A, Rosa L, Venditelli M (2016) Humanoid odometric localization integrating kinematic, inertial and visual information. Auton Robot 40(5):867–879. doi:10.1007/s10514-015-9498-0 Oriolo G, Paolillo A, Rosa L, Venditelli M (2016) Humanoid odometric localization integrating kinematic, inertial and visual information. Auton Robot 40(5):867–879. doi:10.​1007/​s10514-015-9498-0
30.
Zurück zum Zitat Papadopoulos AV, Bascetta L, Ferretti G (2013) Generation of human walking paths. In: IEEE/RSJ international conference on intelligent robots and systems. Tokyo, Japan, pp 1676–1681 Papadopoulos AV, Bascetta L, Ferretti G (2013) Generation of human walking paths. In: IEEE/RSJ international conference on intelligent robots and systems. Tokyo, Japan, pp 1676–1681
31.
Zurück zum Zitat Poppe R (2007) Vision-based human motion analysis: an overview. Comput Vis Image Underst 108(1):4–18CrossRef Poppe R (2007) Vision-based human motion analysis: an overview. Comput Vis Image Underst 108(1):4–18CrossRef
32.
Zurück zum Zitat Puydupin-Jamin AS, Johnson M, Bretl T (2012) A convex approach to inverse optimal control and its application to modeling human locomotion. In: IEEE international conference on robotics and automation. Minnesota, USA, pp 531–536 Puydupin-Jamin AS, Johnson M, Bretl T (2012) A convex approach to inverse optimal control and its application to modeling human locomotion. In: IEEE international conference on robotics and automation. Minnesota, USA, pp 531–536
33.
Zurück zum Zitat Quigley M, Conley K, Gerkey BP, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) ROS: an open-source robot operating system. In: ICRA workshop on open source software Quigley M, Conley K, Gerkey BP, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) ROS: an open-source robot operating system. In: ICRA workshop on open source software
34.
Zurück zum Zitat Rauter M (2013) Reliable human detection and tracking in top-view depth images. In: IEEE conference on computer vision and pattern recognition workshops, CVPRW, pp 529–534 Rauter M (2013) Reliable human detection and tracking in top-view depth images. In: IEEE conference on computer vision and pattern recognition workshops, CVPRW, pp 529–534
35.
Zurück zum Zitat Siddiqui M, Liao W, Medioni GG (2009) Vision-based short range interaction between a personal service robot and a user. Intell Serv Robot 2(3):113–130CrossRef Siddiqui M, Liao W, Medioni GG (2009) Vision-based short range interaction between a personal service robot and a user. Intell Serv Robot 2(3):113–130CrossRef
36.
Zurück zum Zitat Simon D (2006) Optimal state estimation: Kalman, H infinity, and nonlinear approaches. Wiley, New YorkCrossRef Simon D (2006) Optimal state estimation: Kalman, H infinity, and nonlinear approaches. Wiley, New YorkCrossRef
37.
Zurück zum Zitat Sisbot EA, Marin-Urias LF, Alami R, Siméon T (2007) A human aware mobile robot motion planner. IEEE Trans Robot 23(5):874–883CrossRef Sisbot EA, Marin-Urias LF, Alami R, Siméon T (2007) A human aware mobile robot motion planner. IEEE Trans Robot 23(5):874–883CrossRef
38.
Zurück zum Zitat Sisbot EA, Marin-Urias LF, Broquère X, Sidobre D, Alami R (2010) Synthesizing robot motions adapted to human presence. Int J Soc Robot 2(3):329–343CrossRef Sisbot EA, Marin-Urias LF, Broquère X, Sidobre D, Alami R (2010) Synthesizing robot motions adapted to human presence. Int J Soc Robot 2(3):329–343CrossRef
39.
Zurück zum Zitat Spinello L, Arras KO (2011) People detection in RGB-D data. In: IEEE/RSJ international conference on intelligent robots and systems, pp 3838–3843 Spinello L, Arras KO (2011) People detection in RGB-D data. In: IEEE/RSJ international conference on intelligent robots and systems, pp 3838–3843
40.
Zurück zum Zitat Stasse O, Davison A, Sellaouti R, Yokoi K (2006) Real-time 3D SLAM for a humanoid robot considering pattern generator information. In: IEEE/RSJ international conference on intelligent robots and systems. Beijing, China, pp 348–355 Stasse O, Davison A, Sellaouti R, Yokoi K (2006) Real-time 3D SLAM for a humanoid robot considering pattern generator information. In: IEEE/RSJ international conference on intelligent robots and systems. Beijing, China, pp 348–355
41.
Zurück zum Zitat Stilman M, Nishiwaki K, Kagami S, Kuffner J (2006) Planning and executing navigation among movable obstacles. In: IEEE international conference on intelligent robots and systems. Beijin, China Stilman M, Nishiwaki K, Kagami S, Kuffner J (2006) Planning and executing navigation among movable obstacles. In: IEEE international conference on intelligent robots and systems. Beijin, China
42.
Zurück zum Zitat Triggs B, McLauchlan PF, Hartley RI, Fitzgibbon AW (2000) Bundle adjustment – a modern synthesis. In: Triggs B., Zisserman A., Szeliski R (eds) Vision algorithms: theory and practice. IWVA 1999. Lecture Notes in Computer Science, vol 1883. Springer, Berlin, pp 298–372 Triggs B, McLauchlan PF, Hartley RI, Fitzgibbon AW (2000) Bundle adjustment – a modern synthesis. In: Triggs B., Zisserman A., Szeliski R (eds) Vision algorithms: theory and practice. IWVA 1999. Lecture Notes in Computer Science, vol 1883. Springer, Berlin, pp 298–372
43.
Zurück zum Zitat Wurm KM, Hornung A, Bennewitz M, Stachniss C, Bur-gard W (2010) Octomap: a probabilistic, flexible, and compact 3D map representation for robotic systems. In: IEEE international conference on robotics and automation, workshop on best practice in 3D perception and modeling for mobile manipulation. Anchorage, Alaska Wurm KM, Hornung A, Bennewitz M, Stachniss C, Bur-gard W (2010) Octomap: a probabilistic, flexible, and compact 3D map representation for robotic systems. In: IEEE international conference on robotics and automation, workshop on best practice in 3D perception and modeling for mobile manipulation. Anchorage, Alaska
44.
Zurück zum Zitat Xia L, Chen CC, Aggarwal J (2011) Human detection using depth information by kinect. In: IEEE computer society conference on computer vision and pattern recognition workshops, CVPRW, pp 15–22 Xia L, Chen CC, Aggarwal J (2011) Human detection using depth information by kinect. In: IEEE computer society conference on computer vision and pattern recognition workshops, CVPRW, pp 15–22
45.
Zurück zum Zitat Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv (CSUR) 38(4):13CrossRef Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv (CSUR) 38(4):13CrossRef
Metadaten
Titel
A closed-loop approach for tracking a humanoid robot using particle filtering and depth data
verfasst von
Pablo Arturo Martínez
Xiao Lin
Mario Castelán
Josep Casas
Gustavo Arechavaleta
Publikationsdatum
25.05.2017
Verlag
Springer Berlin Heidelberg
Erschienen in
Intelligent Service Robotics / Ausgabe 4/2017
Print ISSN: 1861-2776
Elektronische ISSN: 1861-2784
DOI
https://doi.org/10.1007/s11370-017-0230-0

Weitere Artikel der Ausgabe 4/2017

Intelligent Service Robotics 4/2017 Zur Ausgabe

Neuer Inhalt