Skip to main content
Erschienen in: Artificial Life and Robotics 1/2022

29.11.2021 | Original Article

Using sim-to-real transfer learning to close gaps between simulation and real environments through reinforcement learning

verfasst von: Yuto Ushida, Hafiyanda Razan, Shunta Ishizuya, Takuto Sakuma, Shohei Kato

Erschienen in: Artificial Life and Robotics | Ausgabe 1/2022

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

We attempt to develop an autonomous mobile robot that supports workers in the warehouse to reduce their burden. The proposed robot acquires a state-action policy to circumvent obstacles and reach a destination via reinforcement learning, using a LiDAR sensor. Regarding the real-world applications of reinforcement learning, the policies previously learned under a simulation environment are generally diverted to real robot, owing to unexpected uncertainties inherent to simulation environments, such as friction and sensor noise. To address this problem, in this study, we proposed a method to improve the action control of an Omni wheel robot via transfer learning in an environment. In addition, as an experiment, we searched the route for reaching a goal in an real environment using transfer learning’s results and verified the effectiveness of the policy acquired.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Gen A (2006) Development of outdoor cleaning robot SUBARUROBOHITER RS1. J Robot Soc Japan 24(2):153–155CrossRef Gen A (2006) Development of outdoor cleaning robot SUBARUROBOHITER RS1. J Robot Soc Japan 24(2):153–155CrossRef
2.
Zurück zum Zitat Yuzi H (2006) An approach to development of a human-symbiotic robot. J Robot Soc Japan 24(3):296–299CrossRef Yuzi H (2006) An approach to development of a human-symbiotic robot. J Robot Soc Japan 24(3):296–299CrossRef
3.
Zurück zum Zitat Yoichi S et al (2006) Security services system using security robot Guardrobo. J Robot Soc Japan 24(3):308–311CrossRef Yoichi S et al (2006) Security services system using security robot Guardrobo. J Robot Soc Japan 24(3):308–311CrossRef
4.
Zurück zum Zitat Sutton RS, Barto AG (1998) Reinforcement Learning. The MIT Press, CambridgeMATH Sutton RS, Barto AG (1998) Reinforcement Learning. The MIT Press, CambridgeMATH
5.
Zurück zum Zitat Sunayama R et al. (2007) Acquisition of collision avoidance behavior using reinforcement learning for autonomous mobile robots ROBOMEC lecture, pp.2A1-E19-1-4 Sunayama R et al. (2007) Acquisition of collision avoidance behavior using reinforcement learning for autonomous mobile robots ROBOMEC lecture, pp.2A1-E19-1-4
6.
Zurück zum Zitat Yuto U et al. (2020) Policy transfer from simulation to real world for autonomous control of an omni wheel robot, proceedings of 2020 IEEE 9th Global Conference on Consumer Electronics, pp.170-171, October 13-16 Yuto U et al. (2020) Policy transfer from simulation to real world for autonomous control of an omni wheel robot, proceedings of 2020 IEEE 9th Global Conference on Consumer Electronics, pp.170-171, October 13-16
7.
Zurück zum Zitat Watkins CJCH, Dayan P (1992) Technical note: Q-learning. Mach Learn 8:56–68 Watkins CJCH, Dayan P (1992) Technical note: Q-learning. Mach Learn 8:56–68
9.
Zurück zum Zitat Lazaric A (2012) Transfer in Reinforcement Learning: a Framework and a Survey, Reinforcement Learning-State of the art. Springer, Berlin, pp 143–173CrossRef Lazaric A (2012) Transfer in Reinforcement Learning: a Framework and a Survey, Reinforcement Learning-State of the art. Springer, Berlin, pp 143–173CrossRef
10.
Zurück zum Zitat Taylor ME, Stone P (2009) Transfer learning for reinforcement learning domains: a survey. J Mach Learn Res 10:1633–1685MathSciNetMATH Taylor ME, Stone P (2009) Transfer learning for reinforcement learning domains: a survey. J Mach Learn Res 10:1633–1685MathSciNetMATH
11.
Zurück zum Zitat Taylor ME (2009) Transfer in reinforcement learning domains. Studies in computational intelligence, vol. 216. Springer, New York Taylor ME (2009) Transfer in reinforcement learning domains. Studies in computational intelligence, vol. 216. Springer, New York
Metadaten
Titel
Using sim-to-real transfer learning to close gaps between simulation and real environments through reinforcement learning
verfasst von
Yuto Ushida
Hafiyanda Razan
Shunta Ishizuya
Takuto Sakuma
Shohei Kato
Publikationsdatum
29.11.2021
Verlag
Springer Japan
Erschienen in
Artificial Life and Robotics / Ausgabe 1/2022
Print ISSN: 1433-5298
Elektronische ISSN: 1614-7456
DOI
https://doi.org/10.1007/s10015-021-00713-y

Weitere Artikel der Ausgabe 1/2022

Artificial Life and Robotics 1/2022 Zur Ausgabe

Neuer Inhalt