Skip to main content
Top

2020 | OriginalPaper | Chapter

Social Navigation with Human Empowerment Driven Deep Reinforcement Learning

Authors : Tessa van der Heiden, Florian Mirus, Herke van Hoof

Published in: Artificial Neural Networks and Machine Learning – ICANN 2020

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Mobile robot navigation has seen extensive research in the last decades. The aspect of collaboration with robots and humans sharing workspaces will become increasingly important in the future. Therefore, the next generation of mobile robots needs to be socially-compliant to be accepted by their human collaborators. However, a formal definition of compliance is not straightforward. On the other hand, empowerment has been used by artificial agents to learn complicated and generalized actions and also has been shown to be a good model for biological behaviors. In this paper, we go beyond the approach of classical Reinforcement Learning (RL) and provide our agent with intrinsic motivation using empowerment. In contrast to self-empowerment, a robot employing our approach strives for the empowerment of people in its environment, so they are not disturbed by the robot’s presence and motion. In our experiments, we show that our approach has a positive influence on humans, as it minimizes its distance to humans and thus decreases human travel time while moving efficiently towards its own goal. An interactive user-study shows that our method is considered more social than other state-of-the-art approaches by the participants.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Aubret, A., Matignon, L., Hassas, S.: A survey on intrinsic motivation in reinforcement learning. arXiv preprint arXiv:1908.06976 (2019) Aubret, A., Matignon, L., Hassas, S.: A survey on intrinsic motivation in reinforcement learning. arXiv preprint arXiv:​1908.​06976 (2019)
2.
go back to reference Bansal, S., Tolani, V., Gupta, S., Malik, J., Tomlin, C.: Combining optimal control and learning for visual navigation in novel environments. arXiv preprint arXiv:1903.02531 (2019) Bansal, S., Tolani, V., Gupta, S., Malik, J., Tomlin, C.: Combining optimal control and learning for visual navigation in novel environments. arXiv preprint arXiv:​1903.​02531 (2019)
4.
go back to reference Chen, Y., Everett, M., Liu, M., How, J.P.: Socially aware motion planning with deep reinforcement learning. In: 2017 IEEE/RSJ IROS, pp. 1343–1350. IEEE (2017) Chen, Y., Everett, M., Liu, M., How, J.P.: Socially aware motion planning with deep reinforcement learning. In: 2017 IEEE/RSJ IROS, pp. 1343–1350. IEEE (2017)
5.
go back to reference Chen, Y., Liu, M., Everett, M., How, J.P.: Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. In: 2017 IEEE ICRA, pp. 285–292. IEEE (2017) Chen, Y., Liu, M., Everett, M., How, J.P.: Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. In: 2017 IEEE ICRA, pp. 285–292. IEEE (2017)
6.
go back to reference Chen, C., Liu, Y., Kreiss, S., Alahi, A.: Crowd-robot interaction: crowd-aware robot navigation with attention-based deep reinforcement learning. In: 2019 ICRA, pp. 6015–6022. IEEE (2019) Chen, C., Liu, Y., Kreiss, S., Alahi, A.: Crowd-robot interaction: crowd-aware robot navigation with attention-based deep reinforcement learning. In: 2019 ICRA, pp. 6015–6022. IEEE (2019)
7.
go back to reference Cross, E., Hortensius, R., Wykowska, A.: From social brains to social robots: applying neurocognitive insights to human-robot interaction. Philos. Trans. Roy. Soc. London. Ser. B Biol. Sci. 374 (2019) Cross, E., Hortensius, R., Wykowska, A.: From social brains to social robots: applying neurocognitive insights to human-robot interaction. Philos. Trans. Roy. Soc. London. Ser. B Biol. Sci. 374 (2019)
8.
go back to reference Everett, M., Chen, Y., How, J.P.: Motion planning among dynamic, decision-making agents with deep reinforcement learning. In: 2018 IEEE/RSJ IROS, pp. 3052–3059. IEEE (2018) Everett, M., Chen, Y., How, J.P.: Motion planning among dynamic, decision-making agents with deep reinforcement learning. In: 2018 IEEE/RSJ IROS, pp. 3052–3059. IEEE (2018)
9.
go back to reference Fong, T., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots. Robot. Auton. Syst. 42(3–4), 143–166 (2003)CrossRef Fong, T., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots. Robot. Auton. Syst. 42(3–4), 143–166 (2003)CrossRef
10.
go back to reference Gu, T., Dolan, J.: Toward human-like motion planning in urban environments. In: 2014 IEEE Intelligent Vehicles Symposium Proceedings, pp. 350–355. IEEE (2014) Gu, T., Dolan, J.: Toward human-like motion planning in urban environments. In: 2014 IEEE Intelligent Vehicles Symposium Proceedings, pp. 350–355. IEEE (2014)
11.
go back to reference Helbing, D., Molnar, P.: Social force model for pedestrian dynamics. Phys. Rev. E 51(5), 4282 (1995)CrossRef Helbing, D., Molnar, P.: Social force model for pedestrian dynamics. Phys. Rev. E 51(5), 4282 (1995)CrossRef
13.
go back to reference Karl, M., Soelch, M., Becker-Ehmck, P., Benbouzid, D., van der Smagt, P., Bayer, J.: Unsupervised real-time control through variational empowerment. arXiv preprint arXiv:1710.05101 (2017) Karl, M., Soelch, M., Becker-Ehmck, P., Benbouzid, D., van der Smagt, P., Bayer, J.: Unsupervised real-time control through variational empowerment. arXiv preprint arXiv:​1710.​05101 (2017)
15.
go back to reference Klyubin, A.S., Polani, D., Nehaniv, C.L.: Empowerment: a universal agent-centric measure of control. In: 2005 IEEE Congress on Evolutionary Computation, vol. 1, pp. 128–135. IEEE (2005) Klyubin, A.S., Polani, D., Nehaniv, C.L.: Empowerment: a universal agent-centric measure of control. In: 2005 IEEE Congress on Evolutionary Computation, vol. 1, pp. 128–135. IEEE (2005)
16.
go back to reference Klyubin, A.S., Polani, D., Nehaniv, C.: Empowerment: A universal agent-centric measure of control, vol. 1, pp. 128–135 (2005) Klyubin, A.S., Polani, D., Nehaniv, C.: Empowerment: A universal agent-centric measure of control, vol. 1, pp. 128–135 (2005)
17.
go back to reference Kruse, T., Pandey, A.K., Alami, R., Kirsch, A.: Human-aware robot navigation: a survey. Robot. Autonomous Syst. 61(12), 1726–1743 (2013)CrossRef Kruse, T., Pandey, A.K., Alami, R., Kirsch, A.: Human-aware robot navigation: a survey. Robot. Autonomous Syst. 61(12), 1726–1743 (2013)CrossRef
19.
go back to reference Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529 (2015)CrossRef Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529 (2015)CrossRef
20.
go back to reference Mohamed, S., Rezende, D.J.: Variational information maximisation for intrinsically motivated reinforcement learning. In: NeurIPS, pp. 2125–2133 (2015) Mohamed, S., Rezende, D.J.: Variational information maximisation for intrinsically motivated reinforcement learning. In: NeurIPS, pp. 2125–2133 (2015)
21.
go back to reference Oudeyer, P., Kaplan, F., Hafner, V.V.: Intrinsic motivation systems for autonomous mental development. IEEE Trans. Evol. Comput. 11(2), 265–286 (2007)CrossRef Oudeyer, P., Kaplan, F., Hafner, V.V.: Intrinsic motivation systems for autonomous mental development. IEEE Trans. Evol. Comput. 11(2), 265–286 (2007)CrossRef
22.
go back to reference Pfeiffer, M., Schwesinger, U., Sommer, H., Galceran, E., Siegwart, R.: Predicting actions to act predictably: Cooperative partial motion planning with maximum entropy models. In: 2016 IEEE/RSJ IROS, pp. 2096–2101. IEEE (2016) Pfeiffer, M., Schwesinger, U., Sommer, H., Galceran, E., Siegwart, R.: Predicting actions to act predictably: Cooperative partial motion planning with maximum entropy models. In: 2016 IEEE/RSJ IROS, pp. 2096–2101. IEEE (2016)
24.
go back to reference Salge, C., Polani, D.: Empowerment as replacement for the three laws of robotics. Front. Robot. AI 4, 25 (2017)CrossRef Salge, C., Polani, D.: Empowerment as replacement for the three laws of robotics. Front. Robot. AI 4, 25 (2017)CrossRef
26.
go back to reference Shiarlis, K., Messias, J., Whiteson, S.: Acquiring social interaction behaviours for telepresence robots via deep learning from demonstration. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 37–42. IEEE (2017) Shiarlis, K., Messias, J., Whiteson, S.: Acquiring social interaction behaviours for telepresence robots via deep learning from demonstration. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 37–42. IEEE (2017)
27.
go back to reference Sieben, A., Schumann, J., Seyfried, A.: Collective phenomena in crowds-where pedestrian dynamics need social psychology. PLoS One 12(6), e0177328 (2017)CrossRef Sieben, A., Schumann, J., Seyfried, A.: Collective phenomena in crowds-where pedestrian dynamics need social psychology. PLoS One 12(6), e0177328 (2017)CrossRef
28.
go back to reference Sutton, R.S., Barto, A.G., et al.: Introduction to Reinforcement Learning, vol. 2. MIT Press, Cambridge (1998)MATH Sutton, R.S., Barto, A.G., et al.: Introduction to Reinforcement Learning, vol. 2. MIT Press, Cambridge (1998)MATH
29.
go back to reference Templeton, A., Drury, J., Philippides, A.: Walking together: behavioural signatures of psychological crowds. Roy. Soc. Open Science 5(7), 180172 (2018)CrossRef Templeton, A., Drury, J., Philippides, A.: Walking together: behavioural signatures of psychological crowds. Roy. Soc. Open Science 5(7), 180172 (2018)CrossRef
30.
go back to reference Trautman, P., Krause, A.: Unfreezing the robot: navigation in dense, interacting crowds. In: 2010 IEEE/RSJ IROS, pp. 797–803. IEEE (2010) Trautman, P., Krause, A.: Unfreezing the robot: navigation in dense, interacting crowds. In: 2010 IEEE/RSJ IROS, pp. 797–803. IEEE (2010)
31.
go back to reference van den Berg, J., Guy, S.J., Snape, J., Lin, M.C., Manocha, D.: Rvo2 library: Reciprocal collision avoidance for real-time multi-agent simulation van den Berg, J., Guy, S.J., Snape, J., Lin, M.C., Manocha, D.: Rvo2 library: Reciprocal collision avoidance for real-time multi-agent simulation
32.
go back to reference Van den Berg, J., Lin, M., Manocha, D.: Reciprocal velocity obstacles for real-time multi-agent navigation. In: 2008 IEEE International Conference on Robotics and Automation, pp. 1928–1935. IEEE (2008) Van den Berg, J., Lin, M., Manocha, D.: Reciprocal velocity obstacles for real-time multi-agent navigation. In: 2008 IEEE International Conference on Robotics and Automation, pp. 1928–1935. IEEE (2008)
Metadata
Title
Social Navigation with Human Empowerment Driven Deep Reinforcement Learning
Authors
Tessa van der Heiden
Florian Mirus
Herke van Hoof
Copyright Year
2020
DOI
https://doi.org/10.1007/978-3-030-61616-8_32

Premium Partner