Abstract
Autonomous robots can generate exploratory behavior by self-organization of the sensorimotor loop. We show that the behavioral manifold that is covered in this way can be modified in a goal-dependent way without reducing the self-induced activity of the robot. We present three strategies for guided self-organization, namely by using external rewards, a problem-specific error function, or assumptions about the symmetries of the desired behavior. The strategies are analyzed for two different robots in a physically realistic simulation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
There are some formal requirements on the parameters, for instance that the determinant of the Jacobian matrix of the sensorimotor loop is positive.
The teaching signal vector is given by \(x_t^{\rm G} = \left(0\,x_{t,2}\,x_{t,3}\right) ^{\top}, \) where x t,i are the sensor values at time t.
The teaching signal is y G t,1 = y t,2 and y G t,2 = y t,1.
The area coverage of the trajectory is calculated using a box-counting method.
References
Abu-Mostafa YS (1995) Hints. Neural Comput 7(4):639–671
Amari S (1998) Natural gradients work efficiently in learning. Neural Comput 10(2):251–276
Cannon WB (1939) In: The wisdom of the body. Norton, New York
Choi J, Wehrspohn RB, Gösele U (2005) Mechanism of guided self-organization producing quasi-monodomain porous alumina. Electrochimica Acta 50(13):2591–2595
Der R (2001) Self-organized acquisition of situated behavior. Theory Biosci 120:179–187
Der R, Herrmann M, Liebscher R (2002) Homeokinetic approach to autonomous learning in mobile robots. In: Dillman R, Schraft RD, Wörn H (eds) Robotik 2002, no.1679 in VDI-Berichte. VDI, Berichte, pp 301–306
Der R, Hesse F, Martius G (2006) Rocking stamper and jumping snake from a dynamical system approach to artificial life. Adapt Behav 14(2):105–115
Der R, Liebscher R (2002) True autonomy from self-organized adaptivity. In: Proceedings of EPSRC/BBSRC International workshop on biologically inspired robotics. HP Labs, Bristol
Der R, Martius G, Hesse F, Güttler F (2009) Videos of self-organized behavior in autonomous robots. http://robot.informatik.uni-leipzig.de/videos
Di Paolo E (2003) Organismically-inspired robotics: homeostatic adaptation and natural teleology beyond the closed sensorimotor loop. In: Murase K, Asakura T (eds) Dynamical systems approach to embodiment and sociality, pp 19–42
Dongyong Y, Jingping J, Yuzo Y (2000) Distal supervised learning control and its application to CSTRsystems. In: SICE 2000. Proceedings of the 39th SICE annual conference, Iizuka, pp 209–214
Herrmann JM (2001) Dynamical systems for predictive control of autonomous robots. Theory Biosci 120:241–252
Ijspeert AJ, Hallam J, Willshaw D (1999) Evolving swimming controllers for a simulated lamprey with inspiration from neurobiology. Adapt Behav 7(2):151–172
Jordan MI, Rumelhart DE (1992) Forward models: Supervised learning with a distal teacher. Cognit Sci 16(3):307–354
Kelso JAS (1995) Dynamic patterns: the selforganization of brain and behavior. The MIT Press, Cambridge
de Margerie E, Mouret JB, Doncieux S, Meyer JA (2007) Artificial evolution of the morphology and kinematics in a flapping-wing mini UAV. Bioinspiration Biomim 2:65–82
Martius G, Herrmann JM (2011) Tipping the scales: guidance and intrinsically motivated behavior. In: Proceedings of advances in artificial life, 11th European Conference (ECAL 2011). MIT Press, pp 506–513
Martius G, Herrmann JM, Der R (2007) Guided self-organisation for autonomous robot development. In: Costa e FA (ed) Proceedings of advances in artificial life, 9th European Conference (ECAL 2007), LNCS, vol. 4648. Springer, San Francisco, pp 766–775
Mazzapioda M, Cangelosi A, Nolfi S (2009) Evolving morphology and control: a distributed approach. In: IEEE congress on evolutionary computation, New Orleans, pp. 2217–2224
Nolfi S, Floreano D (2001) Evolutionary robotics. The biology, intelligence, and technology of self-organizing machines. MIT Press, Cambridge
Peters J, Schaal S (2008) Natural actor-critic. Neurocomputing 71(7-9):1180–1190
Peters J, Vijayakumar S, Schaal S (2005) Natural actor-critic. In: Proceeidngs of the 16th European conference on machine learning (ECML 2005). Springer, Porto, pp. 280–291.
Prokopenko M (2008) Design vs self-organization. In: Prokopenko M (ed) Advances in applied self-organizing systems. Springer, London, pp. 3–17.
Prokopenko M (2009) Guided self-organization. HFSP J 3(5):287–289
Prokopenko M, Gerasimov V, Tanev I (2006) Evolving spatiotemporal coordination in a modular robotic system. In: Nolfi S, Baldassarre G, Calabretta R, Hallam JCT, Marocco D, Meyer JA, Miglino O, Parisi D (eds) SAB, LNCS, vol. 4095. Springer, Heidelberg, pp 558–569.
Rodriguez A (2007) Guided self-organizing particle systems for basic problem solving. Ph.D. thesis, University of Maryland, College Park
Stitt S, Zheng YF (1994) Distal learning applied to biped robots. In: Proceedings of the IEEE International Conference on robotics and automation. IEEE Computer Society, San Diego, pp 137–142
Sutton RS, Barton AG (1998) Reinforcement learning: past, present and future. SEAL, Florham Park, pp 195–197
Verschure PFMJ, Kröse BJA, Pfeifer R (1992) Distributed adaptive control: the self-organization of structured behavior. Robot Auton Sys 9(3):181–196
Acknowledgments
Both authors are grateful to Ralf Der for fruitful discussion. The project was supported by grants #01GQ0811 and #01GQ0432 within the National Bernstein Network Computational Neuroscience.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Martius, G., Herrmann, J.M. Variants of guided self-organization for robot control. Theory Biosci. 131, 129–137 (2012). https://doi.org/10.1007/s12064-011-0141-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12064-011-0141-0