New Horizons in Evolutionary Robotics

This paper considers the field of Evolutionary Robotics (ER) from the perspective of its potential users: roboticists. The core hypothesis motivating this field of research is discussed, as well as the potential use of ER in a robot design process. Four main aspects of ER are presented: (a) ER as an automatic parameter tuning procedure, which is the most mature application and is used to solve real robotics problem, (b) evolutionary-aided design, which may benefit the designer as an efficient tool to build robotic systems (c) ER for online adaptation, i.e. continuous adaptation to changing environment or robot features and (d) automatic synthesis, which corresponds to the automatic design of a mechatronic device and its control system. Critical issues are also presented as well as current trends and pespectives in ER. A section is devoted to a roboticist’s point of view and the last section discusses the current status of the field and makes some suggestions to increase its maturity.

The field of embodied artificial intelligence is maturing, and as such has progressed from what questions (“what is embodiment?”) to how questions: how should the body plan of an autonomous robot be designed to maximize the chance that it will exhibit a desired set of behaviors. In order to stand on its own however, rather than a reaction to classical AI, the field of embodied AI must address why questions as well: why should body and brain both be considered when creating intelligent machines? This report provides three new lines of evidence for why the body plays an important role in cognition: (1) an autonomous robot must be able to adapt behavior in the face of drastic, unanticipated change to its body; (2) under-explored body plans raise new research questions related to cognition; and (3) optimizing body plans accelerates the automated design of intelligent machines, compared to leaving them fixed.

While at present Evolutionary Robotics (ER) is generally not studied in mainstream robotics, the main idea in this article is that ER has the opportunity to gain relevance by taking seriously its natural inspiration. The chasm that separates the behavior of robots today from the robustness and fluidity of organisms in nature is most naturally addressed by an approach that indeed respects the process through which such organisms originated. Yet the challenge is to identify the elusive missing ingredient that would allow ER to realize its full potential.

To expand the potential of the robotic systems by introducing innovative mechanical structures and to endow the systems of sophisticated behaviors taking into account in particular the variability of the environments in which they are intended to evolve constitutes challenges for the research in design. The needs in terms of rationality and efficiency in these new challenges for robotic engineering lead the development of systems to assist engineers in the different phases of their design activities. Generally speaking, design is defined as a process in which for a given description of desired functions and constraints to satisfy called specifications we try to produce the description of an artifact or several of them fitting the specifications. The design process may entail the creation of new solutions or evolutions from an existing solution. To this aim, series of activities are performed by which designer perception of a problem is transformed into an output satisfying the problem. The entire design process is basically an iterative process which can be viewed at a conceptual level as a chain of activities which consists in 1) Clarifying the requirements and needs (the output is frequently a problem statement since it is rarely given at the beginning) 2) Defining the constraints by analyzing the operating environment (functional and performance specification) 3) Generating concepts and solutions 4) Modeling and analyzing the behavior 5) Testing and evaluating the proposed design(s) 6) Refining and optimizing the design of selected solutions. Design methodologies in the field of mechanical engineering has been first introduced on the base of prescriptive models then on descriptive models and more recently as computer-based models [1]. The aim of a prescriptive model is typically to provide guidelines or frameworks to organize and structure the process of creating instructional design activities. Traditionally, these design recommendations are based on the experience coming from identical or similar developments. It is quite evident that this kind of concept generally inhibits innovation.

Evolutionary robotics has been shown to be an effective technique for generating robot behaviors that are difficult to derive analytically from the robot’s mechanics and task environment. Moreover, augmenting evolutionary algorithms with environmental scaffolding via an incremental shaping method makes it possible to evolve controllers for complex tasks that would otherwise be infeasible. In this paper we present a summary of two recent publications in the evolutionary robotics literature demonstrating how these methods can be used to evolve robot controllers for non-trivial tasks, what the obstacles are in evolving controllers in this way, and present a novel research question that can be investigated under this framework.

Evolutionary Algorithms are now mature optimization tools, especially in a multi-objective context. This ability is used here to help explore, analyse and, on this basis, propose a controller for a complex robotics system: a flapping wings aircraft. A multi-objective optimization is performed to find the best parameters of sinusoidal wings kinematics. Multi-objective algorithms generate a set of trade-off solutions instead of a single solution. The feedback is then potentially more informative in a multi-objective context relative to the one of a single objective setup: the set of trade-off solutions can be analyzed to characterize the studied system. Such an approach is applied to study a simulated flapping wing aircraft. The speed-energy relation is empirically evaluated and the analysis of the relations between the parameters of the kinematics and speed has led, in a further step, to the synthesis of an open-loop controller allowing to change speed during flight.

In this paper, we describe possible methods of solving two problems encountered in evolutionary robotics, while applying neuroevolutionary methods to evolve controllers for complex robotic tasks. The first problem is the large number of evaluations required to obtain a solution. We propose that this problem can be addressed by accelerating neuroevolutionary methods using a Kalman filter. The second problem is the difficulty of obtaining a desirable solution that results from the difficulty of defining an appropriate fitness function for a complex robotic task. The solution towards this problem is to apply the principles of behavior based systems to decompose the solution space into smaller subsolutions with lower number of intrinsic dimensions, and incrementally modify the fitness function. We present two case studies towards the solutions to the stated problems.

This paper presents the design of a manipulator working in a highly constrained workspace. The difficulties implied by the geometry of the environment lead to resort to evolutionary-aided design techniques. As the solution space is likely to be shaped strangely due to the particular environment, a special attention is paid to support the algorithm exploration and avoid negative impacts from the problem formulation, the fitness function or the evaluation. In that respect, a specific genome able to encompass all cases is set up and a constraint compliant control law is used to avoid the arbitrary penalization of robots. The presented results illustrate the methodology adopted to work with the developed evolutionary-aided design tool.

Inspired by the major principles of gene regulation and cellular interactions in multi-cellular development, this paper proposes a distributed self-organizing multi-robot system for pattern formation. In our approach, multiple robots are able to self-organize themselves into various patterns driven by the dynamics of a gene regulatory network model. The pattern information is embedded into the gene regulation model, analog to the morphogen gradient in multi-cellular development. Various empirical analysis of the system robustness to the changes in tasks, noise in the robot system and changes in environment has been conducted. Simulation results demonstrate that the proposed method is both effective for pattern formation and robust to environmental changes.

Novelty search is a recent and promising approach to evolve neurocontrollers, especially to drive robots. The main idea is to maximize the novelty of behaviors instead of the efficiency. However, abandoning the efficiency objective(s) may be too radical in many contexts. In this paper, a Pareto-based multi-objective evolutionary algorithmis employed to reconcile novelty search with objective-based optimization by following a multiobjectivization process. Several multiobjectivizations based on behavioral novelty and on behavioral diversity are compared on a maze navigation task. Results show that the bi-objective variant “Novelty + Fitness” is better at fine-tuning behaviors than basic novelty search, while keeping a comparable number of iterations to converge.

This paper deals with online onboard behavior optimization for a autonomous mobile robot in the scope of the European FP7 Symbrion Project. The work presented here extends the (1+1)-online algorithm introduced in [4]. The (1+1)-online algorithm has a limitation regarding the ability to perform global search whenever a local optimum is reached. Our new implementation of the algorithm, termed (1+1)-restart-online algorithm, addresses this issue and has been successfully experimented using a Cortex M3 microcontroller connected to a realistic robot simulator as well as within an autonomous robot based on an Atmel ATmega128 microcontroller. Results from the experiments show that the new algorithm is able to escape local optima and to perform behavior optimization in a complete autonomous fashion. As a consequence, it is able to converge faster and provides a richer set of relevant controllers compared to the previous implementation.

We developed a new approach for automated synthesis of a planning logic for autonomous unmanned vehicles. This new approach can be viewed as an automated iterative process during which an initial version of a logic is synthesized and then gradually improved by detecting and fixing its shortcomings. This is achieved by combining data mining for extraction of vehicle’s states of failure and Genetic Programming (GP) technique for synthesis of corresponding navigation code. We verified the feasibility of the approach using unmanned surface vehicles (USVs) simulation. Our focus was specifically on the generation of a planning logic used for blocking the advancement of an intruder boat towards a valuable target. Developing autonomy logic for this behavior is challenging as the intruder’s attacking logic is human-competitive with deceptive behavior so the USV is required to learn specific maneuvers for specific situations to do successful blocking. We compared the performance of the generated blocking logic to the performance of logic that was manually implemented. Our results show that the new approach was able to synthesize a blocking logic with performance closely approaching the performance of the logic coded by hand.

In multi-modular reconfigurable robotics it is extremely challenging to develop control software that is able to generate robust but still flexible behavior of the ‘robotic organism’ that is formed by several independent robotic modules. We propose artificial evolution and self-organization as methodologies to develop such control software. In this article, we present our concept to evolve a self-organized multi-modular robot. We decompose the network of feedbacks, that affect the evolutionary pathway and show why and how specific sub-components, which are involved in these feedbacks, should be subject of evolutionary adaptation. Self-organization is a major component of our framework and is implemented by a hormone-inspired controller governing the behavior of singular autonomous modules. We show first results, which were obtained by artificial evolution with our framework, and give an outlook of how the framework will be applied in future research.

A persistent challenge in evolutionary robotics is the transfer of evolved morphologies from simulation to reality, especially when these morphologies comprise complex geometry with embedded active elements. In this chapter we describe an approach that automatically evolves target structures based on functional requirements and plans the error-free assembly of these structures from a large number of active components. Evolution is conducted by minimizing the strain energy in a structure due to prescribed loading conditions. Thereafter, assembly is planned by sampling the space of all possible paths to the target structure and following those that leave the most options open. Each sample begins with the final completed structure and removes one accessible component at a time until the existing substructure is recovered. Thus, at least one path to a complete target structure is guaranteed at every stage of assembly. Automating the entire process represents a step towards an interactive evolutionary design and fabrication paradigm, similar to that seen in nature.