From Animals to Animats 10

This study presents an extended model of homeostatic adaptation designed to exploit the internal dynamics of a neural network in the absence of sensory input. In order to avoid typical convergence to asymptotic states under these conditions plastic changes in the network are induced in evolved neurocontrollers leading to a renewal of dynamics that may favour sensorimotor adaptation. Other measures are taken to avoid loss of internal variability (as caused, for instance, by synaptic strength saturation). The method allows the generation of reliable adaptation to morphological disruptions in a simple simulated vehicle using a homeostatic neurocontroller that has been selected to behave homeostatically while performing the desired behaviour but non-homeostatically in other circumstances. The performance is compared with simple homeostatic neural controllers that have only been selected for a positive link between internal and behavioural stability. The extended homeostatic networks perform much better and are more adaptive to morphological disruptions that have never been experienced before by the agents.

We compare the performance of drive- versus perception-based motivational systems in an unstable environment. We investigate the hypothesis that valence systems (systems that evaluate positive and negative nature of events) that are based on internal physiology will have an advantage over systems that are based purely on external sensory input. Results show that inclusion of internal drive levels in valence system input significantly improves performance. Furthermore, a valence system based purely on internal drives outperforms a system that is additionally based on perceptual input. We provide arguments for why this is so and relate our architecture to brain areas involved in animal learning.

Controlling a highly dynamics and unknown system by existing control methods would be difficult because of its complexity. Recent biological studies reveal that animals utilize biological fluctuations to achieve adaptability to the environment and high flexibility. In this paper, we propose a simple, but flexible control method inspired by a biological adaptation mechanism. The proposed method is then applied to control robotic arms. The results of simulation indicated that our proposed method can be applied well to the control of a robot with multi-DOF.

Continuous-time recurrent neural networks affected by random additive noise are evolved to produce phototactic behaviour in simulated mobile agents. The resulting neurocontrollers are evaluated after evolution against perturbations and for different levels of neural noise. Controllers evolved with neural noise are more robust and may still function in the absence of noise. Evidence from behavioural tests indicates that robust controllers do not undergo noise-induced bifurcations or if they do, the transient dynamics remain functional. A general hypothesis is proposed according to which evolution implicitly selects neural systems that operate in noise-resistant landscapes which are hard to bifurcate and/or bifurcate while retaining functionality.

This article describes the robotic integration of a robust omnidirectional visual system with a control architecture inspired by neural structures in a rat’s brain. The visual system relies on an optimal recursive sampling of images into subimages that remains stable under translation and makes self-localization and object recognition possible. The control architecture affords navigation and action selection capacities. The operationality of both systems is demonstrated through a series of experiments assessing their capacity to maintain the energy level of a robot within the limits of a given viability zone.

We used an evolutionary robotics methodology to generate pairs of simulated agents capable of reliably establishing and maintaining a coordination pattern under noisy conditions. Unlike previous related work, agents were only evolved for this ability and not for their capacity to discriminate social contingency (i.e., a live responsive partner) from non-contingent engagements (i.e., a recording). However, when they were made to interact with a recording of their partner made during a successful previous interaction, the coordination pattern could not be established. An analysis of the system’s underlying dynamics revealed (i) that stability of the coordination pattern requires ongoing mutuality of interaction, and (ii) that the interaction process is not only constituted by, but also constitutive of, individual behavior. We suggest that this stability of coordination is a general property of a certain class of interactively coupled dynamical systems, and conclude that psychological explanations of an individual’s sensitivity to social contingency need to take into account the role of the interaction process.

This paper deals with the topic of learning through neuroevolutionary algorithms in non-stationary settings. This kind of algorithms that evolve the parameters and/or the topology of a population of Artificial Neural Networks have provided successful results in optimization problems in stationary settings. Their application to non-stationary problems, that is, problems that involve changes in the objective function, still requires more research. In this paper we address the problem through the integration of implicit, internal or genotypic, memory structures and external explicit memories in an algorithm called Promoter Based Genetic Algorithm with External Memory (PBGA-EM). The capabilities introduced in a simple genetic algorithm by these two elements are shown on different tests where the objective function of a problem is changed in an unpredictable manner.

We present a system that automatically selects and parameterizes a vision based obstacle avoidance method adapted to a given visual context. This system uses genetic programming and a robotic simulation to evaluate the candidate algorithms. As the number of evaluations is restricted, we introduce a novel method using imitation to guide the evolution toward promising solutions. We show that for this problem, our two-phase evolution process performs better than other techniques.

We present the results from an evolutionary robotics simulation model of a recent unpublished experiment on human perceptual crossing in a minimal virtual two-dimensional environment. These experiments demonstrate that the participants reliably engage in rhythmic interaction with each other, moving along a line. Comparing three types of evolved agents with radically different embodiment (a simulated arm, a two-wheeled robot and an agent generating a velocity vector in Euclidean space), we identify differences in evolved behaviours and structural invariants of the task across embodiments. The simulation results open an interesting perspective on the experimental study and generate hypotheses about the role of arm morphology for the behaviour observed.

Optimal feedback control has been proposed as an attractive movement generation strategy in goal reaching tasks for anthropomorphic manipulator systems. Recent developments, such as the iterative Linear Quadratic Gaussian (iLQG) algorithm, have focused on the case of non-linear, but still analytically available, dynamics. For realistic control systems, however, the dynamics may often be unknown, difficult to estimate, or subject to frequent systematic changes. In this paper, we combine the iLQG framework with learning the forward dynamics for a simulated arm with two limbs and six antagonistic muscles, and we demonstrate how our approach can compensate for complex dynamic perturbations in an online fashion.

Recent artificial neural networks for machine learning have exploited transient dynamics around globally stable attractors, inspired by the properties of cortical microcolumns. Here we explore whether similarly constrained neural network controllers can be exploited for embodied, situated adaptive behaviour. We demonstrate that it is possible to evolve globally stable neurocontrollers containing a single basin of attraction, which nevertheless sustain multiple modes of behaviour. This is achieved by exploiting interaction between environmental input and transient dynamics. We present results that suggest that this globally stable regime may constitute an evolvable and dynamically rich subset of recurrent neural network configurations, especially in larger networks. We discuss the issue of scalability and the possibility that there may be alternative adaptive behaviour tasks that are more ‘attractor hungry’.

The distribution of bifurcation angles found in ant foraging networks has been shown to give polarity to the networks so that nest-bound ants reaching a bifurcation can choose the appropriate direction. In this paper, we use an individual-based model to test the hypothesis that this distribution is an emergent property of a population of foraging ants optimising the trade-off between exploitation of the existing network to maximise food intake and exploration of the environment to maximise the population’s ability to rapidly adapt to novel or changing environments. We identify a parameter regulating an ant’s drives to forage existing trails and explore uncovered areas of the environment as a collective variable controlling the distribution of bifurcation angles in the foraging network and we show that when the exploration-exploitation trade-off is realised, the resulting distribution exhibits the same informational characteristics as that found in the original study.

A computer model of learning and representing spatial locations is studied. The model builds on biological constraints and assumptions drawn from the anatomy and physiology of the hippocampal formation of the rat. The emphasis of the presented research is on the usability of a computer model originally proposed to describe episodic memory capabilities of the hippocampus in a spatial task. In the present model two modalities – vision and path integration – are contributing to the recognition of a given place. We study how place cell activity emerges due to Hebbian learning in the model hippocampus as a result of random exploration of the environment. The model is implemented in the Webots mobile robotics simulation software. Our results show that the location of the robot is well predictable from the activity of a population of model place cells, thus the model is suitable to be used as a basic building block of location-based navigation strategies. However, some properties of the stored memories strongly resembles that of episodic memories, which do not match special spatial requirements.

We present a neuromimetic navigation system modelling the columnar structure of the cortex to mediate spatial learning and action planning. The model has been validated on a spatial behavioural task, namely the Tolman & Honzik’s

detour

protocol, which allowed us to test the ability of the system to build a topological representation of the environment, and to use it to exhibit flexible goal-directed behaviour (i.e., to predict the outcome of alternative trajectories to avoid blocked pathways). First, it is shown that the model successfully reproduces the navigation performance of rodents in terms of goal-directed path selection. Second, we report on the neural response patterns characterising the learnt columnar space representation.

We created a neural architecture that can use two different types of information encoding strategies depending on the environment. The goal of this research was to create a simulated agent that could react to two different overlapping chemicals having varying concentrations. The neural network controls the agent by encoding its sensory information as temporal coincidences in a low concentration environment, and as firing rates at high concentration. With such an architecture, we could study synchronization of firing in a simple manner and see its effect on the agent’s behaviour.

Theta frequency oscillations are a prominent feature of the hippocampal EEG during active locomotion and learning. It has also been observed that the relative timing of place cell firing recedes as its place field is traversed – a phenomena known as phase precession. This has led to the development of a theory of theta phase coding, whereby spatial sequences being encountered on a behavioural timescale are compressed into a firing sequence of place cells which is repeated in each theta cycle and stored in an auto-associative network using spike-timing dependent plasticity. This paper provides an abstract, descriptive model of theta phase coding in a spiking neural network, and aims to investigate how learning and recall functions may be separated by the neuromodulatory action of Acetylcholine (ACh). It is demonstrated that ACh is not essential for concurrent learning and recall without interference in this case, thanks to the robust nature of the theta phase coding implementation. However, the neuromodulation of synaptic plasticity offers other advantages, and may be essential to avoid continually consolidating false predictions when learning new routes.

After a short review of properties of biological place cells, mainly found in the hippocampal region of rodents, and a brief presentation of a biologically inspired navigation architecture relying on these cells, we will show how contextual information could facilitate scale changes to large environments. We thus present a simple model of spatial context allowing to both reduce noise effects on place cells (in biological model) and increase its computational performance.

This paper presents an investigation into the robustness to motor noise of an insect-inspired visual navigation method that links together local view-based navigation in a series of visual locales automatically defined by the method. The method is tested in the real world using specialist robotic equipment that allows a controllable level of motor noise to be used. Extensions to the method, which can improve its robustness to severe motor noise and to major disruptions such as being displaced along its route, are investigated.

Three factor Isotropic sequence order (ISO3) learning is a form of differential Hebbian learning where a third factor switches on learning at relevant moments for example, after reward retreival. This switch enables learning only at specific moments and, thus, stablises the corresponding weights. The concept of using a third factor as a gating signal for learning at relevant moments has been extended in this paper to perform second order conditioning (SOC). We present a biological model of the sub-cortical nuclei of the limbic system that is capable of performing SOC in a food seeking task. The 3rd-factor is modelled by dopaminergic neurons of the VTA which are activated via a direct excitatory glutamatergic pathway, and an indirect dis-inhibitory GABAergic pathway. The latter generates an amplification in the number of tonically active DA neurons. This produces an increase in DA outside the event of a primary reward and enables SOC to be accomplished.

We propose a novel methodology for learning and synthesising whole classes of high dimensional movements from a limited set of demonstrated examples that satisfy some underlying ’latent’ low dimensional task constraints. We employ non-linear dimensionality reduction to extract a canonical latent space that captures some of the essential topology of the unobserved task space. In this latent space, we identify suitable parametrisation of movements with control policies such that they are easily modulated to generate novel movements from the same class and are robust to perturbations. We evaluate our method on controlled simulation experiments with simple robots (reaching and periodic movement tasks) as well as on a data set of very high-dimensional human (punching) movements. We verify that we can generate a continuum of new movements from the demonstrated class from only a few examples in both robotic and human data.

Evolutionary algorithms have been successfully used to create controllers for many animats. However, intuitive fitness functions like the survival time of the animat, often do not lead to interesting results because of the bootstrap problem, arguably one of the main challenges in evolutionary robotics: if all the individuals perform equally poorly, the evolutionary process cannot start. To overcome this problem, many authors defined ordered sub-tasks to bootstrap the process, leading to an incremental evolution scheme. Published methods require a deep knowledge of the underlying structure of the analyzed task, which is often not available to the experimenter. In this paper, we propose a new incremental scheme based on multi-objective evolution. This process is able to automatically switch between each sub-task resolution and does not require to order them. The proposed method has been successfully tested on the evolution of a neuro-controller for a complex-light seeking simulated robot, involving 8 sub-tasks.

The active vision and attention-for-action frameworks propose that in organisms attention and perception are closely integrated with action and learning. This work proposes a novel bio-inspired integrated neural-network architecture that on one side uses attention to guide and furnish the parameters to action, and on the other side uses the effects of action to train the task-oriented top-down attention components of the system. The architecture is tested both with a simulated and a real camera-arm robot engaged in a reaching task. The results highlight the computational opportunities and difficulties deriving from a close integration of attention, action and learning.

The information flow along the dorsal visual stream of the primate brain is being thoroughly studied in neuroscience, and this research is being used in artificial intelligence applications. The knowledge regarding one of its most critical stages though, the posterior intraparietal area CIP, remains relatively undeveloped. This paper offers new computational descriptions of the tasks performed by CIP as a fundamental relay station between the visual cortex and the visuomotor areas downstream. Analytical expressions of the transfer functions realized by surface and axes orientation selective neurons (SOS and AOS) of CIP are derived and discussed.

The Lateral Superior Olive (LSO) codes for interaural intensity difference (IID), a cue used for sound localization. Between birth and maturation, the LSO undergoes plasticity driven by input neurons activity. During this developmental phase, a number of inputs are pruned out leading to a refinement of the frequency tuning. The goal of this paper is to show that, using a physiologically plausible network architecture and neuronal model, the activity dependent plasticity of the LSO can be modeled using Spike-Timing Dependent Plasticity (STDP). In particular, we show that the time properties of STDP coupled with the fact that the frequency axis in the LSO can be considered as a delay axis leads to the observed tonotopical map refinement. The response of both the individual neurons as well as population are shown to be in accordance with data taken from physiological analysis.

We have developed a new reinforcement learning (RL) technique called Bayesian-discrimination-function-based reinforcement learning (BRL). BRL is unique, in that it does not have state and action spaces designed by a human designer, but adaptively segments them through the learning process. Compared to other standard RL algorithms, BRL has been proven to be more effective in handling problems encountered by multi-robot systems (MRS), which operate in a learning environment that is naturally dynamic. Furthermore, we have developed an extended form of BRL in order to improve the learning efficiency. Instead of generating a random action when a robot functioning within the framework of the standard BRL encounters an unknown situation, the extended BRL generates an action determined by linear interpolation among the rules that have high similarity to the current sensory input. In this study, we investigate the robustness of the extended BRL through further experiments. In both physical experiments and computer simulations, the extended BRL shows higher robustness and relearning ability against an environmental change as compared to the standard BRL.

This paper introduces a novel multi-cellular developmental system where cells are placed in a continuous space. Cells communicate by diffusing and perceiving substances in the environment and are able to migrate around following affinities with substance gradients. The optimization process is performed using Echo State neural networks on the problem of minimizing tile size variations in the context of a tiling problem. Experimental results show that problem complexity only impacts the number of substances used, rather than the number of cells, which implies some sort of scalability with regards to the size of the phenotype. Symmetry breaking and robustness are addressed by adding noise as an intrinsic property of the model. A (positive) side effect is that the resulting model produces very robust solutions with efficient self-healing behavior in the presence of perturbations never met before.

The purpose of this paper is to outline a new formulation of statistical learning that will be more useful and relevant to the field of robotics. The primary motivation for this new perspective is the mismatch between the form of data assumed by current statistical learning algorithms, and the form of data that is actually generated by robotic systems. Specifically, robotic systems generate a vast unlabeled data stream, while most current algorithms are designed to handle limited numbers of discrete, labeled, independent and identically distributed samples. We argue that there is only one meaningful unsupervised learning process that can be applied to a vast data stream: adaptive compression. The compression rate can be used to compare different techniques, and statistical models obtained through adaptive compression should also be useful for other tasks.

It has been shown recently that dopamine signalled modulation of spike timing-dependent synaptic plasticity (DA-STDP) can enable reinforcement learning of delayed stimulus-reward associations when both stimulus and reward are delivered at precisely timed intervals. Here, we test whether a similar model can support learning in an embodied context, in which timing of both sensory input and delivery of reward depend on the agent’s behaviour. We show that effective reinforcement learning is indeed possible, but only when stimuli are gated so as to occur as near-synchronous patterns of neural activity and when neuroanatomical constraints are imposed which predispose agents to exploratative behaviours. Extinction of learned responses in this model is subsequently shown to result from agent-environment interactions and not directly from any specific neural mechanism.

Both self-learning architecture (embedded structure) and explicit/implicit teaching from other agents (environmental design issue) are necessary not only for one behavior learning but more seriously for life-time behavior learning. This paper presents a method for a robot to understand unfamiliar behavior shown by others through the collaboration between behavior acquisition and recognition of observed behavior, where the state value has an important role not simply for behavior acquisition (reinforcement learning) but also for behavior recognition (observation). That is, the state value updates can be accelerated by observation without real trials and errors while the learned values enrich the recognition system since it is based on estimation of the state value of the observed behavior. The validity of the proposed method is shown by applying it to a dynamic environment where two robots play soccer.

Emotions play several important roles in the cognition of human beings and other life forms, and are therefore a legitimate inspiration to provide adaptability and autonomy to situated agents. However, there is no unified theory of emotions and many discoveries are yet to be made in the applicability of emotions to situated agents. This paper investigates the feasibility and utility of an artificial model of anger and fear based on Interruption Theory of Emotions. This model detects and highlights situations for which an agent’s decision-making mechanism is no longer pertinent. These situations are detected by analyzing discrepancies between the agent’s actions and its intentions, making this model independent from the agent’s environment and tasks. Collective foraging simulations are used to characterize the influence of the model. Results show that the model improves the adaptability of a group of agents by simultaneously optimizing multiple performance criterion.

The current paper studies possible neuronal mechanisms for meta-level cognition of rule switching. In contrast to the conventional approach of hand-designing the cognitive functions, our study employs evolutional processes to search for neuronal mechanisms accounting for meta-level cognitive functions required in the investigated robotic tasks. Our repeated simulation experiments showed that the different rules are embedded in separate self-organized attractors, while rule switching is enabled by the transitions among attractors. Furthermore, the results showed that although certain segregation between the lower sensory-motor level and the higher cognitive level enhance the task performance, meta-level cognition is significantly supported by the embodiment and the lower level sensory-motor properties.

The mechanisms underlying learning in classical conditioning experiments play a key role in many learning processes of real organisms. This paper presents a novel computational model that incorporates a biologically plausible hypothesis on the functions that the main nuclei of the amygdala might play in first and second order classical conditioning tasks. The model proposes that in these experiments the first and second order conditioned stimuli (CS) are associated both (a) with the unconditioned stimuli (US) within the basolateral amygdala (BLA), and (b) directly with the unconditioned responses (UR) through the connections linking the lateral amygdala (LA) to the central nucleus of amygdala (CeA). The model, embodied in a simulated robotic rat, is validated by reproducing the results of first and second order conditioning experiments of both sham-lesioned and BLA-lesioned real rats.

We propose a sub-symbolic connectionist model in which a functionally compositional system self-organizes by learning a provided set of goal-directed actions. This approach is compatible with an idea taken from usage-based accounts of the developmental learning of language, especially one theory of infants’ acquisition process of symbols. The presented model potentially explains a possible continuous process underlying the transitions from rote knowledge to systematized knowledge by drawing an analogy to the formation process of a geometric regular arrangement of points. Based on the experimental results, the essential underlying process is discussed.

Learning when and how to generalize knowledge from past experience to novel circumstances is a challenging problem many agents face. In animals, this generalization can be caused by mediated conditioning—when two stimuli gain a relationship through the mediation of a third stimulus. For example, in sensory preconditioning, if a light is always followed by a tone, and that tone is later paired with a shock, the light will come to elicit a fear reaction, even though the light was never directly paired with shock. In this paper, we present a computational model of mediated conditioning based on reinforcement learning with predictive representations. In the model, animals learn to predict future observations through the temporal-difference algorithm. These predictions are generated using both current observations and other predictions. The model was successfully applied to a range of animal learning phenomena, including sensory preconditioning, acquired equivalence, and mediated aversion. We suggest that animals and humans are fruitfully understood as representing their world as a set of chained predictions and propose that generalization in artificial agents may benefit from a similar approach.

This paper reports a new finding on functionalities of trembling, the bodily manifestation of fear and joy. We consider trembling of a physically-simulated agent consisting of a vision system and a neural system. It is demonstrated that the noise to visual streams generated by trembling enhances signal to noise ratio of the neural system.

We examine the influence of asynchronism in the Spatial Prisoner’s Dilemma game. Previous studies reported that less cooperation is achieved with the asynchronous version of the game than with the synchronous one. Here, we show that, in general, the opposite is the most common outcome. This conclusion is only possible because a larger number of scenarios was tested, namely, different interaction topologies, a transition rule that can be tuned to emulate different levels of determinism in the choice of the next strategy to be adopted and different rates of asynchronism. The influence of stochastic and deterministic periodic updating in the outcome of the system is also compared. We found that these two update disciplines lead basically to the same result. This is an important issue in the simulation of social and biological behavior.

In a simulated guards-and-thieves scenario we study how the behavioral system of an autonomous agent, which consists of multiple perceptual and motor schemas endowed with anticipatory mechanisms, self-organizes for satisfying its drives. Furthermore, we study how schemas acquired for navigation can be re-used off-line, ‘in simulation’, for forecasting future dangers, and planning trajectories leading to goal locations. We argue that off-line simulations permit not only to coordinate with the present, but with the future, too, and to act goal-directed.

Adaptive division of labour is one key characteristic of eusocial insect colonies and of high relevance in biology, ethology, swarm intelligence and robotics. We constructed an individual based model of division of labour in a honeybee colony. Our model incorporates distinct worker cohorts (foragers, storers, nurses), unemployed bees and larvae. Our goal was a model as accurate as possible, thus we implemented a heterogeneous environment, agents’ physiology and the flow of nutrients within the colony. In our model, the bees decide which task to choose, depending on the intensity of stimuli and on individual thresholds, which are modulated in response to task performance. We describe the main aspects of this model and demonstrate the stability of the emerging division of labour. The model predicts the energetic costs of sudden perturbations (removing/adding cohorts of workers of one task), as well as the resulting shifts in task cohort sizes.

We present an adaptive strategy for a group of robots engaged in the localization of multiple targets. The robotic search algorithm is inspired by chemotaxis behavior in bacteria, and the algorithmic parameters are updated using a distributed implementation of the Particle Swarm Optimization technique. We explore the efficacy of the adaptation, the impact of using local fitness measurements to improve global fitness, and the effect of different particle neighborhood sizes on performance. The robustness of the approach in non-static environments is tested in a time-varying scenario.

We present a novel approach to self-organisation of coordinated behaviour among multiple resource-sharing agents. We consider a hierarchical multi-agent system comprising multiple energy-dependent agents split into local neighbourhoods, each with a dedicated controller, and a centralised coordinator dealing only with the controllers. The coordinated behaviour is required in order to achieve a balance between the overall resource consumption by the multi-agent collective and the stress on the community. Minimising the resource consumption increases the stress, while reducing the stress may lead to unrestricted and highly unpredictable demand, harming the individual agents in the long-run. We identify underlying forces in the system’s dynamics, suggest a number of quantitative measures used to contrast different strategies, and introduce a novel strategy based on persistent sensorimotor time-loops:

homeotaxis

. Homeotaxis subsumes the homeokinetic principle, extending it both in terms of scope (multi-agent self-organisation) and the state-space, and allows to select the best adaptive strategy for the considered system.

Ant foraging is a paradigmatic example of self-organized behavior. We give new experimental evidence for previously unobserved short-term adaptiveness in ant foraging and show that current mathematical foraging models cannot predict this behavior. As a true extension, we develop Itô diffusion models that explain the newly discovered behavior qualitatively and quantitatively. The theoretical analysis is supported by individual-based simulations. Our work shows that randomness is a key factor in allowing self-organizing systems to be adaptive. Implications for technical applications of Swarm Intelligence are discussed.

In social insect colonies, many tasks are performed by higher-order entities, such as groups and teams whose task solving capacities transcend those of the individual participants. In this paper, we investigate the emergence of such higher-order entities using a colony of up to 12 physical robots. We report on an experimental study in which the robots engage in a range of different activities, including exploration, path formation, recruitment, self-assembly and group transport. Once the robots start interacting with each other and with their environment, they self-organise into teams in which distinct roles are performed concurrently. The system displays a dynamical hierarchy of teamwork, the cooperating elements of which comprise higher-order entities. The study shows that teamwork requires neither individual recognition nor inter-individual differences, and as such might contribute to the ongoing debate on the role of such characteristics for the division of labour in social insects.

We study

social

control of a cow herd in which some of the animals are controlled by a sensing and actuation device mounted on the cow. The control is social in that it aims at exploiting the existing gregarious behavior of the animals, rather than controlling each individual directly. As a case study we consider the open-loop control of the herd’s position using location-dependent stimuli. We propose a hybrid dynamical model for capturing the dynamics of the animals during periods of grazing and periods of stress. We assume that stress can either be induced by the sensing and actuation device or by social amplification due to observing/overhearing nearby stressed congeners. The dynamics of the grazing part of the proposed model have been calibrated using experimental data from 10 free-ranging cows, and various assumptions on the animal behavior under stress are investigated by a parameter sweep on the hybrid model. Results show that the gregarious behavior of the animals must be increased during stress for control by undirected stimuli to be successful. We also show that the presence of social amplification of stress allows for robust, low-stress control by controlling only a fraction of the herd.

It is becoming state-of-the-art to form large-scale multi-agent systems or artificial swarms showing adaptive behavior by constructing high numbers of cooperating, embodied, mobile agents (robots). For the sake of space- and cost-efficiency such robots are typically miniaturized and equipped with only few sensors and actuators resulting in rather simple devices. In order to overcome these constraints, bio-inspired concepts of self-organization and emergent properties are applied. Thus, accuracy is usually not a trait of such systems, but robustness and fault tolerance are. It turns out that they are applicable to even hard problems and reliably deliver approximated solutions. Based on these principles we present a heuristic for the Euclidean Steiner tree problem which is NP-hard. Basically, it is the problem of connecting objects in a plane efficiently. The proposed system is investigated from two different viewpoints: computationally and behaviorally. While the performance is, as expected, clearly suboptimal but still reasonably well, the system is adaptive and robust.

Robotic agents can self-organize their interaction with the environment by an adaptive “homeokinetic” controller that simultaneously maximizes sensitivity of the behavior and predictability of sensory inputs. Based on previous work with single robots, we study the interaction of two homeokinetic agents. We show that this paradigm also produces quasi-social interactions among artificial agents. The results suggest that homeokinetic learning generates social behavior only in the the context of an actual encounter of the interaction partner while this does not happen for an identical stimulus pattern that is only replayed. This is in agreement with earlier experiments with human subjects.

We aim to realize human-robot social game interaction as a kind of communication. We proposed a hypothetical development of social game interaction between an infant and a care-giver from a mechanism-sided standpoint, based on developmental psychology. Social games have rules, specific relationship between action and response. Applying the hypothesis, we also propose a scheme to design a robot in which a partner can teach interaction rules through interaction. To investigate the proposed scheme, we built a dynamic model which realizes imitation and ruled interaction and switches them observing partner’s response. In the experiment, the partner can teach and the robot can acquire a rule adaptively through interaction without explicit teaching and subsequently it is also achieved about another rule without reset.

In this paper we present a model of generative proto-imitation that replicates external signals without associating with objects, as in higher-level imitation. A mixed population of adults, that have fixed associations objects-signals, and infants, that do not have associations but imitate unconditionally, endowed with a kinship and interaction structure, allows infants to develop signal affinity with their kin in a variety of conditions and within an initial random world, i.e. in a Babel. Our results indicate that the communicative value of imitation can be discovered after the basic apparatus is in place, rather than that communication is the end to which imitation is the means.

We present an approach for automated design of the functionary of driving agent, able to operate a software model of fast running car. Our objective is to discover a single driving rule (if existent) that is general enough to be able to adequately control the car in all sections of predefined circuits. In order to evolve an agent with such capabilities, we propose an indirect, generative representation of the driving rules as algebraic functions of the features of the perceived surroundings of the car. These functions, when evaluated for the current surrounding of the car yield concrete values of the main attributes of the driving style (e.g., straight line velocity, turning velocity, etc.), applied by the agent in the currently negotiated section of the circuit. Experimental results verify both the very existence of the general driving rules and the ability of the employed genetic programming framework to automatically discover them. The evolved driving rules offer a favorable generality, in that a single rule can be successfully applied (i) not only for all the sections of a particular circuit, but also (ii) for the sections in several a priori defined circuits featuring different characteristics.

Behavior-based control is one of the fundamental control paradigms for autonomous agents to achieve adaptive behavior in a dynamical environment. Existing work has mainly focused on the research aspect rather than on the learning and educational aspect. This paper presents an effort to develop a learning environment for behavior-based agents. It allows educators and students to develop and exercise behavior-based control by setting up entities, behaviors, and behavior networks without involving significant programming effort. Specification of behavior-based agent system is presented and demonstrative examples are provided.

We define a simplified analogue of a kite control task that requires, in its simplest form, a situated artificial agent to switch between two mutually exclusive behaviours. In more complex versions of the task, the agent is required to adapt to changes within its environment that occur on different temporal scales. We describe the failure to evolve successful agents when a decision threshold is defined artificially and conversely the evolution of successful agents when they themselves are allowed to determine their own threshold through interaction with the environment. Agents are demonstrated capable of adapting both their switching behaviour and spatial domain according to environmental changes on three temporal scales, on the fastest of which, the agents behave in an opportunistic manner.

A semiconductor manufacturing factory is a very complicated production system. Typical characteristics of a semiconductor manufacturing factory include: fluctuating demand, jobs with various product types and priorities, un-balanced capacity, jobs’ reentrance to the bottleneck machines, hundreds of processing steps, alternative machines with unequal capacity, etc. Scheduling in a semiconductor manufacturing factory becomes a very difficult task owing to these characteristics. To enhance the performance of dynamic scheduling in a semiconductor manufacturing factory, a self-adaptive agent-based approach is proposed in this study. Firstly, a self-adaptive agent-based scheduling model, which integrates release control, dispatching and machine maintenance scheduling, is presented. Secondly, the negotiation protocol between agents is given. Thirdly, scheduling algorithms for decision making of agents are offered. Unlike in the past studies a single pre-determined scheduling algorithm is used for all agents, in this study every agent develops and modifies its own scheduling algorithm to adapt it to the outside conditions. Finally, production simulation is also applied in this study to generate some test data to evaluate the effectiveness of the proposed methodology.