Introducing an Event-Based Architecture for Concurrent and Distributed Evolutionary Algorithms

Cloud-native applications add a layer of abstraction to the underlying distributed computing system, defining a high-level, self-scaling and self-managed architecture of different microservices linked by a messaging bus. Creating new algorithms that tap these architectural patterns and at the same time employ distributed resources efficiently is a challenge we will be taking up in this paper. We introduce KafkEO, a cloud-native evolutionary algorithms framework that is prepared to work with different implementations of evolutionary algorithms and other population-based metaheuristics by using micro-populations and stateless services as the main building blocks; KafkEO is an attempt to map the traditional evolutionary algorithm to this new cloud-native format. As far as we know, this is the first architecture of this kind that has been published and tested, and is free software and vendor-independent, based on OpenWhisk and Kafka. This paper presents a proof of concept, examines its cost, and tests the impact on the algorithm of the design around cloud-native and asynchronous system by comparing it on the well known BBOB benchmarks with other pool-based architectures, with which it has a remarkable functional resemblance. KafkEO results are quite competitive with similar architectures.