Evolving the Topology of Large Scale Deep Neural Networks

In the recent years Deep Learning has attracted a lot of attention due to its success in difficult tasks such as image recognition and computer vision. Most of the success in these tasks is merit of Convolutional Neural Networks (CNNs), which allow the automatic construction of features. However, designing such networks is not an easy task, which requires expertise and insight. In this paper we introduce DENSER, a novel representation for the evolution of deep neural networks. In concrete we adapt ideas from Genetic Algorithms (GAs) and Grammatical Evolution (GE) to enable the evolution of sequences of layers and their parameters. We test our approach in the well-known image classification CIFAR-10 dataset. The results show that our method: (i) outperforms previous evolutionary approaches to the generations of CNNs; (ii) is able to create CNNs that have state-of-the-art performance while using less prior knowledge (iii) evolves CNNs with novel topologies, unlikely to be designed by hand. For instance, the best performing CNN obtained during evolution has an unexpected structure using six consecutive dense layers. On the CIFAR-10 the best model reports an average error of 5.87% on test data.