Convolutional Neural Networks

A series of successful applications of Convolutional Neural Networks (CNNs) in various computer vision competitions in 2011 and 2012 were a major driving force behind the momentum of deep learning that we see today. As an example, consider the 2012 ImageNet competition where a deep convolutional neural network, later known as AlexNet, reported a top-5 test error rate of 15.3% for image classification. This error rate was remarkably better than the runner up error of 26.2% achieved by a combination of non-deep learning methods. Following the success of deep learning techniques in 2012, many groups in 2013 used deep CNNs in the competition. This state of affairs is in large part attributed to the ability of CNNs to integrate feature extraction into model construction stage. In other words, and in analogy with embedded feature selection methods discussed in Chapter 10, CNNs have an embedded feature extraction mechanism that can be very helpful when working with multidimensional signals such as time-series, digital images, or videos. There are even some efforts to leverage this important property of CNNs for tabular data types. In this chapterwe describe theworking mechanism of these networks, which is essential for proper implementation and use. Last but not least, the practical software implementation in Keras will be discussed.