Evolutionary optimization of image processing for cell detection in microscopy images

The main idea of the original evolution strategy created by Rechenberg is to use the concept of natural evolution to solve heuristic optimization problems. The core element therefore is to introduce random changes of each parameter, following the example of natural mutation. With the addition of simple rules for mutation and selection, this concept is the basis for evolution strategies in heuristic optimization (Back et al. 1991).

As originally described by Rechenberg, it is essential to set the population size, the algorithm uses for each generation (Back et al. 1991). The parent generation describes how many solution candidates are generated in the ‘initialize solutions’ part of the algorithm (Fig. 10). This population of solution candidates contains the available gene pool (parameter sets) for future generations. To keep up the genetic diversity, Rechenberg adapted the concept of mutation. By adding a weighted (sigma) random number between zero and one to the current parameter value, a mutant of the initial solution is created (Michalewicz 2013).

If sigma is close to one, the changes caused by the mutation are significant, which leads to a random search. If the fitness increases during the evolution process, sigma is reduced to only allowing slight changes. This process converts the initial randomness into a more directed search to find the optimal value (Affenzeller et al. 2009).

The implementation in our algorithm uses some variations to the original evolution strategy. Our approach includes the parent generation into the selection process. This concept, called offspring selection, is often used in genetic algorithms (Affenzeller and Wagner 2005). For the presented implementation, it is also beneficial to use more than one parent and produce multiple mutants. This increase in population size allows for a more efficient optimization process with a more varied gene pool. Efficiency is further increased by using the success rule mechanism. This mechanism adds the constraint that a fixed percentage of mutants must have higher fitness than the parents from the previous generation; otherwise, the value of sigma is increased. The longer the optimization process is running, the more specific the search process should be. Therefore, the mutations should not cause a switch to completely different solution candidate close to the optimum (Affenzeller and Wagner 2005).

The presented methodology uses solution candidates with two types of genes. One type represents the actual parameter for the image processing algorithm, whereas the other represents switches which can turn off the use of certain algorithms entirely, as seen in Table 1. The combination of all genes of one solution candidate represents a complete processing workflow, including optimized parameters for all algorithms which will be used.

To reduce complexity, we added artificial limits to ensure that the evolution strategy does not have to test solution candidates who do not make sense. Therefore, the sigma used for the Gaussian blur is limited within the range of 0.1 to 4. This range ensures that the filter is not disabled by using a value near zero or that artificial borders are created due to a very large sigma value. The value range of the bandpass filter parameters is dependent on the image type and will either include 256 possible values for 8-bit images or 65535 possible values for 16-bit images.

Depending on the type of cells, it is possible that cells overlap with each other within the image. Therefore, it is necessary to find an appropriate balance between the ability to differentiate the cell areas from the background and the separation between the cells. This balance is realized by three measures which are used for the quality calculation of each solution candidate. Each of these measures can be weighted differently to vary their influence on the final quality value, which allows the flexibility to adjust to different types of microscopy images.As described in Eq. 1, connected components labeling provides the information, about how many individual cell areas were detected. Using this information, the following evaluation measures are calculated.

Amount of detected cells The primary goal of a cell detection algorithm is to identify as many cells as possible. After detecting the areas of each cell using the generated pipeline, binary morphology closes gaps within the cells (Soille 1999). A minimum threshold for the size is used to filter out false positives which are created when the algorithm detects dust particles or air bubbles as cells. The final number of detected cells is calculated as the number of individual detected regions and used as one of the quality measures for evaluation.

Standard deviation of the detected background area After reducing the influence of light sources and their position, the distribution of the background intensities should be similar to a normal distribution with low standard deviation. The standard deviation of pixel intensities (Eq. 2) within a correctly detected background will be lower than within a background with false negatives. This is because the surface tension of the cells is influencing light rays, resulting in more variation of intensities within the cells, compared to the background.Difference in the histogram of cell areas and the background The histograms of background and cell areas are compared. We assume that correct identification of cells as foreground objects will lead to a significant difference in the histograms of the intensities of the background and the detected cells. Thus, we calculate the histograms of the pixels identified as cell pixels (Eq. 3) and the histogram of the background pixels (Eq. 4) and then calculate the relative difference of all entries in these histograms (Eq. 5):Overall fitness After generating the three individual quality measures, we calculate the overall quality of the whole image processing workflow as:This quality value (fitness) is used to compare the mutated solution candidates with each other to find the optimal solution.

Our goal is on the one hand to find the optimal set of parameters for the image processing algorithms and on the other hand to find the correct selection of processing algorithms to use. We have developed a parallel evolutionary optimization method that uses islands of individuals, which is based on the concept of island GAs (Whitley et al. 1999). The switch system of our solution candidates leads to 16 different algorithm combinations including the constraint that the thresholding always has to be active. These 16 islands are initialized with randomly set parameters.

Each of these islands gets optimized independently from the others, to provide parameters which are optimized not only for the algorithm itself, but also for the specific combination with the other algorithms. Otherwise, the optimization is distorted when an algorithm gets activated during one of the last iterations and there is not enough time left to optimize its parameters. The results from the individual islands are compared to each other by their fitness, and the worst performing ones are discarded.

Islands are often used to keep up the genetic diversity during optimization, but most implementations exchange candidates between the islands (Affenzeller et al. 2009) which is not useful for our approach because we use the islands to compare the quality of specific combination of algorithms.

As shown in Fig. 11, each island undergoes several optimization steps, but only the best of them are selected for further optimization iterations. The final solution delivered by this pipeline contains the optimal combination of algorithms, including their optimized parameters.

Each of the islands uses an evolution strategy with one parent and five mutants per generation. All initial islands are optimized for two generations. The best third of them undergoes an optimization for an additional five iterations until the final solution is optimized alone for another five generations. The algorithm finishes if the last iteration is reached or if the quality improvement converges.

This island concept also enables high hardware scalability by distributing different islands to separate CPU cores. To improve performance even further, it is also possible to use a graphical processing units (GPUs) for the image processing calculations.

These benchmark images provide a variety of different-sized objects to be segmented. The goal is to find a combination of processing algorithms and parameters that simultaneously work well for objects of all sizes, even those of very small size which might be mistaken for background noise. As shown in Fig. 12, there is a measurable difference between the pixels that are part of the cell areas and the ones that are part of the background of the image. The optional parameters for the size constraints were approximately set at 200 pixels for the minimum size and 5000 pixels for the maximum size, even if the actual size of the largest object is more around 1500 pixel. This was done to show that these parameters do not have to set perfectly for the workflow to produce good results. There will only be problems if the parameters eliminate actual cells or particles. For example, if the largest object were 6000 pixels, the parameter would be set to 5000.

Figure 13 shows the result of the cell detection. The optimization process only used the single training image, and the test image was segmented afterward using the generated parameter set. As listed in Table 3, this result used a combination of three algorithms. Due to the high amount of noise in the image, it is no surprise that the Gaussian filter is used for overall noise reduction. Afterward, a bandpass filter is used to reduce the intensity range, especially the low intensities due to the lower cutoff frequency of 18812. Watershed segmentation acts as the final separation method for the borders between background and cell areas.

Table 4 shows the quality measures for the cell particle images. Both images were segmented with good quality. An overall error below one percent shows the potential of our method.

The first benchmark acted as a proof of concept for our automatically generated processing pipeline. To further demonstrate the usefulness of our approach, we compare benchmark results from the generated workflow with the results from two types of state-of-the-art neural networks.

A convolutional neural network (Venkatesan et al. 2018) was built using three convolutional and pooling layer combinations followed by two fully connected layers. Splitting the one training image as shown in Fig. 14 into 20 \(\times \) 20 pixel windows, this network was trained using a balanced dataset with 807216 samples. Each sample therefore contained the pixel intensities and the information about whether the center pixel with the coordinates (10,10) was part of a cell or not. The dataset was balanced to improve the classification accuracy of the neural network (Kotsiantis 2007) and prevent that the network is biased by one dominating class (Lemnaru and Potolea 2012). Using this dataset, it was possible to achieve an overall accuracy on validation data of 99.65 percent after 50 epochs of training.

For both, for the training of the convolutional neural network, as well as our parameter optimization approach, only one image was used. The difference between the neural network training and the parameter optimization is that our approach does not net a binary representation of the image for this optimization. The presented binary images are only necessary for the testing purpose of this paper. Therefore, it is possible to get a better comparison of these two methodologies in situations where the amount of data is highly limited.

In addition to this sliding window neural network approach, we compared our workflow with the state-of-the-art segmentation methodology called u-net (Sommer et al. 2011). Due to the lack of data, this network could only be trained with 33 images and therefore could only achieve a validation accuracy of 88.85 percent using different data augmentation strategies (Wong et al. 2016).

This comparison might be somewhat biased due to the low accuracy value of the u-net. However, considering that the main benefit of our approach is its applicability even on very small sets of images, this bias reflects real-world use cases, where lack of data is often an issue.

All three of these methods were tested using the same test images which were not used for the training and optimization process. The optional parameter for the size constraint was again set to a minimum value of 200 and a maximum value of 5000.

The results listed in Table 5 are generated out of the average values achieved by the classification of both test images. These measures show an overall detection accuracy which is quite high. All of the tested approaches achieve an accuracy of above 97 percent looking at the quality measures.

The sliding window neural network was be able to outperform the evolution strategy. However, it takes way more calculations to classify each pixel within such a high-resolution image. The neural network in a sliding window artificially extends the image to be able to also classify the pixels on the borders. This methodology requires 4214784 classifications for one 2028x2048 pixel image. Concerning the extensive amount of calculations needed for each of these classifications due to the number of weights used in the neural network architecture, it is clear that an application of several processing filters is still more efficient, albeit with a slight loss of accuracy. Considering that the overall accuracy of our method on this data was higher than the one achieved using the u-net, it can be said that this method is not recommended for such small amounts of data.

When inspecting the final detection results in Fig. 15, we can see that the only significant difference between the results from the different methods is in the upper right corner where the sliding window delivered a slightly better segmentation than the evolution strategy. This explains the 1 percent increase in accuracy.

Even though the sliding window neural network slightly outperformed the evolution strategy in terms of quality, the high amount of preprocessing needed to get an appropriate model for the classification arguably does not justify this increase in accuracy. In this example, the processing needed to get an appropriate model from the labeled image took 8.5 minutes, not including the time which is necessary to create the labeled windows out of the manually segmented image. The whole evolution strategy process took 7.5 minutes on the same hardware. Each image classification using the trained sliding window neural network takes nearly a minute due to the high amount of classifications, whereas the application of the generated set of algorithms only took 7 seconds even when all algorithms were applied together which represents the worst case.