4. Image Processing

In simple systems, the context can be easily extracted from the data coming directly from sensors, be it light intensity, distance, or temperature. This allows the right actuation decision to be easily taken, such as to open the curtains, drive straight, turn left, or start the heater. Unfortunately, context discovery from “raw” images is not as straightforward as that. The reason is that data produced by imaging sensors is only seldom perfect. Look at the image in Fig. 4.1. Suppose the goal is to simply count the miniature figures on it. You can say: “But even a child can do this!” Yes, that’s true. However, there is high chance that a computer will come up with a different and wrong answer. Can you see why? One reason is the shadows; they can make the computer algorithm believe that there are only five objects in the image rather than seven. And we did not even mention more difficult tasks, such as recognizing the figures in the image. Take a more serious, real-life example, and consider a traffic light recognition system in an autonomous car. Think about how the image taken by the vehicle camera will look, if there is mist, it is dark, and the red light is partially covered by dirt. The problem is that for different reasons, images are often corrupted by noise or blur; they suffer from uneven or poor illumination, poor contrast, or view obstruction. Hence, context information carried by image signals remains hidden, making classification and further reasoning impossible. An escape from this impasse is possible if one “cosmetically” enhances the image. This is called preprocessing, a process that encompasses a range of operations applied to a raw signal, with the purpose of extracting, against all odds, a reasonable context. This chapter will discuss different image preprocessing techniques, such as brightness and contrast optimization, thresholding, edge detection, noise filtering, and morphologic operations. The collateral damage that might be caused by these processing techniques is also exposed. Finally, automatic object counting is demonstrated using BLOB analysis.