Image Processing, Analysis and Machine Vision

Vision allows humans to perceive and understand the world surrounding them. Computer vision aims to duplicate the effect of human vision by electronically perceiving and understanding an image. Giving computers the ability to see is not an easy task — we live in a three-dimensional (3D) world, and when computers try to analyse objects in 3D space, available visual sensors (e.g. TV cameras) usually give two-dimensional (2D) images, and this projection to a lower number of dimensions incurs an enormous loss of information. Dynamic scenes such as those to which we are accustomed, with moving objects or a moving camera, make computer vision even more complicated.

In this chapter some useful concepts and mathematical tools will be introduced which will be used throughout the book. Some readers with a less mathematical background might find some parts difficult to follow; in this case, skip the mathematical details and concentrate on the intuitive meaning of the basic concepts, which are emphasised in the text. This approach will not affect an understanding of the book.

Data and an algorithm are the two basic parts of any program, and they axe related to each other — data organization often considerably affect the simplicity of the selection and the implementation of an algorithm. The choice of data structures is therefore a fundamental question when writing a program [Wirth 76]. Information about the representation of image data, and the data which can be deduced from them, will here be introduced before explaining different image processing methods. Relations between different types of representations of image data will then be clearer.

Pre-processing is a common name for operations with images at the lowest level of abstraction — both input and output are intensity images. These iconic images are of the same kind as the original data captured by the sensor, with an intensity image usually represented by a matrix of image function values (brightnesses). The aim of pre-processing is an improvement of the image data that suppresses unwilling distortions or enhances some image features important for further processing, although geometric transformations of images (e.g. rotation, scaling, translation) are classified among pre-processing methods here since similar techniques are used.

Image segmentation is one of the most important steps leading to the analysis of processed image data — its main goal is to divide an image into parts that have a strong correlation with objects or areas of the real world contained in the image. We may aim for complete segmentation, which results in a set of disjoint regions uniquely corresponding with objects in the input image, or for partial segmentation, in which regions do not correspond directly with image objects. To achieve a complete segmentation, cooperation with higher processing levels which use specific knowledge of the problem domain is necessary. However, there is a whole class of segmentation problems that can be successfully solved using lower level processing only. In this case, the image commonly consists of contrasted objects located on a uniform background — simple assembly tasks, blood cells, printed characters, etc. Here, a simple global approach can be used and the complete segmentation of an image into objects and background can be obtained. Such processing is context independent; no object-related model is used, and no knowledge about expected segmentation results contributes to the final segmentation.

The last chapter was devoted to image segmentation methods which showed how to construct homogeneous regions of images and/or their boundaries. Recognition of image regions is one of the most important steps on the way to understanding image data, and requires an exact region description in a form suitable for a classifier (Chapter 7). This description step should generate a numeric feature vector, or a non-numeric syntactic description word, which characterizes properties (for example, shape) of the described region. Region description is the third of the four levels given in Chapter 3, implying that the description already comprises some abstraction — for example, 3D real objects can be represented in a 2D plane. Nevertheless, shape properties used for object description are usually computed in two dimensions. If we are interested in a 3D object description, we have to process at least two images of the same object taken from different viewpoints (stereo vision), or derive the 3D shape from a sequence of images if the object is in motion. A 2D shape representation is sufficient in the majority of practical applications, but if 3D information is necessary — if, say, a 3D object reconstruction is the processing goal, or the 3D characteristics bear the important information — the object description task is much more difficult; these topics are introduced in Chapter 9. In the following sections we will limit our discussion to 2D shape features and proceed under the assumption that described objects result from the image segmentation process.

Even the simplest machine vision tasks cannot be solved without the help of recognition. Pattern recognition is used for region and object classification, and basic methods of pattern recognition must be understood in order to study more complex machine vision processes.

Image understanding requires mutual interaction of processing steps. The building blocks necessary for image understanding have been presented in earlier chapters — now an internal image model must be built that represents the machine vision syste’s concept about the processed image of the world.

A number of techniques have been presented so far that perform a range of tasks of varying complexity; some are specific to raw images, such as edge detection or the more elaborate region splitting and merging algorithms. Others are more abstract (or general purpose), such as the studies of graphical representations and pattern recognition techniques. What has been overlooked hitherto, though, is the (perhaps obvious) observation that the best known vision system, our own, is geared specifically to dealing with the 3D world and as yet the gap between images and the real world of 3D objects, with all their problems of relative depth, occlusion etc. has not been seriously examined.

Mathematical morphology, which started to develop in the late Sixties, stands as a relatively separate part of image analysis. Its main protagonists were Matheron [Matheron 67] and Serra [Serra 82], whose monographs are highly mathematical books. Here, we shall present a simple explanation of this topic.

Image processing and analysis based on continuous or discrete image transforms is a classic processing technique. Image transforms are widely used in image filtering, image data compression, image description, etc.; they were actively studied at the end of the Sixties and in the beginning of the Seventies. This research was highly motivated by space flight achievements; first, an efficient method of image data transmission was needed, and second, transmitted images were often received locally or globally corrupted. Both analogue and discrete methods were investigated — analogue optical methods suffered from a lack of suitable high-resolution storage material, and the rapid development of digital computer technology slowed analogue research in the middle of the Seventies. However, much research is now being devoted to optical methods again, especially with new technological achievements in developing high-resolution optical storage materials. The main focus is in possible real-time image processing.

Image processing is often very difficult due to the large amounts of data used to represent an image. Technology permits ever-increasing image resolution (spatially and in grey levels), and increasing numbers of spectral bands, and there is a consequent need to limit the resulting data volume. Consider an example from the remote sensing domain where image data compression is a very serious problem. A Landsat D satellite broadcasts 85 × 10⁶ bits of data every second and a typical image from one pass consists of 6100 × 6100 pixels in 7 spectral bands — in other words 260 megabytes of image data. A Japanese Advanced Earth Observing Satellite (ADEOS) will be launched in 1994 with the capability of observing the Earth’s surface with a spatial resolution of 8 metres for the polychromatic band and 16 metres for the multispectral bands. The transmitted data rate is expected to be 120 Mbps [Arai 90]. Thus the amount of storage media needed for archiving of such remotely sensed data is enormous. One possible way how to decrease the necessary amount of storage is to work with compressed image data.

Texture is a term that refers to properties that represent the surface of an object; it is widely used, and perhaps intuitively obvious, but has no precise definition due to its wide variability. We might define texture as something consisting of mutually related elements; therefore we are considering a group of pixels (a texture primitive or texture element) and the texture described is highly dependent on the number considered (the texture scale) [Haralick 79]. Examples are shown in Figure 13.1; dog fur, grass, river pebbles, cork, chequered textile, and knitted fabric. Many other examples can be found in [Brodatz 66].

In recent years, interest in motion processing has increased with advances in motion analysis methodology and processing capabilities. The usual input to a motion analysis system is an image sequence, with a corresponding increase in the amount of processed data. Motion analysis is often connected with real-time analysis, for example, for robot navigation. Another common motion analysis problem is to obtain comprehensive information about objects present in the scene, including moving and static objects. Detecting 3D shape and relative depth from motion are also fast-developing fields — these issues are considered in Chapter 9.