Machine Vision and Navigation

Image sensors used in current machine vision systems suffer from low dynamic range and poor colour constancy and are brittle and unmalleable, limiting their use in applications for which there will be considerable demand in the future. Most approaches aiming to resolve these inadequacies focus on developing improvements in the lighting, software (processing algorithms) or hardware surrounding the photosensor such as the filters. Other strategies involve changing the architecture of the image sensor and the photosensing material; both have experienced recent success. Although they are yet to break fully into the market, image sensors developed from alternative solution-processed materials such as organic semiconductors and organohalide perovskites have immense potential to address the above issues and to ‘disrupt’ machine vision technology.

This chapter describes a new passive vision sensor based on inspirations taken from Nature. The sensor has 360^∘ horizontal field of view, like some insects, and functions inspired by the peripheral/foveal vision cooperation typical to visual perception in vertebrates. To provide the ability of rapid object detection the sensor exploits a catadioptric camera, while a rotating perspective camera makes it possible to measure distances focusing attention on an already detected object. The sensor is designed as a self-contained perception unit and can be used on various mobile robots, even those that have very limited computing power. This autonomy is achieved by employing a single-board embedded computer with parallel processing capabilities. The chapter describes the design of the new sensor, and the software enabling real-time processing of the acquired images and cooperation between the omnidirectional part and the perspective camera. We demonstrate also that the basic software modules of this sensor may be exploited to further implement functions supporting mobile robot navigation.

Robust scanning technologies that offer 3D view of the world in real time are critical for situational awareness and safe operation of robotic and autonomous systems. Color and depth sensing technologies play an important role in localization and navigation in unstructured environments. Most often, sensor technology must be able to deal with factors such as objects that have low textures or objects that are dynamic, soft, and deformable. Adding intelligence to the imaging system has great potential in simplifying some of the problems. This chapter discusses the important role of scanning technologies in the development of trusted autonomous systems for robotic and machine vision with an outlook for areas that need further research and development. We start with a review of sensor technologies for specific environments including autonomous systems, mining, medical, social, aerial, and marine robotics. Special focus is on the selection of a particular scanning technology to deal with constrained or unconstrained environments. Fundamentals, advantages, and limitations of color and depth (RGB-D) technologies such as stereo vision, time of flight, structured light, and shape from shadow are discussed in detail. Strategies to deal with lighting, color constancy, occlusions, scattering, haze, and multiple reflections are discussed. This chapter also introduces the latest developments in this area by discussing the potential of emerging technologies, such as dynamic vision and focus-induced photoluminescence.

The urgent need to create video sensors and processors for parallel (simultaneous by pixel) image processing with advanced functionality and multichannel picture outputs is shown in the chapter. We consider perspective spheres and areas of application of such sensor processors, in particular, for hardware high-performance architectures of neural networks, convolutional neural structures, parallel matrix-matrix multipliers, and special processor systems. We show and analyze the theoretical foundations, the mathematical apparatus of the matrix and continuous logic, and their basic operations, show their functional completeness, and evaluate their advantages and prospects for application in the design of biologically inspired devices and systems for processing and analysis of array signals. We show that some functions of continuous logic, including operations of normalized equivalence of vector and matrix signals, the operation of a limited difference in continuous logic, are a powerful basis for designing improved smart micro-cells for analog transformations and analog-digital encodings. In the next sections of the chapter, we consider in more detail the design and modeling aspects of such micro-basic cells and continuously logical high-speed ADCs based on them. The picture-type ADC consists of an array of parallel operating channels, each of which is a basic microcell or a set of them. The basic microcell of a 2D ADC structure consists of several digital-analog cells (DC), which is made on 15–35 CMOS transistors. For an iterative type ADC, only one DC cell is needed, which is DC-(G), and it additionally contains a sample and hold device (SHD). In this case, the entire base microcell can be performed on just 35 CMOS transistors. A single ADC channel cell with iteration has a serial-parallel output code. For a non-iterative-type ADC, its base microcell may consist of such a quantity of DC, which depends on the digit capacity of the code. To simulate the proposed schemes, we used OrCAD, and the results are presented below. Conversion time with 6–8-bit binary codes or Gray codes and an input photocurrent range of 0.1–24 μA is 20–30 ns at a supply voltage of 1.8–3.3 V. If the maximum input current is 4 μA, then for ADC with iteration, total power consumption was only 50–100 μW. Low power consumption at such a supply voltage and good dynamic characteristics (the digitization frequency even for 1.5 μm CMOS technologies is 40–50 MHz) shows good prospects since the structure of the linear array of ADCs and its microcells is very simple. The conversion frequency can be increased ten times with more advanced CMOS transistors. Thus, the proposed ADC based on CL BC and CM are promising for creating photoelectric structures with matrix operands, digital optoelectronic processors, linear and matrix image processors (IP), and other neural-like structures that are necessary for neural networks and neuro fuzzy controllers. In the chapter, we consider a generalized method of designing devices for nonlinear transformation of the photocurrent intensity using a set of similar basic modified cells and their circuits implemented using traditional CMOS technology. To implement the required nonlinear transformation function, we use the decomposition method. The type of synthesized functions is determined by the choice of suitable parameters, which are specified as constants or as parameters with which you can choose or change the type of nonlinear transformation. In this chapter, we also show the need for different types of nonlinear intensity conversion of photocurrents and different codes (gray, binary) for AD conversion in such parallel sensor devices and systems, especially for implementing various types of activation functions in hardware implementations of neural networks, consider the use of such parallel matrix arrays to create progressive IP and neural networks (NN). The cells offered by us have a low supply voltage of 1.8–3.3 V, low power consumption (microwatts), the conversion time is less than 1 μs, and consist of several dozen transistors. We also consider the cells for the implementation of various neuron activation functions in neural networks and transient nonlinear conversion with characteristics of S-, N-, and λ-types. In conclusion, we make estimates and show the prospects for such approaches to the design of sensor processors.

Robotic total stations are modern geodetic multi-sensor systems measuring horizontal and vertical angles as well as distances using time-of-flight methods, thus delivering 3D coordinates for static as well as moving objects. Automatic target detection (by rough and fine pointing techniques) and tracking are standard techniques if the objects are signalized with reflectors, and the total station is motorized. Nowadays, these instruments are additionally equipped with one or two cameras to generate images mainly for documentation purposes. This paves the way to detect and track objects that are not signalized by reflectors. Photogrammetric techniques such as SURF (speeded-up robust feature) or SIFT (scale-invariant feature transform) are applied for the detection of special, recognizable object features in the images. The pixel coordinates of these features result in vertical and horizontal angles if the parallaxes between the camera optical center and the total station origin are known or calibrated. If the features are extracted in a sequence of images, the movement of any object can be tracked automatically. For the position determination, reflector-less distance measurement from the total station to the object is additionally required. Until now, this was realized only for static objects. In this contribution, an example of a kinematic application is also shown. The quality of these tracking procedures may be verified by an instrument of higher accuracy. At the end of this contribution, a procedure using laser tracker is presented.

The chapter is devoted to the actual problem of navigating mobile autonomous robots on unknown terrain in the absence of GPS. Such a problem is considered solved if the robot is capable to detect landmark and estimate own coordinates relative to the landmark. A reliable method for solving the problem is the simultaneous use of several measuring systems operating on different physical principles. In classical radar, the reliable detection of the echo signals from immovable landmark, which differ little from the echo signals that are reflected from the surrounding area, is impossible. Comparison of such signals is carried out in the chapter for various terrains at different lengths of electromagnetic waves. It is found that the only difference between them is the possible amplitude jump of signal, reflected from the landmark. This jump occurs during the movement of the robot or scanning the space by the robot antenna. The probability of detecting such a jump, the accuracy of the amplitude estimation, and the speed of the device operation are analyzed in the chapter based on the developed system of stochastic differential equations.

This chapter overviews different machine vision systems in agricultural applications. Several different applications are presented, but a machine vision system which estimates fruit yield, an example of an orchard management application, is discussed at length. From the farmer’s perspective, an early yield prediction serves as an early revenue estimate. From this prediction, resources, such as employees and storage space, can more efficiently be allocated, and future seasons can be better planned. The yield estimate is accomplished using a camera with a color filter that isolates the blossoms on a tree when the tree is in its full blossom. The blossoms in the resulting image can be counted and the yield estimated. An estimate during the blossom period, as compared to when the fruit has begun to mature, provides a crop yield prediction several months in advance. Discussed as well, in this chapter, is a machine vision system which navigates a robot through orchard rows. This system can be used in conjunction with the yield estimation system, but it has additional applications such as incorporating a water or pesticide system, which can treat the trees as it passes by. To be effective, this type of system, must consider the operating scene as it can limit or constraint the system effectiveness. Such systems tend to be unique to the operating environment.

Stereoscopic vision systems (SVS) allow performing digital scene reconstruction through cameras. SVS process the obtained images by their cameras from a three-dimensional scene, identifying similarities between the images that correspond to the same scene and, finally, performing a reconstruction process. SVS can be differentiated by the number of cameras and the geometry used in their design. In this chapter, each SVS is classified by its number of cameras, so that it can be divided into binocular vision systems, where there are SVS of two cameras and multivision systems that consist of SVS of three or more cameras. The aim of this chapter is to provide useful information to students, teachers, and researchers who want to learn about the different methods and applications of SVS used in the industry and current research topics. Also, it will be useful for everyone who wants to implement an SVS and needs an introduction to several available options to use the most convenient according to the application.

In the global world of robotics, robots have missions to achieve interactively with human environments and online learning. Practically, for null error of a robot’s achievement, robotics systems should be provided with minimal certain information in advance. This is crucial for any high-performance robotics systems. For instance, a medical assistive robot in a medical operation room has to be able to learn as precisely as possible about tissues. In the last few years, visual simultaneous localization and mapping (VSLAM) has become an open rich area in mobile robotics research for developing truly autonomous robots. VSLAM aims to estimate simultaneously the robot pose and 3D structure of the scene through a set of matched correspondences and features extracted from multiple images. To increase efficiency, the majority of online VSLAM algorithms use Kalman filter (KF) which is a Gaussian Bayesian filter, to merge the uncertainties in the Cartesian motion and observation model. This chapter concentrates on stereo vision noise source that results in 3D reconstruction of the scene, strategies for image filtering, and feature extraction. Modern and advanced techniques, for example, KF, extended Kalman filter (EKF), and unscented Kalman filter (UKF), will be presented in detail with practical examples in the field of robotics vision research.

Pose estimation requires optimally estimating translation and rotation. Here, we focus on rotation, since it involves nonlinear analysis. We show that the computation can be done systematically if we exploit the fact that the set of rotations forms a group of transformations, called the “special orthogonal group,” denoted by SO(3). For optimizing a function J(R) of a rotation R, we need not parameterize R in any way; we only need to add to R a small rotation so that J(R) decreases. To this end, we define a linear space spanned by infinitesimal rotations, called the “Lie algebra” of SO(3). We describe the computational procedure for minimizing J(R) based on the Lie algebra formulation, which we call the “Lie algebra method.” We apply it to three computer vision problems: (1) Given two sets of 3D points, we optimally estimate the translation and rotation between them in the presence of inhomogeneous anisotropic noise. (2) Given corresponding points between two images, we optimally compute the fundamental matrix. (3) We describe the procedure of bundle adjustment for computing, from images of multiple points in the scene taken by multiple cameras, the 3D locations of all the points and the postures of all the cameras as well as their internal parameters. Finally, we overview the role of Lie algebra in various computer vision applications.

This chapter presents a methodology for the accurate generation and tracking of closed trajectories over arbitrary, large surfaces of unknown geometry, using a robot whose control is based on the use of a non-calibrated vision system. This capability can be applied to relevant industrial robotic maneuvers, like the welding or cutting of commercially-available metal plates. The proposed technique is based on a calibration-free, vision-based robot control methodology referred to as camera-space manipulation. This is combined with a geodesic-mapping approach, with the purpose of generating and tracking a trajectory stored as a CAD model, over an arbitrarily curved surface, along a user-defined position and orientation. In the context of applications to large surfaces, the maneuver precision of the positioning and path-tracking tasks depend on several aspects like camera resolution and mapping procedure, which has the potential of introducing distortion, especially in non-developable surfaces. In terms of the mapping procedure, this chapter discusses two options, referred to as modified geodesic mapping and virtual-projection mapping. A measure used to diminish the distortion caused by the mapping procedure and a technique for achieving closure of a given closed-path, when this is tracked over large, non-developable surfaces, are presented herein. The performance of the proposed methodology was evaluated using an industrial robot with a large workspace, combined with structured lighting used to reduce the complexity of the image analysis process.

In this chapter, a unified passivity-based visual servoing control structure considering a vision system mounted on the robot is presented. This controller is suitable to be applied for robotic arms, mobile robots, as well as mobile manipulators. The proposed control law makes the robot able to perform a moving target tracking in its workspace. Taking advantage of the passivity properties of the control system and considering exact knowledge of the target velocity, the asymptotic convergence of the control errors to zero is proved. Later, a robustness analysis is carried out based on L ₂-gain performance, thus proving that control errors are ultimately bounded even when bounded errors exist in the estimation of the target velocity. Both numerical simulation and experimental results illustrate the performance of the algorithm in a robotic manipulator, a mobile robot, and also a mobile manipulator.

Robotic group collaboration in a densely cluttered terrain is one of the main problems in mobile robotics control. The chapter describes the basic set of tasks solved in model of robotic group behavior during the distributed search of an object (goal) with the parallel mapping. Navigation scheme uses the benefits of authors original technical vision system (TVS) based on dynamic triangulation principles. According to the TVS, output data were implemented fuzzy logic rules of resolution stabilization for improving the data exchange. Modified dynamic communication network model and implemented propagation of information with a feedback method for data exchange inside the robotic group. For forming the continuous and energy saving trajectory, authors are proposing to use two-steps post processing method of path planning with polygon approximation. Combination of our collective TVS scans fusion and modified dynamic data exchange network forming method with dovetailing of the known path planning methods can improve the robotic motion planning and navigation in unknown cluttered terrain.

This chapter proposes a hierarchical navigation system combining the benefits of perception space local planning and allocentric global planning. Perception space permits computationally efficient 3D collision checking, enabling safe navigation in environments that do not meet the conditions assumed by traditional navigation systems based on planar laser scans. Contributions include approaches for scoring and collision checking trajectories in perception space. Benchmarking results show the advantages of perception space collision checking over popular alternatives in the context of real-time local planning. Simulated experiments with multiple robotic platforms in several environments demonstrate the importance of 3D collision checking and the utility of a mixed representation hierarchical navigation system.

In recent years, the idea of autonomous vehicles has taken on importance since some automobile companies have decided to develop their own autonomous cars. However, not every “autonomous car” is fully autonomous since there are different levels of autonomy. Currently, there is a variety of studies and a great deal of research about autonomous vehicles and on how to achieve full autonomy; even more, these are not limited to cars, but also include research surrounding mobile robots, drones, remotely operated vehicles (ROVs), and others. All these robots or vehicles have the same principles, in addition to having the same basics of the hardware. However, not the same can be said about the software because every solution requires unique algorithms for their data processing. In this chapter, the most important topics related to autonomous vehicles are explained as clearly as possible. This chapter covers from its main concepts to path planning, going through the basic components that an autonomous vehicle must have, all the way to the perception it has of its environment, including the identification of obstacles, signs and routes. Then, inquiry will be made into the most commonly used hardware for the development of these vehicles. In the last part of this chapter, the case study “Intelligent Transportation Scheme for Autonomous Vehicles in Smart Campus” is incorporated in order to help illustrate the goal of this chapter. Finally, an insight is included on how the innovation on business models can and will change the future of vehicles.

The analysis of the known methods and navigation systems of flying robots (FR) was performed. Among them, because of a number of shown below reasons, the most preferable are passive combined correlation-extreme systems which implement the survey-comparative method. A basic model for the radiometric channel operation of the correlation-extreme navigation systems is proposed. The factors that lead to distortions of the decisive function formed by the combined correlation-extreme navigation system of flying robots in a developed infrastructure are allocated. A solution of the problem of autonomous low-flying flying robot navigation in a developed infrastructure using the radiometric channel extreme correlation navigation systems (CENS), when the size of the solid angle of associated object is much larger than the size of the partial antenna directivity diagram (ADD), is proposed. The appearance possibility of spurious objects that are close in parameters (geometric dimensions and brightness) to the anchor object, depending on the current image sight geometry formed by the optoelectronic channel of the combined CENS, is taken into account.

Aerial video captured from an airborne platform has an expanding range of applications including scene understanding, photogrammetry, surveying and mapping, traffic monitoring, bridge and civil infrastructure inspection, architecture and construction, delivery, disaster and emergency response, news and film, precision agriculture, and environmental monitoring and conservation. Some of the challenges in analyzing aerial video to track pedestrians, vehicles, and objects include small object size, relative motion of the object and platform, sensor jitter, and quality of imaging optics. An analytic image stabilization approach is described in this chapter where pixel information from the focal plane of the camera is stabilized and georegistered in a global reference frame. The aerial video is stabilized to maintain a fixed relative displacement between the moving platform and the scene. The proposed algorithm can be used to stabilize aerial imagery even when the available GPS and IMU measurements from the platform and sensor are inaccurate and noisy. Camera 3D poses are optimized using a homography-based robust cost function, but unlike most existing methods, the homography transformations are estimated without using any image-to-image estimation techniques. We derive a direct closed-form analytic expression from 3D camera poses that is robust even in the presence of significant scene parallax (i.e. very tall 3D buildings and man-made or natural structures). A robust non-linear least squares cost function is used to deal with outliers and speeds up computation by avoiding the use of RANdom SAmple Consensus (RANSAC). The proposed method and its efficiency is validated using several datasets and scenarios including DARPA Video and Image Retrieval and Analysis Tool (VIRAT) and high resolution wide area motion imagery (WAMI). scenarios.

It is well-known that there has been a considerable progress in multimedia technologies during the last decades, namely TV, photography, sound and video recording, communication systems, etc., which came into the world during at least half of the previous century and were developed as analog systems, and nowadays have been almost completely replaced by digital systems. The aforementioned motivates a deep study of multimedia compression and intensive research in this area. Data compression is concerned with minimization of the number of information carrying units used to represent a given data set. Such smaller representation can be achieved by applying coding algorithms. Coding algorithms can be either lossless algorithms that reconstruct the original data set perfectly or lossy algorithms that reconstruct a close representation of the original data set. Both methods can be used together to achieve higher compression ratios. Lossless compression methods can either exploit statistical structure of the data or compress the data by building a dictionary that uses fewer symbols for each string that appears on the data set. Lossy compression, on the other hand, uses a mathematical transform that projects the current data set onto the frequency domain. The coefficients obtained from the transform are quantized and stored. The quantized coefficients require less space to be stored. This chapter is focused on the recently published advances in image and video compression to date considering the use of the integer discrete cosine transform (IDCT), wavelet transforms, and fovea centralis.

This chapter presents a method to solve the stairway localization and recognition problem for both indoor and outdoor cases by using a convolutional neural network technique. To blind and visually impaired persons, these assistive technology application has an important impact on their daily life. The algorithm should be able to solve the problem of stair classification for both cases, indoor and outdoor scenes. The proposed idea describes the strategy for introducing an affordable method that can recognize stairways without taking into account the environments. First, this method uses stair features to classify images by using convolutional neural networks. Second, stairway candidate is extracted by using the Gabor filter a linear filter. Third, the set of lines that belong to the ground plane are removed by using the behavioral distance measurement between two consecutive frames. Finally, from this step, we extract the tread depth and the riser height of the stairways.

This chapter comprises the review of new methods of phase triangulation, which allow 3D geometry measurements under the conditions of arbitrary measured object surface light-scattering properties, varying measurement setting external illumination, and limited depth of field of optical elements of the source and receiver of optical radiation. The application of the proposed methods provides higher metrological characteristics of measuring systems and expands the functionality and the range of application of optical-electronic systems for geometric control in the production environment.

Blown powder deposition is an additive manufacturing procedure and has the ability to fabricate complicated and intricate geometries with excellent material properties. Reliable fabrication of complicated shapes and geometries necessitates precise control over the fabrication process. In order to do so, process monitoring tools capable of visualizing various phenomena that occur during the deposition process are needed. Knowledge of process dynamics is critical in optimizing and developing robust and effective deposition procedures.

The work presented in the current chapter involves the incorporation of an Infra-Red (IR) camera as a vision-based monitoring tool for blown powder deposition process. The data processing methodology necessary for analyzing IR data is also presented. In this chapter, the thermal history of the process was captured under different powder feed settings. These deposition processes were performed under the control of vision-based closed loop control systems. Using the IR camera, the influence of the control systems was captured as the thermal history of the deposits. This data was analyzed for tracking changes in the area of the material near solidus temperature.

The later section of the chapter focuses on further dissecting thermographic data to identify the material above the solidus temperature. Image processing techniques related to edge detection were used to identify these regions. The IR camera data was also used to track the regions of interest through the deposition and make other characteristic observations pertaining to phase change in relation to thin wall geometry.

Accurate fault detection and isolation in machines requires image processing of measurement signals which are contaminated with noise. Typically, faults are revealed by sharp trend shifts in the signals and these trend shifts should be preserved during image processing. Linear filters can smooth out the sharp trend shifts while removing noise. However, nonlinear filters such as the weighted recursive median (WRM) filters show good noise reduction while preserving key image features if their integer weights are determined optimally. The ant colony optimization (ACO) method coupled with local search to calculate the integer weights of WRM filters. It is found that the filter weight optimization problem is mathematically equivalent to the quadratic assignment problem which can be solved by ACO. Optimal parameters for the ACO are found using numerical experiments. The WRM filter is demonstrated for abrupt and gradual faults in gas turbine machine and is found to yield noise reduction of 52–64% for simulated noisy data considered in this chapter.

The general objective addresses the challenge of the miniaturized microwave characterization of nanodevices. The method is based on a measurement setup that consists of a vector network analyzer (VNA) connected through coaxial cables to miniaturized homemade coplanar waveguide (CPW) probes (one signal contact and two ground contacts), which are themselves mounted on three-axis piezoelectric nanomanipulators SmarAct™. The device under test (DUT) is positioned on a sample holder equipped also with nanopositioners and a rotation system with μ-degree resolution. The visualization is carried out by a scanning electron microscope (SEM) instead of conventional optics commonly found in usual on-wafer probe stations. This study addresses the challenge related to the control of nanomanipulators in order to ensure precisely the contact between the probe tips and the DUT to be characterized. The DUT is inserted between the central ribbon and the ground planes of the coplanar test structure (width of the central ribbon = 2.3 μm, distance between the central ribbon and the ground planes = 1.8 μm). First, we use classical automatic linear tools to identify the transfer function of a system of three linear nanopositioners along the X, Y, and Z axes. This part allows the precise control of each nanomanipulator using LabVIEW™, with an overshoot of the final value (according to a minimal response time in X and Y) or without an overshoot of the final value (in order to avoid any crashing of the probe tips on the substrate in Z). Second, we propose an angular control methodology (using Matlab™) in order to align the probe tips on the CPW ports of the DUT. Finally, the detection of the points of interest (use of the Harris detector) allows one to determine the set point value of each linear nanopositioner X, Y, and Z. These three steps ensure the precise positioning of the probe tips to ensure accurate microwave characterization of the DUT.

This paper presents the development of user-friendly design and training tool for convolutional neural networks (CNNs) and support vector machines (SVMs) as an application development environment based on MATLAB. As the first test trial, an application of deep CNN (DCNN) for anomaly detection is developed and trained using a large number of images to distinguish undesirable small defects such as crack, burr, protrusion, chipping, spot, and fracture phenomena that occur in the production process of resin molded articles. Then, as the second test trial, a SVM incorporated with the AlexNet and another SVM incorporated with our original sssNet are, respectively, designed and trained to classify sample images into accepting as OK or rejecting as NG categories with high categorization rate. In the case of these SVMs, the training can be conducted by using only images of OK category. The AlexNet and the sssNet are different types of DCNNs, whose compressed feature vectors have 4096 and 32 elements, respectively. The two lengths of compressed feature vectors are used as the inputs for the two types of SVMs, respectively. The usability and operability of the developed design and training tool for DCNNs and SVMs are demonstrated and evaluated through training and classification experiments.

Due to the increase of frequency and weight of commercial ship trips in waterways, bridges are more vulnerable to ship–bridge collision accidents. There are plenty of reports of such cases all over the world, leading to millions of economic losses. For ancient bridges, irreparable damage might come in the sense of cultural value except for economic losses. The development of computer vision-based technology provides an active defense method to prevent damage in advance. This chapter presents a computer vision-based method for ship–bridge collision assessment and warning for an ancient arch bridge across the Beijing–Hangzhou Grand Canal in Hangzhou, China. The structural characteristics and current status of the arch bridge are analyzed. The traffic volume and parameters of passing ships including the velocity and weight are investigated. The water area in both sides of the bridge is divided into three different security districts corresponding to different warning levels. Image processing techniques are exploited to identify the types of ships for tracking, and the risk of ship–bridge collision is assessed.