Ship feature recognition methods for deep learning in complex marine environments

At present, in the industry, target feature learning and recognition technology has made great progress. For example, leading technological companies such as China SenseTime Technology and Questyle Technology have incorporated classic algorithms into engineering applications. However, there is no fundamental breakthrough in innovative research based on classic algorithms. Analyzing the principle of classic algorithms, YOLO and SSD are both very concise. They do not perform deep fitting on the sample dataset, though they do only on the linear connection of multidimensional data sample points. Therefore, the meaning of the test is only in the test set and the data sample. The numerical value of the points is simple and relative; therefore, only a single stage allows them to have a fast detection effect relative to Faster RCNN.

Furthermore, target recognition has been listed as a trending research topic in various areas. Shi et al. [38] proposed an accurate and effective target detection method in the literature, which is called the feature fusion and enhancement for single shot detection (FFESSD) model. It is verified through experiments, proving that when the input rate is 54.3 FPS and the input image size reaches 300 × 300, the average accuracy rate of the FFESSD can reach 79.1%. When the input rate is 30.2 FPS and the input image size is 512 × 512, the average accuracy rate can reach 81.8%, which is significantly better than the conventional SSD model, the deconvolution single shot detector (DSSD), feature fusion SSD (FSSD) and other advanced detectors. Scholars, such as Liu et al. [25] proposed a new type of SSD model in the literature. Through verification in the MICCAI challenge, it was found to have the fastest and best accuracy and recall rate. In general, it has excellent performance in polyp detection. The target detection method in SAR ship detection has also become a very prevalent study point. Scholars, such as Zhang et al. [53] proposed an improved Faster-RCNN method in the literature. The method could improve the detection results of small-scale distribution ships, while simultaneously improving the recall rate, which could provide a high-resolution remote sensing image-based detection method for offshore and inland ships. Chang et al. [5] proposed a YOLOv2-reduced method in the literature. Through testing and verification on the SAR (SSDD) and DSSD, it was found that compared with Faster R-CNN, the YOLOv2-reduced method improved the accuracy of ship detection and substantially reduced the calculation time. Zhang et al. [54] proposed a high-speed ship detection method for SAR images based on a convolutional neural network (ABYI, G-CNN for short) in the literature. The experimental results show that the detection speed of G-CNN is faster than that of other existing methods, such as faster regional convolutional neural networks (Faster R-CNN), SSD, and the YOLO method. Li et al. [21] proposed a SAR-based context-based feature fusion single-shot multi-box detector (CBFF-SSD) framework in the literature and adopted parallel deep learning processing with multiple neural processing units. The high-efficiency hardware architecture (NPU) of the processor is composed of two-dimensional processing elements (PE), which can calculate multiple output element maps at the same time. Experiments show that it can be further optimized in the detection of small targets, reducing the computational complexity and parameter size. Scholars, such as Ivan Lorencin used AI algorithms in the literature to identify marine targets from the air [16] and classified marine targets into ships and other targets through classification.

In 2016, Liu et al. [26] proposed a deep neural network method called SSD, which uses a single object to detect objects in images, according to the literature. It discretizes the output space of the bounding box into a set of default boxes at different aspect ratios and proportions for each function graph position. For a 300 × 300 input, SSD reaches a 74.3% mAP. When VOC2007 is tested at 59 FPS, the SSD reaches a 76.9% mAP for 512 × 512 inputs, which is better than the latest Faster R-CNN model. Fu et al. [10] proposed a method to introduce other contexts into the latest general object detection in the literature in 2017. In the PASCAL VOC and COCO tests, the DSSD of a 513 × 513 input reaches an 81.5% mAP in the VOC2007 test, an 80.0% mAP in the VOC2012 test, and a 33.2% mAP in the COCO test, which is better than the latest R-FCN method in each dataset. Based on the traditional SSD, literature [17] obtains better generalizability by changing the structure to make it close to the classifier network instead of adding layers. For the Pascal VOC 2007 test set, when using the VOC 2007 and VOC 2012 training sets, the mAP is 78.5% at a 300 × 300 input and a rate of 35.0 FPS. With an input of 512 × 512 and 16.6 FPS, the mAP can reach 80.8%. The accuracy is better than those of the traditional SSD, YOLO, Faster-RCNN and RFCN, and it is faster than the RCNN and RFCN accuracies. Reference [48] also proposed a context-aware single-shot multi-frame object detector (CSSD) based on a traditional SSD. In addition, the reference combined a high detection accuracy and real-time speed and experimentally demonstrated that CSSD can achieve better detection efficiency, especially when detecting small objects. The literature [4] proposed a multilevel feature fusion method for small object detection. The experimental results show that the mAP obtained by the multilevel feature fusion method on PASCAL VOC2007 is 2–3% higher than the baseline SSD, especially for detecting small objects. Their speeds are 43 FPS and 40 FPS, which are better than the 29.4 and 26.4 FPS of the latest DSSD. Reference [24] proposed a novel method based on pyramids and an SSD, called the function single shot detector. In the Pascal VOC 2007 test, the network can reach an 82.7 mAP with an input size of 300 × 300 and a speed of 65.8 FP. The performance on other test sets is also better than those of the other datasets. With multiscale context fusion technology, reference [7] proposes the WeaveNet method, which greatly simplifies the structure. The PASCAL VOC 2007 and PASCAL VOC 2012 tests show that WeaveNet brings vast performance improvements.

To suppress the detection frame that eliminates the number of repeated times throughout the process of optimizing the computing power of a large amount of data, we propose an optimization algorithm using the SSD network topology. The algorithm is a deep learning algorithm based on a feedforward CNN optimized computing power, which arbitrarily identifies the picture boundary according to the size of the target picture. We used a feature extractor to extract features from network images and compared the VGG-16 network with a convolutional neural network (CNN) in the experiment. We designed a new feature recognition framework to obtain two completely different feature detectors. As shown in Fig. 2 compared with the traditional convolutional neural network (CNN), it has a stronger feature recognizability.

In Fig. 2 we use image features generated from different spatial and different resolution layers to construct a new feature extractor. Due to the irregular pyramid structure’s characteristics of multilayer images with different sizes and resolutions, it has extremely high robustness requirements for network feature recognizabilities. Based on this, we construct several convolutional feature layers at the network endpoint of each feature map detector of different scales. Due to the different characteristics of the characteristic layer of each convolutional network, the design refers to the output result of the optimized calculation of the matched convolutional network filter on the prediction set. There are c channels in the feature recognition, and the resolution of the image feature is designed as an m × n feature layer. Therefore, the calculation kernel of the 3 × 3 × c convolutional network is used for classification calculation, otherwise the coordinate system of the network feature recognition of the alternating frame is replaced.

Accordingly, the output function of the calculation result of the m × n convolutional neural network prediction model is in the corresponding data result.

First, automatically identifying and labeling the ship tag information is carried out on the target ship picture recognized by the system. Second, the SSD network model is used to input the image with the ship information label for the feature recognizer. Finally, we discuss the output of the detection results after the recognition of the computing power. When the corresponding relationship is clearly specified, the loss function and reverse compensation can be performed on the data point-to-point calculation method Fig. 3.

Network training is divided into three steps:

The DP-SSD network algorithm is used to detect ships in the following steps:

We use the DP-SSD model to measure and evaluate Kaggle's ship dataset. We adopt experimental scenes at different times in the same year and under different complicated maritime navigation environments. For example, Shanghai, Fuzhou City, Fujian Province, and Zhoushan City, Zhejiang Province have a total of more than 10 voyages, collecting more than 50 video sets and more than 2000 experimental pictures. The test results of the detailed datasets are as follows:

First, the Kaggle ship dataset is used to test the feature recognition of more than 1000 ships, including special scenes with different backgrounds and complex environments, such as multitarget ships, coral reefs, lighthouses, etc. Our experiment adaptively selects more than 1000 multitarget and special scene pictures from a total of 25 video detection results in 5 voyages.

We test the source code pixel by pixel based on the results of ship feature recognition in complex maritime scenes constructed by the SSD model. Our experimental results show that the maritime ship identification method constructed by the SSD method has the relative limitations of the real frame boundary. To avoid this situation, we convert the mask to a frame structure.

According to this, all ship targets can be placed in the framework of the model identified by SSD, as shown in Fig. 5.

Our article reveals a new method of ship target recognition and evaluation in complex maritime navigation scenarios. First, we identify whether the sailing scenes of sea ships are true or false in different situations. If any target other than the ship is detected, it is considered a false alarm. If the ship target is not found, the ship picture is not output, and the value range is defined as a false negative. If the ship monitoring system does not find or output a picture with ship characteristics, it is defined as a true negative value. Among them, true positives are defined as TP; false-positives are defined as FP; false negatives are denoted as FN; true negatives are denoted as TN. The measurement results show that there are huge measurement differences in different complex environments at sea in different scenarios. We use different indicators, such as accuracy, recall, specificity, F1 measurement, and F2 measurement, to measure the experiment. The measurement results are as follows: (see Table 1).

Our measurement results propose key evaluation indicators for the fourth part, which can accurately measure the performance of DP-SSD target feature recognition and the adaptability of different scenarios in complex environments. Finally, we use methods, such as accurate recall histograms, mAP, FPS and running time to measure and evaluate the evaluation algorithm of the ship target recognition method in a complex navigation environment.

In summary, the fourth part proposes the key performance indicators to evaluate all of the algorithm models, such as the accurate recall histogram, mAP, FPS and running time.

We use the Python programming language for measurement and evaluation, NVIDIA GeForce RTX 2080Ti graphics card to measure and evaluate ship target recognition data in complex sailing environments, and Intel (R) Core (TM) i9 9900 KF CPU @ 3.6 GHz processor for experimental measurement. All data are shown in parameter Table 2.

Our experiment is based on the training dataset, which has the characteristics of the limitation of the number of sampling targets. Based on this, we use the process of repeated training and sampling, apply the mean method to train the model to improve the reliability of ship feature recognition and reduce the number of overfits, and use different data processing theories to conduct experiments and evaluations, as follows:

We use Faster RCNN (VGG16), YOLO, YOLOv2, YOLOv3, SSD300, SSD512, RefineDet320, RefineDet512, and SSD (DP-SSD300 and DP-SSD512) for measurement and evaluation, as shown in Fig. 6.

The experimental results show that the method of ship target recognition in a complex sailing environment using SSD has the characteristics of classifying and regressing target pictures, while also having the ability to recognize picture pixels. The method of ship target recognition in a complex navigation environment has the strongest detection efficiency for the hardware operating environment and has the ability to identify ship targets in different scenarios, as well as in complex navigation environments. Accordingly, this method is more accurate and effective than two-stage detection methods, such as Faster RCNN.

As shown in the Figs. 1, 2, 3, 4, 5, 6, 7, 8, 9 and 10 on the right side of the above figure indicate that the recall rate is 10–100%, and the step size is 10%. Legend 1 indicates that the recall rate is 10%, 2 indicates that the recall rate is 20%, and 10 indicates that the recall rate is 100%. Therefore, the target detection characteristics of the SSD, which has three backbones, are more accurate than those of the other methods in the ship target recognition method in a complex navigation environment. However, they are slightly lower than the performance characteristics of the two-stage target recognition method.

It can be clearly seen from the results that DP-SSD is considerably better than the other method in terms of target detection performance, whether it is the average accuracy or the detection speed. The root cause analysis is due to the multiscale network proposed in this paper. At the same time, it also has the ability to detect multilevel features, which makes the final detection result more accurate and more efficient.

The target detection of the RefineDet method requires multiple stages for achievement. It first generates several detection frames at the same time, then performs statistical regression classification on the detected target pictures, and finally determines the target recognition result.

Because the RefineDet method is very cumbersome in the process of ship detection, the proposed scheme has comparative advantages over the former in terms of detection accuracy and time.

The confidence test of the ship area is shown in Fig. 8. The different methods and even the different scales in the same method can have a considerable impact on the average accuracy of detection. When the confidence level of the ship recognition area is easy, the average recognition precision of DP-SSD300 and DP-SSD512 proposed in this paper are both up to 90.13%, the performance is superior to 89.21% of YOLO v2, 88.97% of YOLO v2 544 × 544, 87.38% of YOLO v3, 88.15% of SSD300, 76.22% of SSD512 and 89.79% of RefineDet512, and slightly lower than YOLO's 91.99% and 90.37% of RefineDet320. However, when the confidence level is more moderate and harder, the performance of our proposed models cannot achieve the desired results.

Regarding the complexity of all of the algorithms, as illustrated in Fig. 9 the running time of DP-SSD is approximately 0.3 s, which is less than 0.37 s of YOLO, 0.7 s of SSD512 and RefineDet320, though it is slightly better than the 0.34 s of RefineDet512.

The difference between the object and the external environment gives the ship the opportunity to miss the detection situation. In response to this problem, the DP-SSD network we propose adopts a variety of feature combinations in the learning stage to try to make the learning network converge to a target. The environmental differentiation has a robust combination of features to achieve the generalization of testing.

In addition, the multiscale, multi-shape convolution allows the method proposed in this article to obtain more differentiated features, from contours to textures.

The choice of filter is of great importance to the ship’s target. It is manifested at least at the level of connection speed and feature extraction. Specifically, lower-level data features can reflect direct features, such as geometric features, and the higher the level High is, the more hidden the characteristics reflected by the data characteristics are.

The DP-SSD method for real-time ship detection is shown in Fig. 10. When the proposed scheme can correctly detect the ship wave object, the target frame is green. It can be seen from the figure that our method is capable of operating at different distances and scales. We identify single or even multiple targets, which is more feasible for the scheme proposed in this article.