Method for monitoring and controlling penetration of complex groove welding based on online multi-modal data

In industrial production, there are problems of hand polishing error and the thermal deformation of weldment, resulting in unstable groove welding. In this article, we propose a model for online monitoring of the penetration of complex groove welding. We build an active–passive cooperative vision system based on gas metal arc welding (GMAW). A mapping relationship from multi-modal data to the backside melting width is established. The multi-modal data consists of laser line images and molten pool images. The groove angle is extracted from the laser line image based on the segmentation model with the addition of online hard example mining. The molten pool image information is extracted based on DenseNet and ASPP model. Then, the above information is reconstructed and fused to predict the backside melting width. The Mean Square Error (MSE) of the predicted backside melting width is better than 0.28 mm for complex grooves and is 57% lower than that without adding an angle, which verifies the model's accuracy. The model has a run time of fewer than 0.015 s, which meets the time requirement for online monitoring. Finally, the backside melting width is controlled based on fuzzy proportional-integral-derivative (PID) control. The MSE of the control result does not exceed 0.11 mm.