The structural and morphological characterization of deep-buried rock mass is of great significance to the construction of deep underground engineering, and the digital panoramic camera technology is an important means to obtain the structural morphology of rock mass in the borehole quickly and effectively. Faced with the highlighted problems of the borehole camera system and its analysis software in the complex environment, a method of fast mosaic and fusion of circular image from the borehole video is proposed. In this method, the borehole video is transformed into a number of ordered narrowband images, and the image features are detected and matched, so as to realize the rapid mosaic and fusion of borehole panoramic images. The results show that this method can quickly complete the continuous mosaic and fusion of narrowband images without the help of the compass and depth encoder, and the obtained image resolution is higher than before, and also the actual working time can be half with the process of automation and intelligent. This method can quickly and effectively form a high-precision panoramic image without deviation based on the inherent characteristics of the borehole video image. It can reduce the operational burden of researchers and improve the work efficiency, which promotes the development of borehole camera technology and provides a more convenient and effective technical means for deep rock mass engineering investigation.
Introduction
It is necessary to know the structural characteristics and stability information of deep-buried rock mass in geotechnical engineering, hydropower engineering, petroleum exploration, and geological disaster prevention engineering. These structures (Xu et al. 2017), such as joint, fault, soft rock, and fracture zone, are important parts for rock mass and can be obtained by a digital panoramic borehole camera system with high-precision panoramic video images, that is, borehole camera technology (Bae et al. 2011). The obtained panoramic video images have accurately recorded the characteristics of the rock mass structures and their morphology in the borehole (Wang et al. 2016). In engineering practice, how to quickly and effectively obtain the panoramic image of the real rock mass structure inside the borehole and how to accurately present the in situ information of rock mass structure and morphology have important practical engineering significance for engineering geological exploration and engineering construction. They are also very important for deep underground engineering construction (Schepers et al. 2001; Alameda-Hernández et al. 2019).
At present, the analysis and processing of borehole camera videos are basically in the traditional sense of the scan line stacking or multiple scan line stitching method (Kanaori 1983; Wang et al. 2017). They completely rely on or indirectly reference the azimuth angle of the compass (or an electronic compass) and the depth information of the encoder to continuously generate a single or multiple scan lines of each frame of the video image (Han et al. 2020), and then combine the generated scan lines of each frame image in pile to form the corresponding borehole images (Mebrahtu et al. 2020), or go on simple and orderly matching for the multiple scan lines or the ribbon image of rectangular windows in order to better accumulate to the generation of panoramic images for the borehole (Deng et al. 2019). The main problems of this approach after the consideration of all complex field environments of offshore drilling, deep-buried tunnels, and coal drilling (Strømsvik and Gammelsæter 2020) are (1) the information carried by the traditional single or multiple scan lines is limited, and it is difficult to ensure that the camera probe in the hole is carried out at a very slow and uniform speed during the field investigation, which leads to a phenomenon that the recorded video images are very fast or very slow at some times, accompanied by irregular rotation and oscillation back and forth. Therefore, the scan line method is unable to record more scan lines when the casting probe is too fast and needs to delete some overlapped parts when the casting probe is too slow. As a result, the forming borehole image may be severely deformed and needs subsequent depth correction, so the image quality is very poor. (2) After scanning lines are stacked into the image, the quality of the borehole image fully depends on the accuracy of the data of the compass (or electronic compass) and the encoder. However, both the compass and the encoder have a certain amount of physical inertia and are inevitably prone to accidental errors in practice. In particular, the compass and the encoder themselves cannot self-modify and find the correct position and direction when the camera probe rotates, stretches, and dips in the hole due to human factors driven by the cable. All of these will further result in a poor quality of the scan line method while forming the image. (3) Another, the existing implementation software of the scan line method cannot achieve the adaptive and intelligent processing of borehole image at present. Subsequent data corrections such as the depth correction and the azimuth correction rely heavily on manual processing and practical experience (Xie et al. 2019). For the data processing of the video images from multiple holes or ultra-deep holes, it needs several days or even several weeks to carry out special data processing, which consumes much time and energy. Therefore, it is urgent to carry out intelligent and automatic analysis and processing of images for borehole camera videos.
Anzeige
In view of the above problems, a new method of fast mosaic and fusion for the circular image obtained from borehole panoramic video is proposed in this paper. Its aim is to solve the intelligent matching of borehole video images and the automatic mosaic of image data, to solve the problem that completely depends on the compass or electronic compass, and to realize the intelligent mosaic and fusion of camera video images and the rapid real-time formation of borehole panoramic image, which improve the work efficiency and reduce the operational burden for front-line scientific research workers. This method will provide a great convenience for the actual drilling exploration process and following borehole video image processing, which can improve the timeliness and high efficiency of borehole camera exploration in the deep rock mass.
Borehole camera technology in the deep rock mass
Digital panoramic borehole camera system
The digital panoramic drilling camera system is a new set of advanced and intelligent exploration equipment (Li et al. 2013). It integrates electronic technology, video technology, digital technology, and computer technology to record the hole wall in situ without disturbance from a panoramic perspective (Wang et al. 2018). By directly imaging the hole wall, the disturbance effect of drilling coring is avoided, and the structural morphology of the hole wall can be accurately explored, and the rock strata inside the rock mass can be reflected in detail (Norbert and Katarzyna 2014). The system can simultaneously observe the hole-wall condition of 360°, with the capability of real-time monitoring (Zou et al. 2018), and it can display, analyze, and save the whole borehole data on-site (Guo et al. 2020).
The system is mainly composed of a panoramic camera probe, a control box, a depth encoder, and a cable, as shown in Fig. 1. The panoramic camera probe is the key equipment of the system, which contains a truncated cone mirror and a miniature CCD camera to obtain hole-wall panoramic images, a light source to provide imaging detection lighting, and a magnetic compass for image azimuth determination (Wang et al. 2018). The whole probe adopts high-pressure sealing technology, which can be detected in deep water. The depth encoder is one of the positioning devices of the system, which consists of a measuring wheel, a photoelectric angle encoder, a depth signal acquisition board, and an interface board. The software part of the system mainly includes data acquisition software and data analysis software. The system software can digitize the recorded drill hole video, analyze the panoramic drill hole image, identify and interpret the structural surface, store and maintain the drill hole image data, and also provide a reliable drill hole image and rock mass structure data for practical engineering.
Fig. 1
The system and its panoramic camera probe
×
Borehole camera video feature
The digital panoramic borehole camera system can obtain the real image of the rock mass structure in real time (Assous et al. 2014). It is an important research topic about how the obtained panoramic borehole image can reflect the rock mass structure more quickly and effectively. Therefore, we directly read the original video image in the camera captured by the panoramic camera probe and quickly convert the video image into a high-quality panoramic image, which are the key to the formation of the panoramic borehole image.
Anzeige
Panoramic borehole image is an indirect reflection of the rock mass structure inside the hole. It is the imaging result of the rock mass structure reflected on the cone mirror after being illuminated. The rock mass structure of the borehole wall determines the composition of the color depth of the borehole image (Saricam and Ozturk 2018). Due to the differences in reflectivity of cracks, fractures, and holes, the borehole images show bright and dim colors, as shown in Fig. 2, which is a camera image obtained from a borehole camera video at a depth of about 152 m at the Dadu River Bridge of the Sichuan–Tibetan Railway.
Fig. 2
An image from a borehole camera video obtained by the digital borehole panoramic camera system
×
In the process of converting the video image obtained by the digital borehole panoramic camera system into the real and usable borehole panoramic image, the video image mainly presents the following characteristics and technical problems: (1) the imaging characteristics of hole-wall rock mass structure are from near to far annular shape, as shown in Fig. 2; (2) the hole-wall structure is continuously imaged from near to far in the process of down-casting of the probe, and the images in the video have annular deformation and staggered overlaps; (3) due to the inclination of the borehole, the cable circumvention, or the influence of the probe’s self-weight, the panoramic camera probe will inevitably rotate and shake, and experience other accidental events, resulting in the rotation and deformation of the borehole panoramic image data (Feng et al. 2013) (Młynarczuk et al. 2016). The original camera video recorded by the probe has itself absolute continuity and inherent image characteristics (Zou et al. 2020). Therefore, how to form a borehole panoramic image more accurately through the continuity and the inherent characteristics or revise orientation and depth information is necessary. We use the image mosaic and data fusion of panoramic images to improve the image quality from borehole camera video and the image analysis process.
The fast-forming image method for borehole camera video
To solve the existing problems of borehole camera equipment, a fast-forming image method is provided by using image mosaic and data fusion techniques according to the wall imaging features of drilling holes. The specific steps for an obtained borehole camera video are as follows. Some related data of the drill hole are also taken as an example to illustrate.
Analysis of sampling length and sampling resolution
The digital panoramic borehole camera system uses a truncated cone mirror to reflect the 360° hole-wall image into a plane image by the specific optical transformation. The circular image with a panoramic view in the video can be formed through this optical transformation. The original video obtained by using the system is not convenient for direct observation and lacks practical significance, so it needs further transformation and correction.
The theory and technology of the method used are about related imaging distortion correction, image mosaic, and data fusion techniques (Weixing et al. 2016), such as checkerboard calibration, non-uniform interpolation, SIFT (scale invariant feature transform) feature detection, BBF (best bin first) algorithm based on a k-d tree, RANSAC (random sample consensus) algorithm, and weighted smoothing (Price et al. 2006). These can be used to overcome problems such as the low quality of formed scan line image, the slow speed of forming an image, and the long and complicated process of data analysis (Shum and Szeliski 2002).
The main steps of this method are described in Fig. 3, namely, firstly, the digital panoramic borehole camera system is used to obtain the original borehole video image, analyze the video, and obtain the narrowband image of each frame in the video; then, go on image feature matching and feature detection for the narrowband image; and the actual width offset and height offset of each image can be further determined or revised according to the actual azimuth data and depth data; lastly, the pixel offset of the narrowband image of each frame in the video is determined, and then the narrowband images and related pixel offsets are used to go on image mosaic and data fusion for forming the expanded panoramic image. Therefore, the actual borehole image can be obtained after information labeling.
Fig. 3
The flow chart of the fast-forming image method for borehole camera video
×
The main steps of the method
Narrowband image acquisition
For the obtained borehole camera video, firstly, using a Hough circle detection method, such as the HoughCircles() function of OpenCV, to automatically identify the center of the camera image in the video and the maximum and minimum of imaging radius in the camera image, and to automatically identify the data of compass (or electronic compass) and depth encoder in the image in real time; then, the center O(x, y) of the camera image, the inner radius Rmax, and the outer Rmin in the imaging annular of borehole wall are adjusted automatically according to the actual situation, as shown in Fig. 4. The borehole camera video in Fig. 4 is recorded by using the 2 million high-definition digital panoramic borehole camera system, which is the latest product developed by the Institute of Rock and Soil Mechanics, Chinese Academy of Sciences (Wang et al. 2016).
Fig. 4
The process of the narrowband image acquisition
×
Faced with the annular imaging area between the inner radius Rmax and the outer Rmin in the camera image shown in Fig. 4, set the valid range of the annular image as H, that is, the height of the collected narrowband image, and set the middle pixel number of imaging annular radius as W, that is, the width of the collected narrowband image, in order to ensure that the annular image includes more information. The computational formulas of the height H and width W for the forming narrowband image are shown in Eq. (1):
Accordingly, for the annular image in the camera image from the obtained camera video, taking the center O(x, y) of the borehole images as the center of the circle, we start with radius Rmin and the azimuth value of the electronic compass and collect W pixels as a line of the narrowband image, and then add the radius value until the radius is Rmax. Lastly, a narrowband image with the width W and the height H is formed, and the corresponding values of the narrowband image, such as the azimuth and the depth values, are saved at the same time. Therefore, starting from the first frame (camera image) of the borehole camera video, each frame of the video forms a narrowband image with width W and height H until the end frame of the video, as shown in Fig. 5.
Fig. 5
One of the obtained narrowband images. a The difference images of two adjacent narrowband images and their detected feature points. b The corresponding matching relationship with the next narrowband image of a. c The last matching result between two adjacent images
×
Feature detection and matching data generation
The borehole camera image itself has the following characteristics: the narrow strip image formed by each frame of the video image is relatively narrow; most areas of hole-wall rock are basically the same; most of the regions have no special corner inflection point, and the image noise is very serious; the interference signals caused by water flow, sand, mud, and other debris in the image are relatively strong, some time may be serious. Therefore, the feature detection and image matching of the narrowband images have great interference and general reliability. We take the difference image of two adjacent narrowband images as the input image of feature detection in order to well eliminate the inherent interference of the image itself. Another, in consideration of the accuracy and timeliness of image feature detection and feature matching, we take the SURF (speeded up robust feature) algorithm to get the feature points for the obtained narrowband image. The alternative algorithms are the SIFT algorithm, the ORB algorithm, the FAST algorithm, and the Harris corner algorithm for the camera image under different complex applications. Therefore, the difference images of two adjacent narrowband images are successively used to detect and match image features and form the original feature points. The results of feature detection are as shown in Fig. 6(a).
Fig. 6
The feature detection and matching of two adjacent narrowband images in the borehole camera video. a The original mosaic image. b The image after data fusion
×
The basic principle of image feature detection is based on the invariable feature of the visual image feature scale. The idea of scale-invariant features is that not only can objects photographed at any scale detect consistent key points but also each detected feature point corresponds to a scale factor. Ideally, for the same object point at different scales in two images, the ratio between the calculated scale factors should be equal to the ratio of image scales. Therefore, an accelerated, robust SURF detection method is adopted. The SURF features not only are scale invariant but also have high computing efficiency. Therefore, the SURF features are used to detect feature points on narrowband images. Multiple feature points can be quickly detected for each narrowband image, and the corresponding matching relationship with the next narrowband image can be established, as shown in Fig. 6(b).
According to the matching effect of feature points in Fig. 6(b), there is a deviation in the effect of partial matching, so it is necessary to further filter the matching points to obtain the excellent matching points. Therefore, Lowe’s algorithm is used in this paper to obtain excellent matching points further. The empirical value of the ratio threshold of Lowe’s algorithm is generally between 0.3 and 0.9, and we use ratio = 0.5 in this paper. In order to ensure the uniqueness of correspondence, we select the best matching points as the last result, which is shown in Fig. 6(c).
According to the above image feature detection results and the selecting principle of Lowe’s algorithm, the matching point with the best ratio value is used as the only optimal matching point for the two adjacent frames of narrowband images. The optimal matching point between the ith and the (i + 1)th narrowband images can be set as Mi(xi, yi), which is in the ith narrowband images. The next obtained optimal matching point between the (i + 1)th and the (i + 2)th narrowband images can also be set as Mi + 1(xi + 1, yi + 1), which is in the (i + 1)th narrowband images. Therefore, the offset between the ith and the (i + 1)th camera images can be obtained, that is, the offset ΔOffi(Δxi, Δyi) of the next frame relative to the last frame can be obtained by using this process. The values of the offsets between frames can be obtained by following Eq. (2):
where Δxi means that the next frame has move Δxi pixels in the horizontal direction, and Δyi means that the next frame has move Δyi pixels in the vertical direction relative to the last frame. The frame number i, the offset values Δxi and Δyi, the original azimuth value of the electronic compass, and the depth data of the encoder are important original data and must be saved as a document for future further data revision and image improvement.
Quick mosaic and fusion of narrowband images
The principle of quick image mosaic for the narrowband images is as follows. Firstly, take the first narrowband image as a new panoramic image I1, and then add the next offset image to the panoramic image I1 and form I2. The next offset image is formed as follows. According to the obtained offset values Δxi and Δyi, the ith narrowband image moves Δxi pixels in circles, and then take the up and down (Δyi/2) lines in the middle line of the ith narrowband image to form a new offset image ΔIi, that is, the lines from (y/2 − Δyi/2)th to (y/2 + Δyi/2)th are used to form the offset image ΔIi after each line moves in circle Δxi pixels in the horizontal direction. By that way, a number of panoramic images I1, I2, I3, …, Ii, … can be formed. They are added and lastly formed an original panoramic image of the borehole camera video, as shown in Fig. 7(a).
Fig. 7
The original mosaic image (a) and the image after data fusion (b)
×
From Fig. 7(a), the junction between two narrowband images in the mosaic image is not natural. This is due to the difference of light color and rock, which makes the transition between two images occasionally deviated, so the data fusion method is needed to solve this unnatural junction. A weighted fusion method is used to slowly transition from the previous image to the second image in the overlapping part of the panoramic image Ii. The pixel values of the overlapped part in the offset image ΔIi are added to a certain weight to make the panoramic image ΔIi-1.
The steps are as follows: firstly, the (i + 1)th offset image ΔIi + 1 moves Δxi + 1 pixels in circles while the offset image ΔIi + 1 will have an overlapped part with Δyi + 1 lines relative to the offset image ΔIi; then, add a weight value w to each line in the overlapped part and form the new panoramic image Ji + 1. The weight value w while processing the overlapped part can be obtained in Eq. (3):
$$ w=\frac{H-n}{{\Delta y}_{i+1}} $$
(3)
where n is the processing nth line in the offset image ΔIi. Finally, the panoramic image J with the better effect is formed, as shown in Fig. 7(b). In addition, the partial enlargement image is shown in Fig. 8, which is the comparison between the original mosaic image on the left (Fig. 7(a)) and the image after image fusion on the right (Fig. 7(b)). The data fusion technique creates a smooth transfer from the last narrowband image to the next narrowband image.
Fig. 8
The partial enlargement image after image fusion, the left is the enlarged image of Fig. 7a and the right is the enlarged image at the same area of Fig. 7b
×
In this process, due to that the drilling borehole is very deep, the finally formed panoramic image is very long, and related data are very large. This method saves each formed borehole panoramic image with a fixed length L, which general L can set as 2 m, in order to prevent the image from being too long to save the image for a better reviewing.
Mosaic image optimization and annotation
For the lastly formed panoramic image, the gray stretch and detail enhancement of each formed image are carried out by using an image enhancement method, such as image histogram equalization. The purpose of image gray stretching is to prevent the image from being too dark or too bright and perform a uniform gray equalization process on the entire image. Then, the entire image can be smooth. Then, the image details are enhanced by using the image enhancement method to highlight the information of rock mass structures in the image.
To prevent the image from being too large or too long, a certain aspect ratio is adopted to convert the formed panoramic image pixels into actual scale meter, to make the image more convenient for viewing and understanding and conform to the esthetic view. For example, the aspect ratio of 0.618, which is close to the golden scale, can be used to scale the horizontal and vertical pixels for the borehole panoramic image.
Another, the final borehole image must be marked with depth and orientation according to the azimuth and depth data obtained in the camera video, which can be called as the borehole panoramic image. At the same time, in order to facilitate the actual engineering application, the 3D histogram image can be generated according to the borehole panoramic image in four directions, namely, set the perspective as the east, the south, the west, and the north, respectively. Finally, a borehole panoramic image and four 3D histogram images can be obtained, respectively, for a certain section of the camera video. Taking the formed image in Fig. 7 as an example, the last forming borehole images after information annotation and histogram generation are shown in Fig. 9.
Fig. 9
The last forming borehole images with a borehole panoramic image and four 3D histogram images for each group image
×
Results and discussions
Taking an actual high-definition borehole camera video as an example, we use the above method to process this video and discuss some questions during actual engineering applications. This video has 44,980 frames with a duration of 30 min. It comes from an actual geological engineering survey at the Dadu River Bridge of the Sichuan–Tibetan Railway. The method described in this paper is used to realize the fast matching and fusion of panoramic images from the borehole camera video, and some results of the obtained borehole images are shown in Fig. 10.
Fig. 10
The obtained borehole images from a geological engineering survey in the Sichuan–Tibetan Railway. a An image after using the revised feature matching method. b An image without better selection of feature points
×
Results show that this method can realize the image mosaic and fusion of a large number of the high-definition video image with fast speed. It can automatically form a series of high-quality borehole panoramic images without manual operation. The method described in this paper well simplifies the borehole camera video processing and provides great convenience for the actual drilling investigation process, which greatly improves the real time and efficiency of the borehole investigation in the deep rock mass engineering.
The selection of feature matching points
In order to further exclude the key points of the non-matching relationship due to image occlusion and background chaos, a matching method comparing the nearest neighbor distance to the next nearest neighbor distance is proposed. We take a key point in one image and find the first two key points that are closest to the Euclidean distance in the other image. In these two keys, if the ratio of the nearest distance divided by the next closest distance is less than a certain threshold T, then we accept this pair of matching points. For false matching, due to the high dimension of feature space, a similar distance may have a lot of other false matching, so its ratio value is relatively high. Obviously, if the ratio threshold T is reduced, the number of matching points will be reduced, but more stable. Therefore, a large number of matching results for two adjacent narrowband images show that there are few pairs of matching points when the ratio is less than 0.5, and there are many pairs of wrong matching points when the ratio is greater than 0.7. So, the value principle of ratio in this paper is as follows: ratio = 0.5 for the matching of high-definition camera video with high-accuracy borehole panoramic image, ratio = 0.7 for the matching of standard definition camera video, ratio = 0.6 for general applications by default.
However, there are more than one pair of matching points for a good image or zero or wrong pair of matching points for a bad image. Zero or wrong pair of matching points may result in the wrong image or interrupt the continuity of the running process of image mosaic for a whole camera video, which must add other interventions. More than one pair of matching points will make the selecting process of key points be more complex and produce extra time, which can be improved. Therefore, it is necessary to consider the uniqueness of pairs of key points in the obtained feature points during the image mosaic process in any complicated situation. The uniqueness of pairs of key points in any narrowband image can ensure the timeless fast-forming borehole panoramic image.
Another, the narrowband images of the borehole wall have their particularity (each frame is narrow, the vertical matching scale range is small, the horizontal matching range is large, the local range has strong continuity, and the image features are seriously homogenized) and their specificity (only for the narrowband image of the borehole wall obtained in step (1)). Due to imaging large noise interference of the borehole wall, the rock mass structures are almost similar in a certain range and exist a large number of rotation and brightness changes in different scales.
Therefore, we select one pair of key points for two adjacent narrowband images, and each image corresponds to one key point of the pair. The one pair of key points is the most matching key points after using Hamming distance to select the unique pair of key points for two adjacent narrowband images. Then, the selected unique pair of key points is proof-read after comparing the obtained data by using a compass (or electronic compass) and depth encoder. Some obtained borehole panoramic images after using the selected feature matching method are shown in Fig. 11.
Fig. 11
A comparison with images before and after using the revised feature matching method. a The image after cumulative error correction. b The original scan line image
×
The elimination of accumulated errors
In the process of feature detection and matching of narrowband images, it is difficult to ensure that the selected unique pair of key points each time is accurate and unbiased. Although each time the error is very small (within one pixel), when tens of thousands of images are matched continuously, the accumulated error will be very large. Therefore, it is necessary to eliminate accumulation. Every matching key point may have a certain local small error. However, the overall trend of the accumulated local error is related to the rotation trend of the panoramic camera probe (Ozkaya and Mattner 2003). Therefore, this paper first uses a Gaussian hyperbolic filtering method to perform the hyperbolic Gaussian filtering of the obtained matching data in step (2), which can eliminate local random small errors; then accumulate the offset for each matching data in turn, and perform the same offset on the next narrowband image before performing feature detection and matching; and finally, statistically analyze the azimuth data in the image and the changing trend of the matched offset so as to compare and correct each other. Within a certain range, the reliability of the offset data obtained by image feature detection and matching is better than the azimuth value obtained by the compass or electronic compass, so as for the obtained depth.
Therefore, for the correction elimination adopted in this paper, when the difference between each offset is less than a quarter of the image width, that is, pixels W/4, the current offset value is used; otherwise, the azimuth value obtained by the compass or electronic compass is used, and the mutual correction is restarted based on the median value between the azimuth value and the offset value. The comparison result between the mosaic image after the elimination of accumulated errors is shown in Fig. 12(a) and compared with the original scan line image Fig. 12(b). Figure 12 is from the same certain area of the actual drilling image.
Fig. 12
The comparison results of the image after cumulative error correction and the original scan line image
×
The accuracy analysis of formed borehole image
In order to further illustrate the advantages of the method described in this paper, the traditional scan line method and the image mosaic and data fusion method described in this paper are used for the comparison and analysis of image accuracy. Take the corresponding original video in Fig. 12 as an example, and get Fig. 12 after processing with the traditional scan line method and image mosaic and fusion method, respectively. Take the same position of Fig. 12 and enlarge it 10 times to get Fig. 13, and observe carefully, as shown in Fig. 13.
Fig. 13
Comparison of the resolution of the mosaic image and the scan line image after partial enlargement
×
The compared results are shown in Fig. 13, which are the comparison diagrams of the enlarged images of the same caving stone, the same hole, and the same rock crack from Fig. 12. Figure 12 is a comparison of the original image of a certain area in the same borehole image formed by using our method and the original scan line method, respectively. The images are a certain caving stone, a certain hole, and a certain rock crack enlarged in the borehole image. The left figure is the forming effect of our method, and the right figure is the form effect of the original scan line method. It can be seen that the quality of the borehole image obtained by using our method is much higher than that of the original scan line method in clarity and accuracy after viewing and comparing at the same position and with the same magnification.
The lateral accuracy of the standard definition image obtained by using the digital panoramic camera system is 1024 pixels, and the design accuracy of the longitudinal depth direction is 1 mm. However, due to the complexity of the field measurement and the instability of the cable’s moving speed, the longitudinal accuracy of the standard image generated by using the original system cannot reach 1 mm. The actual lateral and longitudinal resolution are closely related to the actual measured aperture size, the data of encoder depth correction, and compass or electronic compass azimuth correction. However, because the mosaic image does not completely rely on the compass (or electronic compass) and encoder, it can even completely throw away the depth and azimuth correction data to automatically complete the continuous mosaic and fusion of panoramic image in the hole. By relying on the physical features of the hole-wall image to achieve continuous mosaic and fusion of the panoramic image, it effectively avoids the interference of many human factors and ensures the reliability and effectiveness of the physical image itself.
Because the original video image is obtained by using the camera, the accuracy of the hole-wall image is the accuracy of the camera; the accuracy of the mosaic image obtained in this paper can be the same in theory, which is from the cutting transformation and mosaic fusion of the borehole camera video. Therefore, their image resolutions should be the same or can be close to the resolution and accuracy of the camera if the method is better enough, that is, the accuracy of the mosaic image obtained in this paper should also be close to the accuracy of the camera, which is far greater than the 1-mm accuracy. In fact, in order to force the lateral accuracy to be consistent with the longitudinal offset of the mosaic data in the same actual size, we have to expand the lateral pixels of the mosaic image to more than 5000 pixels. This further shows that the actual lateral and longitudinal resolutions of the mosaic fusion image can be improved by an order of magnitude or at least 0.2 mm, as shown in Fig. 13 and Table 1.
Table 1
The resolution analysis between the mosaic image and the original scan line image
Resolution type
Design accuracy
Actual accuracy
Magnification 10 times
The original scan line image
Lateral
1024PX
1024PX
Unclear
Longitudinal
1 mm
> 1 mm
The mosaic image
Lateral
1024PX
> 5000PX
Clear
Longitudinal
1 mm
0.1 mm
According to Fig. 13 and Table 1, the resolution of the mosaic image obtained for the same video is much higher than that of the traditional scan line image. After the same position is magnified 10 times, the texture of the rock mass structure in the mosaic image is still clear while the scan line image becomes blurred. Therefore, the resolution and accuracy of the mosaic image are much higher than those of the traditional scan line method, which further promotes the upgrading of the borehole camera system and its software analysis method.
The analysis of actual work efficiency
In order to illustrate the actual work efficiency of the method described in this paper in practical engineering application, the traditional scan line method and the mosaic method are used to analyze the same 30-min borehole video and to form borehole panoramic images. The traditional scan line method is used to complete the panoramic image analysis of the whole borehole, and the final total time is about 87 min, of which the program running time is about 38 min. In the process of manual depth correction of encoder and azimuth correction of the electronic compass, the operation is complex and tedious of than the traditional scan line method. However, it takes only about 16 min without manual operation if using the mosaic method described in this paper. For our method, almost no human intervention is needed to generate high-quality borehole panoramic images quickly. This process needs about 14 min for the running of software. The specific data of actual work efficiency is shown in Table 2.
Table 2
Comparative results of actual work efficiency
Object
Method
Total time consumed
Complexity
Program time consumed
30-min video
Scan line method
87 min
Complicated
38 min
Mosaic method
16 min
Automation
14 min
According to Table 2, the actual operation steps of the mosaic method have been able to realize automatic processing, and the running time has been greatly shortened with half of the time. The mosaic method does not completely rely on the auxiliary physical compass or electronic compass and the depth encoder, and the software operation is simple and easy. It can realize the intelligent and automatic analysis and processing of the image mosaic and data fusion process, which greatly improves work efficiency and reduces the burden of the researchers.
Conclusions
In view of the problems of the existing digital panoramic camera system and its analysis software in the complex field applications, this paper proposes a method of image mosaic and data fusion for the fast-forming panoramic image of borehole camera video. In this method, the original camera video is transformed into many continuous narrowband images, and the image features are detected and matched continuously, so as to form the borehole panoramic image faster and better. Therefore, some conclusions can be obtained in this paper:
(1)
The mosaic method does not rely entirely on the data of the compass or electronic compass and depth encoder to perform the continuous mosaic of panoramic video images. In order to further ensure the reliability of the image mosaic, the use of the present compass and encoder can be a choice. If the method can be further improved in the future, the compass and encoder can be absolutely abandoned after calibrating the start and stop positions or several specific points.
(2)
By using the fixed features of hole-wall image and imaging information of the borehole wall to realize the continuous mosaic and data fusion of borehole camera video, the wrong or distorted image can be eliminated or improved. This method can automatically obtain the matched data, that is, the azimuth offset and depth offset according to the fixed features of each frame in the camera video, and finally quickly form the high-quality panoramic image without bias.
(3)
The resolution and sharpness of the mosaic and fusion image finally formed by using this method are much higher than those of the traditional scan line method. The lateral and longitudinal resolutions and the quality of the formed image are improved by an order of magnitude or at least 0.2 mm.
(4)
This method is simple and easy to operate and can realize the intelligent and automatic analysis of the image mosaic and data fusion process with half of the time, which greatly reduce the burden of the researchers and improve their work efficiency.
Acknowledgments
All of the images and data are from our actual drilling engineering and permitted by the owners.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
Sales Excellence widmet sich lösungs- und anwendungsorientierten Konzepten, beleuchtet Entwicklungen und bietet Informationen zu Chancen und Herausforderungen im Vertrieb der Gegenwart und Zukunft. Jetzt kostenlos testen!
