2.1.

Principle of the Virtual Scale Function

The principle of distance estimation to the object to adaptively change the length of the virtual scale was based on the triangulation method. A schematic of the tip of the endoscope is shown in Fig. 1(a). The endoscope has a window at the tip to emit the red laser beam diagonally so that the position of the laser beam on the object changes according to the distance between the object and the tip of the endoscope. The output power of the red laser was weak and did not have an effect on the mucosa. When the image sensor of the endoscope detects the laser spot, the distance from the tip of the endoscope to the object illuminated by the laser is calculated from the position of the laser spot.

Fig. 1

Schematic of the distance estimation function.

An endoscope prototype with a built-in red laser with a diameter of 12.8 mm was implemented for colonoscopy. The endoscope is part of the ELUXEO system (FUJIFILM Co., Tokyo, Japan) comprising a VP-7000 processor and BL-7000 light source. It supports image-enhanced functions, such as linked color imaging (LCI) or blue-light imaging (BLI), which are effective for the detection and characterization of polyps.^15–19 The image signal from the processor was input to a personal computer EX-1 (FUJIFILM Co., Tokyo, Japan), and dedicated software was installed to estimate the distance in real-time. The output of the image signal was displayed on a monitor with a suitable representation of the virtual scale. The laser beam spot was located at the left end of the virtual scale. The virtual scale contained markings at 5, 10, and 20 mm, as shown in Fig. 4. The length of the virtual scale and the steps were adaptively changed according to the distance to the object, which was calculated from the position of the laser spot. The red laser and the virtual scale can be switched on and off by a button on the handle of the endoscope. This virtual scale function can operate in the conventional white light imaging mode as well as in the LCI mode. When the operator changes to the BLI mode, the virtual scale is automatically switched off.

2.2.

Evaluation of Fundamental Performance

The length of the virtual scale is calculated from the position of the laser spot assuming that the object is on the plane facing the tip of the endoscope. A measurement error will occur according to the distance of the object and the tilt angle along the direction of the virtual scale bar.

To evaluate the fundamental performance, the measurement error of the adaptive virtual scale was evaluated using graph paper with ruled lines (0.5-mm increments) as the measurement object. From the endoscopic image of the graph paper with the virtual scale, the actual length between the position indicated by the left end and the 5, 10, or 20 mm marks of the virtual scale was read off from the graph paper. The range of the measurable actual length in this graph paper was $\le 30\mathrm{mm}$. The measurement error was defined as the difference of the actual length from the corresponding step of the virtual scale. First, the measurement error was read off when the distance to the graph paper, which faced the tip of the endoscope, varied from 4 to 30 mm. The maximum measurement error, mean ± standard deviation of the measurement error, and the standardized measurement accuracy, defined as the absolute value of the measurement error divided by the actual size multiplied by 100%, were within the measurement range. The measurement error was read off when the graph paper was tilted left or right by 30 deg with respect to the vertical centerline of the endoscopic image when the graph paper was placed at distances of 10, 20, or 30 mm. The origin of the tilt angle was defined as the reference point at which the graph paper faces the endoscope. It was defined as positive when the left side of the endoscope image moved to the front and negative when the right side moved to the front.

2.3.

Image-Based Estimation of the Polyp Size

To evaluate the accuracy of this virtual scale function, the method used to estimate polyp sizes by expert endoscopists, and their comparisons to the open biopsy forceps method were demonstrated based on the use of phantom images. Silicone pseudo-polyps were attached to the colon model LM-107 (Koken Co., Ltd., Tokyo, Japan). In total, 33 silicon pseudo-polyps with diameters in the range of 2.5 to 28 mm, 21 hemispherical types, six spherical types, and two pedunculated types were studied. Each size was measured with a caliper (unit of 0.1 mm). Endoscopic images were acquired at randomly set distances in the range of 4 to 30 mm from the polyp to the tip of the endoscope for each polyp. The position of the endoscope against the polyp was also randomly set to acquire the view in which the polyp was visualized anteriorly as well as the scene in which the polyp was visualized on the side to emulate actual clinical practice. Each polyp was imaged with the virtual scale or with the biopsy forceps (Radial Jaw4 Jumbo, Boston Scientific, Marlborough, Massachusetts), which had a maximum width of 9 mm in the open state. When images were acquired with biopsy forceps, the forceps were fully opened and pressed against the polyp. When images were acquired with a virtual scale, the spot of the red laser, that is, the left edge of the virtual scale, was aligned with the left edge of the polyp. In total, 33 patterns of images were prepared for biopsy forceps and virtual scale tests.

The endoscopists estimated the sizes of the polyps from the images. As a preliminary preparation for the test, the evaluators were briefed on the principles, directions, and precautions of the virtual scale function. Subsequently, the test method was explained, and four images, which depicted pseudo-polyps with the sizes of 5, 10, 15, or 20 mm, as measured with the biopsy forceps or the virtual scale method, were presented. These image examples were not used in the subsequent tests. The polyp size estimation test was conducted by displaying the polyp images on the endoscopic monitor without any information regarding the actual polyp size. In total, 33 images with the biopsy forceps method were presented first followed by another set of 33 images with the virtual scale method. Within each image set, the order of images was random. Evaluators estimated the polyp size from a displayed image with reference to the biopsy forceps or virtual scale and then filled the answer sheet.

For the analysis of the measurement accuracy, the measurement error (defined as the difference of the actual and estimated sizes), and standardized measurement accuracy (defined as the absolute value of measurement error divided by the actual size multiplied by 100%) were compared between the forceps and virtual scale sets. Correct estimation is defined as when the absolute value of the measurement error $<1\mathrm{mm}$. Based on this, the numbers and rates of correct estimation, overestimation, and underestimation were analyzed. Continuous data are presented as the mean ± standard deviation. Categorical data are presented as numbers or rates. The difference between biopsy forceps and the virtual scale in categorical data was analyzed using the Fischer’s exact test. The differences in continuous data acquired from the same polyp were analyzed using a paired t- or Wilcoxon signed-rank test. Statistical significance was set at $P<0.05$. Statistical analysis was performed with R (version 3.6.1, R Foundation for Statistical Computing, Vienna, Austria).

3.1.

Demonstration of the Real-Time Adaptive Virtual Scale

The virtual scale behavior, which changes the size on the monitor in real-time according to the distance to the object, was demonstrated (Fig. 2) using a chart sheet and pseudo-polyps. When the operator pushed the button on the handle of the endoscope, the red laser was on and the virtual scale was indicated as a sky-blue bar, which appeared on the monitor. The real-time behavior confirmed that the virtual scale moved and changed the length along with the red laser spot according to the distance to the object. The measurable distance range was from 4 to 30 mm. When the image display range was $<20\mathrm{mm}$, the total length of the scale was displayed up to 10 mm, and when the image display range was $<10\mathrm{mm}$, the total length of the scale was up to 5 mm. When the laser spot was not detected or the distance was outside the effective range, an error occurred, and the virtual scale turned yellow.

Fig. 2

Demonstration of the virtual scale function (Video 1, MPEG, 12 MB [URL: https://doi.org/10.1117/1.JBO.26.9.096002.1]).

In clinical practice, when an endoscopist points the tip of the endoscope in front of the object and arranges the spot of the laser beam at the left end of the object, the endoscopist can compare the size of the object with the virtual scale overlaid on the object without any other additional device.

3.2.

Evaluation of Fundamental Performance

Measurement errors estimated (for 5, 10, and 20 mm steps of the virtual scale) when the distance to the graph paper faced the tip of the endoscope varied from 4 to 30 mm, as shown in Fig. 3. The maximum measurement errors (and the standardized measurement accuracies) were 0.5 mm (9.1%), 0.5 mm (4.8%), and 0.7 mm (3.4%), and the mean ± standard deviations were $-0.2\pm 0.2\mathrm{mm}$ ($4\pm 2.6\%$), $-0.3\pm 0.1\mathrm{mm}$ ($2.8\pm 1.4\%$), and $-0.3\pm 0.3\mathrm{mm}$ ($1.7\pm 1.3\%$), respectively.

Fig. 3

Measurement errors obtained by reading the lengths from the graph paper (at 5, 10, and 20 mm steps of the virtual scale) from the endoscope image when the distance to the graph paper, which faced the tip of the endoscope, varied from 4 mm to 30 mm.

The measurement errors when the graph paper was placed at the distances of 10, 20, or 30 mm and was tilted left or right by 30 deg with respect to the vertical centerline of the endoscopic image, are shown in Fig. 4. As the tilt angle and the scale increased, the error also increased. In particular, when the tilt angle was positive, that is, the left side moved to the front, a large error occurred and resulted in an underestimation. The observation distance also affected the measurement error when the graph paper was tilted. By increasing the distance to the object according to the scale, the tolerance for the tilt angle was improved. For example, the measurement errors (scale at 10 mm steps) when the tilt angle was $-20/+20\mathrm{deg}$ at the distances of 10, 20, and 30 mm were 1.4 mm (16.3%)/$-3.8\mathrm{mm}$ (27.5%), $0.3\mathrm{mm}(3.1\%)/-1.8\mathrm{mm}$ (15.3%), and $-0.1\mathrm{mm}$ (1%)/ $-1.8\mathrm{mm}$ (15.3%), respectively. The endoscopic images of these conditions are shown in Fig. 5.

Fig. 4

Measurement errors obtained by reading the lengths of the polyps from the graph paper (step intervals of 5, 10, and 20 mm of the virtual scale on the endoscope images) in instances at which the graph paper was tilted left or right by 30 deg. The distances Z to the graph paper were (a) 10, (b) 20, and (c) 30 mm.

Fig. 5

Endoscopic images of the graph paper (virtual scale). The distances and tilt angles of the graph paper were (a) 10 mm, $-20\mathrm{deg}$, (b) 10 mm, $+20\mathrm{deg}$, (c) 20 mm, $-20\mathrm{deg}$, (d) 20 mm, $+20\mathrm{deg}$, (e) 30 mm, $+20\mathrm{deg}$, and (f) 30 mm, $-20\mathrm{deg}$.

3.3.

Image Evaluation of Polyp Size Estimation by Endoscopists

Three endoscopists estimated the polyp size for 33 pseudo-polyps images with the use of forceps and 33 images with the use of a virtual scale (Fig. 6). All evaluators were expert physicians who were certified as specialized endoscopists by the Japan Gastroenterological Endoscopy Society. In total, 99 estimated results for each method were analyzed.

Fig. 6

Image samples for the polyp size estimation test. (a) Polyp with a size of 10 mm with the use of forceps, and (b) polyp with a size of 15 mm estimated with the use of the virtual scale.

The number of images (rates) that were correctly estimated, overestimated, and underestimated using forceps and using the virtual scale were 39 (39.4%) and 60 (60.8%), 18 (18.2%) and 6 (6.1%), and 42 (42.4%) and 33 (33.3%), respectively. The two groups were statistically independent by the Fischer’s exact test ($P=0.0028$). These data are shown in Table 1.

Table 1

Association of image numbers (rates) of correct, over-, and under estimations from 33 images with biopsy forceps and 33 images with virtual scale by three expert endoscopists (P=0.0028, Fischer’s exact test).

The standardized measurement accuracy data are shown in Fig. 7. These obtained values are $11.9\%\pm 9.4$ (biopsy forceps case) and $5.3\%\pm 5.5$ (virtual scale case). The results of the virtual scale showed a smaller difference from the actual size and data distribution compared with the biopsy forceps.

Fig. 7

Standardized measurement accuracy from the polyp size estimation outcomes with the biopsy forceps or with virtual scale obtained from three expert endoscopists.

The standardized measurement accuracy with the virtual scale was significantly higher than that with biopsy forceps according to the Wilcoxon signed-rank test ($P<0.001$). For the analysis, the Wilcoxon signed-rank test was used because the difference data between biopsy forceps and virtual scale were not normally distributed.

Standardized measurement accuracy from the polyp size estimation outcomes with the biopsy forceps or with virtual scale obtained from three expert endoscopists.