BubGAN: Bubble generative adversarial networks for synthesizing realistic bubbly flow images
Introduction
Dense object detecting and counting is a common but time-consuming and challenging task. The applications can be found in the field of pedestrian surveillance (Marsden et al., 2016), vehicle detection in aerial images (Moranduzzo and Melgani, 2014), cell or bacterial colony counting in medical images (Xie et al., 2016, Ferrari et al., 2017, Lu et al., 2017), oil droplet characterization in petroleum engineering (Paolinelli et al., 2018) and bubble counting in bubble columns or nuclear reactors. In bubble column or nuclear reactor applications, accurate separation and reconstruction of bubble shape have equal importance to the number counting, since bubble shape contains important geometrical information for the study of mass, momentum and energy transport in these systems (Paolinelli et al., 2018, Chinak et al., 2018). The demand for bubble shape acquisition introduces additional challenges for algorithm development and benchmarking in this field.
High speed imaging is a powerful technique which can be used to record bubbly flow images at high spatial and temporal resolutions. The image processing technique can be used to extract bubble parameters from these images. Recently, the reported image processing algorithms (Honkanen et al., 2005, Lau et al., 2013, Yu et al., 2009, Lelouvetel et al., 2011, Prakash et al., 2016, Karn et al., 2015, Fu and Liu, 2016, Cerqueira et al., 2018, Zhong et al., 2016, Fu and Liu, 2016, Zaruba et al., 2005, Bröder and Sommerfeld, 2007, Hosokawa et al., 2009) can deal with complicated conditions such as high void fraction flows, severe bubble deformation and overlapping, etc. These algorithms can reduce the cost of processing bubbly flow images considerably and provide detailed information about bubble size distribution, shape, volume, etc. One issue in developing these algorithms is the way to benchmark the accuracy of these algorithms. Currently, the benchmark strategies can be divided into two categories. The first one is to compare the algorithm with different measurement techniques, which include conductivity probe (Worosz et al., 2016, Wang et al., 2018), X-ray (Song and Liu, 2018), or global gas and liquid flow meters. The benchmark with global instruments provides an overall error estimation without local uncertainty information. The comparison with conductivity probe or X-ray method can be made for time- or line-averaged parameters. The other issue in this benchmarking strategy is that these measurement methods may contain large uncertainty and can may not be used to assess image processing algorithms (Wang et al., 2018).
The second strategy is to use synthetic images as the benchmarking data (Fu and Liu, 2016, Fu and Liu, 2018). With the automation of image synthesizing process, the cost and time for the benchmarking of image processing algorithm can be reduced. The primary error source for this method comes from the gap between the real bubbly flow images and the synthetic ones. Current algorithms for synthetic images are mostly limited to generating simple bubble shapes such as spherical or elliptical bubbles. These physical models usually assume the bubble edge intensity follows a concentric circular/elliptical arrangements (CCA) (Strokina et al., 2016, Ilonen et al., 2018). With given bubble size and distribution information, synthetic bubbly flow images can be generated for benchmarking purpose. However, these algorithms are not capable of modeling fine structures of bubble shapes and intensity variations. Synthetic bubbles of different size can have a similar intensity distribution. Therefore, image processing algorithms benchmarked with these synthetic images can have a quite different performance in processing real bubbly flow images.
Conventional image processing algorithms are mostly based on certain features of bubble images such as curvature, intensity gradient, and topology information. The feature design and selection require the expertise in the related field to achieve an optimized performance. The deep learning algorithms such as convolutional neural networks (CNN), which do not require the input of extracted features from images, can be an alternative solution for bubble detection (Ilonen et al., 2018, Poletaev et al., 2016). The CNN can process bubbly flow image in their raw form without pre-processing for number density detection. For those supervised deep learning algorithms, labeled images are required for algorithm training. Presently, the researches in using deep learning algorithm for bubbly recognition and separation in the chemical and nuclear engineering fields are limited. One of the bottlenecks is the lack of a large amount of high-quality labeled data for algorithm training. In other fields, using synthetic data becomes a trend for training deep learning algorithms to reduce the manual labeling cost and to improve the efficiency. However, this can be effective only if we can generate synthetic data which is close to the real world (Tremblay et al., 2018).
To bridge the gap between real bubbly flow images and synthetic ones, we proposed a new approach called bubble generative adversarial networks (BubGAN) for generating realistic bubbly flow images. Conventional GAN algorithms can generate realistic images but they cannot be directly used in this research.
The bubbly flow images directly generated with GAN have no label information of the bubbles. In addition, the resolution of the generated images with GAN is relatively low compared to the recorded high-resolution images from high speed cameras. To overcome these shortcomings, the proposed BubGAN algorithm adopts the “divide and conquer strategy” to achieve high resolution labeled image generation. The BubGAN combines the conventional image processing algorithms (Fu and Liu, 2016, Fu and Liu, 2016) and the generative adversarial networks (Goodfellow et al., 2014, Radford et al., 2015) for realistic bubble synthesis. In the algorithm, the GAN will be only responsible for single bubble generation rather than generating the bubbly flow images directly. The image processing algorithm is responsible for GAN training data preparation and bubble assembling for bubbly flow image synthesis. With given bubbly flow boundary conditions, the synthetic bubbly flow images can be generated by assembling single synthetic bubbles on an image background canvas. In these images, bubble location, edge boundary, rotation, etc., of all bubbles will be labeled for either existing algorithm benchmarking or development of new algorithms.
The structure of this paper is organized as follows. In Section 2, the experimental setup for acquiring bubbly flow images is described. High-speed images are recorded in an upward rectangular two-phase flow test loop. In Section 3, the methodologies used in the BubGAN are introduced. A million-bubble database is generated for bubbly flow image synthesis with the BubGAN algorithm. Section 4 presents both qualitative and quantitative studies of the BubGAN results.
Section snippets
Experimental setup
In this study, the experimental bubbly flow images are recorded by a high-speed camera in a rectangular channel. The schematic of the test facility is shown in Fig. 1 (a). The rectangular channel is designed for adiabatic air-water two-phase upflow at room temperature and atmospheric pressure. The 3.0 m tall test section features a 30 mm × 10 mm rectangular cross section. Compressed air injected into the test section is measured by four gas flow meters based on the laminar differential pressure
Methodology
This section presents the methodologies used in the BubGAN for the generation of synthetic bubbly flow images. As shown in Fig. 2, the BubGAN algorithm consists of three major steps. The first step is to process the recorded high-speed images to extract single bubble images. The second step trains a conditional GAN to generate synthetic bubble images based on the training data acquired from the previous step. This deep convolutional generative adversarial networks (DCGAN) is conditioned on four
Quantitatively assessment of BubGAN
It becomes feasible to generate realistic bubbly flow images using the proposed BubGAN algorithm. To assess the diversity of the million-bubble database, two sets of synthetic bubbles sampled from the database are shown in Fig. 7. The first image collection shows bubbles with various aspect ratios and rotation angles. The second image collection shows bubbles with various edge ratios and circularities. It is seen that the million-bubble database contains bubbles with a large variance in bubble
Conclusion
This paper proposed a BubGAN algorithm for bubbly flow image synthesis. The BubGAN combines image processing algorithms and the conditional-GAN to achieve automatic bubbly flow image generation and labeling. The generated bubbly flow images show a significant improvement compared to the conventional bubble models. This can help to assess the accuracy of existing image processing algorithms and to guide future algorithm development. The deep learning based object counting algorithms can also
Conflict of interest
The authors declared that there is no conflict of interest.
Acknowledgment
We would like to acknowledge Yufeng Ma, Srijan Sengupta and Ahmed Ibrahim for their helpful discussions on this project. We would like to thank Sung-Ho Bae for his discussion on the implementation of bubble features in the conditional GAN.
References (36)
- et al.
Image processing techniques for the measurement of two-phase bubbly pipe flows using particle image and tracking velocimetry (PIV/PTV)
Chem. Eng. Sci.
(2018) - et al.
Hydrodynamics and heat transfer in an inclined bubbly flow
Int. J. Heat Mass Transf.
(2018) - et al.
Bacterial colony counting with convolutional neural networks in digital microbiology imaging
Pattern Recognit.
(2017) - et al.
Development of a robust image processing technique for bubbly flow measurement in a narrow rectangular channel
Int. J. Multiph. Flow.
(2016) - et al.
Experimental study of bubbly flow using image processing techniques
Nucl. Eng. Des.
(2016) - et al.
Comparison of bubble detectors and size distribution estimators
Pattern Recognit. Lett.
(2018) - et al.
An integrative image measurement technique for dense bubbly flows with a wide size distribution
Chem. Eng. Sci.
(2015) - et al.
Development of an image measurement technique for size distribution in dense bubbly flows
Chem. Eng. Sci.
(2013) - et al.
Experimental study of non-uniform inlet conditions and three-dimensional effects of vertical air-water two-phase flow in a narrow rectangular duct
Int. J. Heat Fluid Flow.
(2013) - et al.
The development of two-phase flow structures in air-water planar bubble jets
Int. J. Multiph. Flow.
(2013)
Characterization of droplet sizes in large scale oil-water flow downstream from a globe valve
Int. J. Multiph. Flow.
Sensitivity studies on the multi-sensor conductivity probe measurement technique for two-phase flows
Nucl. Eng. Des.
Vanga, Experimental study on bubble motion in a rectangular bubble column using high-speed video observations
Flow Meas Instrum
A flexible image analysis method for measuring bubble parameters
Chem. Eng. Sci.
Planar shadow image velocimetry for the analysis of the hydrodynamics in bubbly flows
Meas. Sci. Technol.
3D bubble reconstruction using multiple cameras and space carving method
Meas. Sci. Technol.
Cited by (38)
Neural networks for classification and segmentation of thermally-induced droplet breakup in spray-flame synthesis
2024, Journal of Aerosol ScienceDeep generative modeling-based data augmentation with demonstration using the BFBT benchmark void fraction datasets
2023, Nuclear Engineering and DesignApplication of deep learning for segmentation of bubble dynamics in subcooled boiling
2023, International Journal of Multiphase FlowVoid fraction detection technology of gas-liquid two-phase bubbly flow based on convolutional neural network
2023, Experimental Thermal and Fluid ScienceDevelopment and assessment of a particle tracking velocimetry (PTV) measurement technique for the experimental investigation of oil drops behaviour in dispersed oil–water two-phase flow within a centrifugal pump impeller
2023, International Journal of Multiphase FlowCitation Excerpt :Then, by training the model, the authors could input bubble parameters, such as aspect ratio, rotation angle, circularity and edge ratio, and the GAN returned a bubble image. By adding different bubble images, the technique described in Fu and Liu (2019) was able to create high-quality synthetic images. In the present work, a similar procedure, but with a simpler GAN model than the one adopted by Fu and Liu (2019), is applied to generate dispersed O/W synthetic flow images.
An intelligent strategy for phase change heat and mass transfer: Application of machine learning
2023, Advances in Heat Transfer