Top

International Journal of Computer Vision

Published in:

07-06-2023 | S.I. : Physics-Based Vision meets Deep Learning

Making the Invisible Visible: Toward High-Quality Terahertz Tomographic Imaging via Physics-Guided Restoration

Authors: Weng-Tai Su, Yi-Chun Hung, Po-Jen Yu, Shang-Hua Yang, Chia-Wen Lin

Published in: International Journal of Computer Vision | Issue 9/2023

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Terahertz (THz) tomographic imaging has recently attracted significant attention thanks to its non-invasive, non-destructive, non-ionizing, material-classification, and ultra-fast nature for object exploration and inspection. However, its strong water absorption nature and low noise tolerance lead to undesired blurs and distortions of reconstructed THz images. The diffraction-limited THz signals highly constrain the performances of existing restoration methods. To address the problem, we propose a novel multi-view Subspace-Attention-guided Restoration Network (SARNet) that fuses multi-view and multi-spectral features of THz images for effective image restoration and 3D tomographic reconstruction. To this end, SARNet uses multi-scale branches to extract intra-view spatio-spectral amplitude and phase features and fuse them via shared subspace projection and self-attention guidance. We then perform inter-view fusion to further improve the restoration of individual views by leveraging the redundancies between neighboring views. Here, we experimentally construct a THz time-domain spectroscopy (THz-TDS) system covering a broad frequency range from 0.1 to 4 THz for building up a temporal/spectral/spatial/material THz database of hidden 3D objects. Complementary to a quantitative evaluation, we demonstrate the effectiveness of our SARNet model on 3D THz tomographic reconstruction applications.

previous article Semi-Supervised Domain Generalization with Stochastic StyleMatch

next article Skeletonizing Caenorhabditis elegans Based on U-Net Architectures Trained with a Multi-worm Low-Resolution Synthetic Dataset

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

Project site: https://github.com/wtnthu/THz_Tomography.

Dataset site: https://github.com/wtnthu/THz_Data.

Abbas, A., Abdelsamea, M., & Gaber, M. M. (2021). Classification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network. Applied Intelligence, 51(2), 854–864.CrossRef

Abraham, E., Younus, A., Delagnes, T. C., & Mounaix, P. (2010). Non-invasive investigation of art paintings by terahertz imaging. Applied Physics A, 100(3), 585–590.CrossRef

Born, M., & Wolf, E. (2013). Principles of optics: Electromagnetic theory of propagation, interference and diffraction of light.

Bowman, T., Chavez, T., Khan, K., Wu, J., Chakraborty, A., Rajaram, N., Bailey, K., & El-Shenawee, M. (2018). Pulsed terahertz imaging of breast cancer in freshly excised murine tumors. Journal of Biomedical Optics, 23(2), 026004.CrossRef

Calvin, Y., Shuting, F., Yiwen, S., & Emma, P.-M. (2012). The potential of terahertz imaging for cancer diagnosis: A review of investigations to date. Quantitative Imaging in Medicine and Surgery, 2(1), 33.

Cao, J., Li, Y., Zhang, K., & Van Gool, L. (2021). Video super-resolution transformer. arXiv preprint arXiv:2106.06847.

Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020). End-to-end object detection with transformers. In Proceedings of European conference on computer vision (pp. 213–229). Springer.

Chapman, D., Homlinson, W., Johnston, R., Washburn, D., Pisano, E., Gmür, N., Zhong, Z., Menk, R., Arfelli, F., & Sayers, D. (1997). Diffraction enhanced X-ray imaging. Journal Physics in Medicine & Biology, 42(11), 2015.CrossRef

Chen, H., Wang, Y., Guo, T., Xu, C., Deng, Y., Liu, Z., Ma, S., Xu, C., Xu, C., & Gao, W. (2021). Pre-trained image processing transformer. In Proceedings of IEEE/CVF conference on computer vision and pattern recognition (pp. 12299–12310).

Cheng, S., Wang, Y., Huang, H., Liu, D., Fan, H., & Liu, S. (2021). NBNet: Noise basis learning for image denoising with subspace projection. In Proceedings of IEEE/CVF international conference on computer vision and pattern recognition (pp. 4896–4906).

Clarke, L., Velthuizen, R., Camacho, M., Heine, J., Vaidyanathan, M., Hall, L., Thatcher, R., & Silbiger, M. (1995). MRI segmentation: Methods and applications. Magnetic Resonance Imaging, 13(3), 343–368.CrossRef

Cloetens, P., Barrett, R., Baruchel, J., Guigay, J.-P., & Schlenker, M. (1996). Phase objects in synchrotron radiation hard X-ray imaging. Journal of Physics D: Applied Physics, 29(1), 133.CrossRef

de Gonzalez, A. B., & Darby, S. (2004). Risk of cancer from diagnostic X-rays: Estimates for the UK and 14 other countries. The Lancet, 363(9406), 345–351.CrossRef

Dorney, T. D., Baraniuk, R. G., & Mittleman, D. M. (2001). Material parameter estimation with terahertz time-domain spectroscopy. Journal of the Optical Society of America A, 18(7), 1562–1571.CrossRef

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., & Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

Fitzgerald, R. (2000). Phase-sensitive X-ray imaging. Physics Today, 53(7), 23–26.CrossRef

Fukunaga, K. (2016). THz technology applied to cultural heritage in practice

Geladi, P., Burger, J., & Lestander, T. (2004). Hyperspectral imaging: Calibration problems and solutions. Chemometrics and Intelligent Laboratory Systems, 72(2), 209–217.CrossRef

Hack, E., & Zolliker, P. (2014). Terahertz holography for imaging amplitude and phase objects. Optics Express, 22(13), 16079–16086.CrossRef

He, K., Zhang, X., Ren, S., Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In Proceedings of IEEE/CVF international conference on computer vision (pp. 1026–1034).

Hung, Y.-C., & Yang, S.-H. (2019a). Kernel size characterization for deep learning terahertz tomography (pp. 1–2).

Hung, Y.-C., & Yang, S.-H. (2019b). Terahertz deep learning computed tomography. In Proceedings of international infrared, millimeter, and terahertz waves (pp. 1–2). IEEE.

Janke, C., Först, M., Nagel, M., Kurz, H., & Bartels, A. (2005). Asynchronous optical sampling for high-speed characterization of integrated resonant terahertz sensors. Optics Letters, 30(11), 1405–1407.CrossRef

Jansen, C., Wietzke, S., Peters, O., Scheller, M., Vieweg, N., Salhi, M., Krumbholz, N., Jördens, C., Hochrein, T., & Koch, M. (2010). Terahertz imaging: Applications and perspectives. Applied Optics, 49(19), 48–57.CrossRef

Jin, K. H., McCann, M. T., Froustey, E., & Unser, M. (2017). Deep convolutional neural network for inverse problems in imaging. IEEE Transactions on Image Processing, 26(9), 4509–4522.MathSciNetCrossRefMATH

Kak, A. C. (2001). Algorithms for reconstruction with nondiffracting sources. Principles of Computerized Tomographic Imaging, 49–112.

Kamruzzaman, M., ElMasry, G., Sun, D.-W., & Allen, P. (2011). Application of NIR hyperspectral imaging for discrimination of lamb muscles. Journal of Food Engineering, 104(3), 332–340.CrossRef

Kang, E., Min, J., & Ye, J. C. (2017). A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction. Journal of Medical physics, 44(10), 360–375.CrossRef

Kawase, K., Ogawa, Y., Watanabe, Y., & Inoue, H. (2003). Non-destructive terahertz imaging of illicit drugs using spectral fingerprints. Optics Express, 11(20), 2549–2554.CrossRef

Kim, J., Lim, H., Ahn, S.C., Lee, S. (2018). RGBD camera based material recognition via surface roughness estimation. In: Proceedings of IEEE Winter Conference Applied Computer Vision (pp. 1963–1971).

Li, X., & Jarrahi, M. (2020). A 63-pixel plasmonic photoconductive terahertz focal-plane array. In Proceedings of IEEE/MTT-S international microwave symposium (IMS) (pp. 91–94).

Liu, F., Jang, H., Kijowski, R., Bradshaw, T., & McMillan, A. B. (2018). Deep learning MR imaging-based attenuation correction for PET/MR imaging. Radiology, 286(2), 676–684.CrossRef

Ljubenovic, M., Bazrafkan, S., Beenhouwer, J. D., & Sijbers, J. (2020). CNN-based deblurring of terahertz images (pp. 323–330).

Mao, X., Shen, C., & Yang, Y.-B. (2016) Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In Proceedings of the advances in neural information processing systems (pp. 2802–2810).

Meyer, C. D. (2000). Matrix analysis and applied linear algebra. SIAM.CrossRef

Mittleman, D. M. (2018). Twenty years of terahertz imaging. Optics Express, 26(8), 9417–9431.CrossRef

Mittleman, D., Gupta, M., Neelamani, R., Baraniuk, R., Rudd, J., & Koch, M. (1999). Recent advances in terahertz imaging. Applied Physics B, 68(6), 1085–1094.CrossRef

Nunes-Pereira, E., Peixoto, H., Teixeira, J., & Santos, J. (2020). Polarization-coded material classification in automotive LIDAR aiming at safer autonomous driving implementations. Applied Optics, 59(8), 2530–2540.CrossRef

Ozdemir, A., & Polat, K. (2020). Deep learning applications for hyperspectral imaging: A systematic review. Journal of the Institute of Electronics and Computer, 2(1), 39–56.CrossRef

Peterson, J., Paerels, F., Kaastra, J., Arnaud, M., Reiprich, T., Fabian, A., Mushotzky, R., Jernigan, J., & Sakelliou, I. (2001). X-ray imaging-spectroscopy of Abell 1835. Journal of Astronomy & Astrophysics, 365(1), 104–109.CrossRef

Popescu, D. C., & Ellicar, A. D. (2010). Point spread function estimation for a terahertz imaging system. EURASIP Journal on Advances in Signal Processing, 2010(1), 575817.CrossRef

Popescu, D.C., Hellicar, A., & Li, Y. (2009). Phantom-based point spread function estimation for terahertz imaging system (pp. 629–639).

Qin, X., Wang, X., Bai, Y., Xie, X., & Jia, H. (2020). FFA-Net: Feature fusion attention network for single image dehazing. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34, pp. 11908–11915).

Recur, B., Guillet, J.-P., Manek-Hönninger, I., Delagnes, J.-C., Benharbone, W., Desbarats, P., Domenger, J.-P., Canioni, L., & Mounaix, P. (2012). Propagation beam consideration for 3D THz computed tomography. Optics Express, 20(6), 5817–5829.CrossRef

Recur, B., Younus, A., Salort, S., Mounaix, P., Chassagne, B., Desbarats, P., Caumes, J., & Abraham, E. (2011). Investigation on reconstruction methods applied to 3D terahertz computed tomography. Optics Express, 19(6), 5105–5117.CrossRef

Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of international conference on medical image computing and computer-assisted intervention (pp. 234–241).

Rotermund, H. H., Engel, W., Jakubith, S., Von Oertzen, A., & Ertl, G. (1991). Methods and application of UV photoelectron microscopy in heterogenous catalysis. Ultramicroscopy, 36(1–3), 164–172.CrossRef

Round, A. R., Wilkinson, S. J., Hall, C. J., Rogers, K. D., Glatter, O., Wess, T., & Ellis, I. O. (2005). A preliminary study of breast cancer diagnosis using laboratory based small angle x-ray scattering. Physics in Medicine & Biology, 50(17), 4159.CrossRef

Saeedkia, D. (2013). Handbook of terahertz technology for imaging, sensing and communications (pp. 542–578). Cambridge: Woodhead Publishing.CrossRef

Sakdinawat, A., & Attwood, D. (2010). Nanoscale X-ray imaging. Nature Photonics, 4(12), 840.CrossRef

Schultz, R., Nielsen, T., Zavaleta, R. J., Wyatt, R., & Garner, H. (2001). Hyperspectral imaging: A novel approach for microscopic analysis. Cytometry, 43(4), 239–247.CrossRef

Su, W.-T., Hung, Y.-C., Yu, P.-J., Lin, C.-W., & Yang, S.-H. (2023). Physics-guided terahertz computational imaging: A tutorial on sate-of-the-art techniques. IEEE Signal Processing Magazine, 40(2), 32–45.CrossRef

Su, W.-T., Hung, Y.-C., Yu, P.-J., Yang, S.-H., & Lin, C.-W. (2022). Seeing through a black box: Toward high-quality terahertz tomographic imaging via multi-scale spatio-spectral image fusion. In Proceedings of the European conference on computer vision.

Tuan, T. M., Fujita, H., Dey, N., Ashour, A. S., Ngoc, T. N., & Chu, D.-T. (2018). Dental diagnosis from X-ray images: An expert system based on fuzzy computing. Biomedical Signal Processing and Control, 39, 64–73.CrossRef

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., & Polosukhin, I. (2017) Attention is all you need. In Proceedings of advances in neural information processing systems (vol. 30).

Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., & Li, H. (2022). Uformer: A general U-shaped transformer for image restoration. In Proceedings of IEEE/CVF conference on computer vision and pattern recognition (pp. 17683–17693).

Wong, T. M., Kahl, M., & Bolívar, P.H., Kolb, A. (2019). Computational image enhancement for frequency modulated continuous wave (FMCW) THz image. Journal of Infrared, Millimeter, and Terahertz Waves, 40(7), 775–800.

Wong, T. M., Kahl, M., Haring-Bolívar, P., Kolb, A., & Möller, M. (2019). Training auto-encoder-based optimizers for terahertz image reconstruction (pp. 93–106).

Wu, B., Xu, C., Dai, X., Wan, A., Zhang, P., Yan, Z., Tomizuka, M., Gonzalez, J., Keutzer, K., & Vajda, P. (2020). Visual transformers: Token-based image representation and processing for computer vision. arXiv preprint arXiv:2006.03677.

Xie, H., Yao, H., Zhang, S. P., Zhou, S. C., & Sun, W. X. (2020). Pix2Vox++: Multi-scale context-aware 3D object reconstruction from single and multiple images. International Journal of Computer Vision, 128(12), 2919–2935.CrossRef

Xie, X. (2008). A review of recent advances in surface defect detection using texture analysis techniques. ELCVIA: Electronic Letters on Computer Vision and Image Analysis, 1–22

Yujiri, L., Shoucri, M., & Moffa, P. (2003). Passive millimeter wave imaging. IEEE Microwave Magazine, 4(3), 39–50.CrossRef

Zhang, H., Goodfellow, I., Metaxas, D., & Odena, A. (2019). Self-attention generative adversarial networks. In Proceedings of international conference on machine learning (pp. 7354–7363).

Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017). Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7), 3142–3155.MathSciNetCrossRefMATH

Zhang, K., Zuo, W. M., & Zhang, L. (2018). FFDNet: Toward a fast and flexible solution for CNN-based image denoising. IEEE Transactions on Image Processing, 27(9), 4608–4622.MathSciNetCrossRef

Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2020). Residual dense network for image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence.

Zhou, S., Zhang, J., Pan, J., Xie, H., Zuo, W., & Ren, J. (2019). Spatio-temporal filter adaptive network for video deblurring. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 2482–2491).

Zhu, B., Liu, J. Z., Cauley, S. F., Rosen, R. B., & Rosen, M. S. (2018). Image reconstruction by domain-transform manifold learning. Nature, 555(7697), 487–492.CrossRef

Title: Making the Invisible Visible: Toward High-Quality Terahertz Tomographic Imaging via Physics-Guided Restoration
Authors: Weng-Tai Su
Yi-Chun Hung
Po-Jen Yu
Shang-Hua Yang
Chia-Wen Lin
Publication date: 07-06-2023
Publisher: Springer US
Published in: International Journal of Computer Vision / Issue 9/2023
Print ISSN: 0920-5691
Electronic ISSN: 1573-1405
DOI: https://doi.org/10.1007/s11263-023-01812-y

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 9/2023

A Family of Approaches for Full 3D Reconstruction of Objects with Complex Surface Reflectance

Learning to Remove Shadows from a Single Image

Blur Invariants for Image Recognition

On Making SIFT Features Affine Covariant

Perspective-1-Ellipsoid: Formulation, Analysis and Solutions of the Camera Pose Estimation Problem from One Ellipse-Ellipsoid Correspondence

Learning Geometric Transformation for Point Cloud Completion

Premium Partner