Skip to main content
Top
Published in: International Journal of Computer Vision 11/2023

04-07-2023 | Manuscript

Deep Physics-Guided Unrolling Generalization for Compressed Sensing

Authors: Bin Chen, Jiechong Song, Jingfen Xie, Jian Zhang

Published in: International Journal of Computer Vision | Issue 11/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

By absorbing the merits of both the model- and data-driven methods, deep physics-engaged learning scheme achieves high-accuracy and interpretable image reconstruction. It has attracted growing attention and become the mainstream for inverse imaging tasks. Focusing on the image compressed sensing (CS) problem, we find the intrinsic defect of this emerging paradigm, widely implemented by deep algorithm-unrolled networks, in which more plain iterations involving real physics will bring enormous computation cost and long inference time, hindering their practical application. A novel deep Physics-guided unRolled recovery Learning (RL) framework is proposed by generalizing the traditional iterative recovery model from image domain (ID) to the high-dimensional feature domain (FD). A compact multiscale unrolling architecture is then developed to enhance the network capacity and keep real-time inference speeds. Taking two different perspectives of optimization and range-nullspace decomposition, instead of building an algorithm-specific unrolled network, we provide two implementations: PRL-PGD and PRL-RND. Experiments exhibit the significant performance and efficiency leading of PRL networks over other state-of-the-art methods with a large potential for further improvement and real application to other inverse imaging problems or optimization models.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Footnotes
1
In the field of CS imaging, the condition of orthogonality \(\textbf{A}\textbf{A}^\top =\textbf{I}_M\) is widely implemented by orthogonalizing an i.i.d. random Gaussian matrix or a Hadamard matrix (Zhang et al., 2014b).
 
2
Note that this phenomenon indicates that our baseline network is “degraded” with large unrolled stage number instead of “overfitted” as the training loss and test PSNR both become worse. It has been previously discovered and studied in (He et al., 2016).
 
3
For reproducible research, the complete source code and pre-trained models of our PRL networks are available at https://​github.​com/​Guaishou74851/​PRL.
 
Literature
go back to reference Adler, J., & Öktem, O. (2018). Learned primal-dual reconstruction. IEEE Transactions on Medical Imaging, 37(6), 1322–1332. Adler, J., & Öktem, O. (2018). Learned primal-dual reconstruction. IEEE Transactions on Medical Imaging, 37(6), 1322–1332.
go back to reference Agustsson, E., & Timofte, R. (2017). NTIRE 2017 challenge on single image super-resolution: dataset and study. In Proceedings of IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp. 126–135. Agustsson, E., & Timofte, R. (2017). NTIRE 2017 challenge on single image super-resolution: dataset and study. In Proceedings of IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp. 126–135.
go back to reference Björck, Å., Elfving, T., & Strakos, Z. (1998). Stability of Conjugate Gradient and lanczos methods for linear least squares problems. SIAM Journal on Matrix Analysis and Applications, 19(3), 720–736.MathSciNetMATH Björck, Å., Elfving, T., & Strakos, Z. (1998). Stability of Conjugate Gradient and lanczos methods for linear least squares problems. SIAM Journal on Matrix Analysis and Applications, 19(3), 720–736.MathSciNetMATH
go back to reference Blumensath, T., & Davies, M. E. (2009). Iterative hard thresholding for compressed sensing. Applied and Computational Harmonic Analysis, 27(3), 265–274.MathSciNetMATH Blumensath, T., & Davies, M. E. (2009). Iterative hard thresholding for compressed sensing. Applied and Computational Harmonic Analysis, 27(3), 265–274.MathSciNetMATH
go back to reference Boufounos, Petros. T., & Baraniuk, Richard. G. (2008). 1-bit Compressive sensing. In Proceedings of IEEE conference on information sciences and systems (CISS), pp. 16–21. Boufounos, Petros. T., & Baraniuk, Richard. G. (2008). 1-bit Compressive sensing. In Proceedings of IEEE conference on information sciences and systems (CISS), pp. 16–21.
go back to reference Cai, J.-F., Candès, E. J., & Shen, Z. (2010). A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20(4), 1956–1982.MathSciNetMATH Cai, J.-F., Candès, E. J., & Shen, Z. (2010). A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20(4), 1956–1982.MathSciNetMATH
go back to reference Candès, E. J., & Wakin, M. B. (2008). An introduction to compressive sampling. IEEE Signal Processing Magazine, 25(2), 21–30. Candès, E. J., & Wakin, M. B. (2008). An introduction to compressive sampling. IEEE Signal Processing Magazine, 25(2), 21–30.
go back to reference Chen, B., & Zhang, J. (2022). Content-aware scalable deep compressed sensing. IEEE Transactions on Image Processing, 31, 5412–5426. Chen, B., & Zhang, J. (2022). Content-aware scalable deep compressed sensing. IEEE Transactions on Image Processing, 31, 5412–5426.
go back to reference Chen, D., & Davies, Mike. E. (2020). Deep decomposition learning for inverse imaging problems. In Proceedings of European conference on computer vision (ECCV), pp. 510–526. Chen, D., & Davies, Mike. E. (2020). Deep decomposition learning for inverse imaging problems. In Proceedings of European conference on computer vision (ECCV), pp. 510–526.
go back to reference Chen, D., Tachella, J., & Davies, M. E. (2021a). Equivariant imaging: learning beyond the range space. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4379–4388. Chen, D., Tachella, J., & Davies, M. E. (2021a). Equivariant imaging: learning beyond the range space. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4379–4388.
go back to reference Chen, H., Zhang, Y., Zhang, W., Liao, P., Li, K., Zhou, J., & Wang, G. (2017). Low-dose CT via convolutional neural network. Biomedical Optics Express, 8(2), 679–694. Chen, H., Zhang, Y., Zhang, W., Liao, P., Li, K., Zhou, J., & Wang, G. (2017). Low-dose CT via convolutional neural network. Biomedical Optics Express, 8(2), 679–694.
go back to reference Chen, J., Sun, Y., Liu, Q., & Huang, R. (2020). Learning memory augmented cascading network for compressed sensing of images. In Proceedings of European conference on computer vision (ECCV), pp. 513–529. Chen, J., Sun, Y., Liu, Q., & Huang, R. (2020). Learning memory augmented cascading network for compressed sensing of images. In Proceedings of European conference on computer vision (ECCV), pp. 513–529.
go back to reference Chen, Y., & Pock, T. (2016). Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1256–1272. Chen, Y., & Pock, T. (2016). Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1256–1272.
go back to reference Chen, Z., Guo, W., Feng, Y., Li, Y., Zhao, C., Ren, Y., & Shao, L. (2021). Deep-learned regularization and proximal operator for image compressive sensing. IEEE Transactions on Image Processing, 30, 7112–7126.MathSciNet Chen, Z., Guo, W., Feng, Y., Li, Y., Zhao, C., Ren, Y., & Shao, L. (2021). Deep-learned regularization and proximal operator for image compressive sensing. IEEE Transactions on Image Processing, 30, 7112–7126.MathSciNet
go back to reference Coban, S., Andriiashen, V., & Ganguly, P. (2020). Apple CT Data: simulated parallel-beam tomographic datasets. Zenodo. Coban, S., Andriiashen, V., & Ganguly, P. (2020). Apple CT Data: simulated parallel-beam tomographic datasets. Zenodo.
go back to reference Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080–2095.MathSciNet Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080–2095.MathSciNet
go back to reference Denker, A., Schmidt, M., Leuschner, J., Maass, P., & Behrmann, J. (2020). Conditional normalizing flows for low-dose computed tomography image reconstruction. arXiv preprint arXiv:2006.06270. Denker, A., Schmidt, M., Leuschner, J., Maass, P., & Behrmann, J. (2020). Conditional normalizing flows for low-dose computed tomography image reconstruction. arXiv preprint arXiv:​2006.​06270.
go back to reference Dong, C., Loy, C. C., He, K., & Tang, X. (2014a). Learning a deep convolutional network for image super-resolution. In Proceedings of European conference on computer vision (ECCV), pp. 184–199. Dong, C., Loy, C. C., He, K., & Tang, X. (2014a). Learning a deep convolutional network for image super-resolution. In Proceedings of European conference on computer vision (ECCV), pp. 184–199.
go back to reference Dong, W., Shi, G., Li, X., Ma, Y., & Huang, F. (2014b). Compressive sensing via nonlocal low-rank regularization. IEEE Transactions on Image Processing, 23(8), 3618–3632.MathSciNetMATH Dong, W., Shi, G., Li, X., Ma, Y., & Huang, F. (2014b). Compressive sensing via nonlocal low-rank regularization. IEEE Transactions on Image Processing, 23(8), 3618–3632.MathSciNetMATH
go back to reference Donoho, D. L. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52(4), 1289–1306.MathSciNetMATH Donoho, D. L. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52(4), 1289–1306.MathSciNetMATH
go back to reference Elad, M. (2010). Sparse and redundant representations: from theory to applications in signal and image processing (Vol. 2). Springer.MATH Elad, M. (2010). Sparse and redundant representations: from theory to applications in signal and image processing (Vol. 2). Springer.MATH
go back to reference Fan, Z.-E., Lian, F., & Quan, J.-N. (2022). Global sensing and measurements reuse for image compressed sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 8954–8963. Fan, Z.-E., Lian, F., & Quan, J.-N. (2022). Global sensing and measurements reuse for image compressed sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 8954–8963.
go back to reference Fowler, J. E., Mun, S., Tramel, E. W., et al. (2012). Block-based compressed sensing of images and video. Foundations and Trends in Signal Processing, 4(4), 297–416.MATH Fowler, J. E., Mun, S., Tramel, E. W., et al. (2012). Block-based compressed sensing of images and video. Foundations and Trends in Signal Processing, 4(4), 297–416.MATH
go back to reference Gan, L. (2007). Block compressed sensing of natural images. In Proceedings of IEEE international conference on digital signal processing (ICDSP), pp. 403–406. IEEE. Gan, L. (2007). Block compressed sensing of natural images. In Proceedings of IEEE international conference on digital signal processing (ICDSP), pp. 403–406. IEEE.
go back to reference Gilton, D., Ongie, G., & Willett, R. (2019). Neumann networks for linear inverse problems in imaging. IEEE Transactions on Computational Imaging, 6, 328–343.MathSciNet Gilton, D., Ongie, G., & Willett, R. (2019). Neumann networks for linear inverse problems in imaging. IEEE Transactions on Computational Imaging, 6, 328–343.MathSciNet
go back to reference Gu, J., & Dong, C. (2021). Interpreting super-resolution networks with local attribution maps. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 9199–9208. Gu, J., & Dong, C. (2021). Interpreting super-resolution networks with local attribution maps. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 9199–9208.
go back to reference He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 770–778. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 770–778.
go back to reference Huang, J. -B., Singh, A., & Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 5197–5206. Huang, J. -B., Singh, A., & Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 5197–5206.
go back to reference Huang, Y., Würfl, T., Breininger, K., Liu, L., Lauritsch, G., & Maier, A. (2018). Some investigations on robustness of deep learning in limited angle tomography. In Proceedings of international conference on medical image computing and computer-assisted intervention (MICCAI), pp. 145–153. Huang, Y., Würfl, T., Breininger, K., Liu, L., Lauritsch, G., & Maier, A. (2018). Some investigations on robustness of deep learning in limited angle tomography. In Proceedings of international conference on medical image computing and computer-assisted intervention (MICCAI), pp. 145–153.
go back to reference Jacques, L., Laska, J. N., Boufounos, P. T., & Baraniuk, R. G. (2013). Robust 1-bit compressive sensing via binary stable embeddings of sparse vectors. IEEE Transactions on Information Theory, 59(4), 2082–2102.MathSciNetMATH Jacques, L., Laska, J. N., Boufounos, P. T., & Baraniuk, R. G. (2013). Robust 1-bit compressive sensing via binary stable embeddings of sparse vectors. IEEE Transactions on Information Theory, 59(4), 2082–2102.MathSciNetMATH
go back to reference Kafle, S., Joseph, G., & Varshney, P. K. (2021). One-bit compressed sensing using untrained network prior. In Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 2875–2879. Kafle, S., Joseph, G., & Varshney, P. K. (2021). One-bit compressed sensing using untrained network prior. In Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 2875–2879.
go back to reference Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In Proceedings of international conference on learning representations (ICLR), pp. 1–15. Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In Proceedings of international conference on learning representations (ICLR), pp. 1–15.
go back to reference Kokkinos, F., & Lefkimmiatis, S. (2018). Deep image demosaicking using a cascade of convolutional residual denoising networks. In Proceedings of European conference on computer vision (ECCV), pp. 303–319. Kokkinos, F., & Lefkimmiatis, S. (2018). Deep image demosaicking using a cascade of convolutional residual denoising networks. In Proceedings of European conference on computer vision (ECCV), pp. 303–319.
go back to reference Kruse, J., Rother, C., & Schmidt, U. (2017). Learning to push the limits of efficient FFT-based image deconvolution. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4586–4594. Kruse, J., Rother, C., & Schmidt, U. (2017). Learning to push the limits of efficient FFT-based image deconvolution. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4586–4594.
go back to reference Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., & Ashok, A. (2016). ReconNet: Non-iterative reconstruction of images from compressively sensed measurements. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 449–458. Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., & Ashok, A. (2016). ReconNet: Non-iterative reconstruction of images from compressively sensed measurements. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 449–458.
go back to reference Lefkimmiatis, S. (2017). Non-local color image denoising with convolutional neural networks. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3587–3596. Lefkimmiatis, S. (2017). Non-local color image denoising with convolutional neural networks. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3587–3596.
go back to reference Lefkimmiatis, S. (2018). Universal denoising networks: A novel CNN architecture for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3204–3213. Lefkimmiatis, S. (2018). Universal denoising networks: A novel CNN architecture for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3204–3213.
go back to reference Leuschner, J., Schmidt, M., Ganguly, P. S., Andriiashen, V., Coban, S. B., Denker, A., Bauer, D., Hadjifaradji, A., Batenburg, K. J., Maass, P., et al. (2021). Quantitative comparison of deep learning-based image reconstruction methods for low-dose and sparse-angle CT applications. Journal of Imaging, 7(3), 44. Leuschner, J., Schmidt, M., Ganguly, P. S., Andriiashen, V., Coban, S. B., Denker, A., Bauer, D., Hadjifaradji, A., Batenburg, K. J., Maass, P., et al. (2021). Quantitative comparison of deep learning-based image reconstruction methods for low-dose and sparse-angle CT applications. Journal of Imaging, 7(3), 44.
go back to reference Li, Y., Li, K., Zhang, C., Montoya, J., & Chen, G.-H. (2019). Learning to reconstruct computed tomography images directly from sinogram data under a variety of data acquisition conditions. IEEE Transactions on Medical Imaging, 38(10), 2469–2481. Li, Y., Li, K., Zhang, C., Montoya, J., & Chen, G.-H. (2019). Learning to reconstruct computed tomography images directly from sinogram data under a variety of data acquisition conditions. IEEE Transactions on Medical Imaging, 38(10), 2469–2481.
go back to reference Liu, T., Chaman, A., Belius, D., & Dokmanic, I. (2020). Interpreting U-nets via task-driven multiscale dictionary learning. arXiv preprint arXiv:2011.12815. Liu, T., Chaman, A., Belius, D., & Dokmanic, I. (2020). Interpreting U-nets via task-driven multiscale dictionary learning. arXiv preprint arXiv:​2011.​12815.
go back to reference Liu, Y., Long, Z., & Zhu, C. (2018). Image completion using low tensor tree rank and total variation minimization. IEEE Transactions on Multimedia, 21(2), 338–350. Liu, Y., Long, Z., & Zhu, C. (2018). Image completion using low tensor tree rank and total variation minimization. IEEE Transactions on Multimedia, 21(2), 338–350.
go back to reference Liu, Y., Long, Z., Huang, H., & Zhu, C. (2019). Low CP rank and tucker rank tensor completion for estimating missing components in image data. IEEE Transactions on Circuits and Systems for Video Technology, 30(4), 944–954. Liu, Y., Long, Z., Huang, H., & Zhu, C. (2019). Low CP rank and tucker rank tensor completion for estimating missing components in image data. IEEE Transactions on Circuits and Systems for Video Technology, 30(4), 944–954.
go back to reference Long, Z., Liu, Y., Chen, L., & Zhu, C. (2019). Low rank tensor completion for multiway visual data. Signal Processing, 155, 301–316. Long, Z., Liu, Y., Chen, L., & Zhu, C. (2019). Low rank tensor completion for multiway visual data. Signal Processing, 155, 301–316.
go back to reference Long, Z., Zhu, C., Liu, J., & Liu, Y. (2021). Bayesian low rank tensor ring for image recovery. IEEE Transactions on Image Processing, 30, 3568–3580.MathSciNet Long, Z., Zhu, C., Liu, J., & Liu, Y. (2021). Bayesian low rank tensor ring for image recovery. IEEE Transactions on Image Processing, 30, 3568–3580.MathSciNet
go back to reference Long, Z., Zhu, C., Liu, J., Comon, P., & Liu, Y. (2022). Trainable subspaces for low rank tensor completion: Model and analysis. IEEE Transactions on Signal Processing, 70, 2502–2517.MathSciNet Long, Z., Zhu, C., Liu, J., Comon, P., & Liu, Y. (2022). Trainable subspaces for low rank tensor completion: Model and analysis. IEEE Transactions on Signal Processing, 70, 2502–2517.MathSciNet
go back to reference Lustig, M., Donoho, D., & Pauly, J. M. (2007). Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine, 58(6), 1182–1195. Lustig, M., Donoho, D., & Pauly, J. M. (2007). Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine, 58(6), 1182–1195.
go back to reference Lustig, M., Donoho, D. L., Santos, J. M., & Pauly, J. M. (2008). Compressed sensing MRI. IEEE Signal Processing Magazine, 25(2), 72–82. Lustig, M., Donoho, D. L., Santos, J. M., & Pauly, J. M. (2008). Compressed sensing MRI. IEEE Signal Processing Magazine, 25(2), 72–82.
go back to reference Ma, K., Duanmu, Z., Qingbo, W., Wang, Z., Yong, H., Li, H., & Zhang, L. (2016). Waterloo exploration database: New challenges for image quality assessment models. IEEE Transactions on Image Processing, 26(2), 1004–1016.MathSciNetMATH Ma, K., Duanmu, Z., Qingbo, W., Wang, Z., Yong, H., Li, H., & Zhang, L. (2016). Waterloo exploration database: New challenges for image quality assessment models. IEEE Transactions on Image Processing, 26(2), 1004–1016.MathSciNetMATH
go back to reference Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of IEEE international conference on computer vision (ICCV), 2, pp. 416–423. Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of IEEE international conference on computer vision (ICCV), 2, pp. 416–423.
go back to reference Mousavi, A., & Baraniuk, R. G. (2017). Learning to invert: Signal recovery via deep convolutional networks. In Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 2272–2276. Mousavi, A., & Baraniuk, R. G. (2017). Learning to invert: Signal recovery via deep convolutional networks. In Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 2272–2276.
go back to reference Mousavi, A., Patel, A. B., & Baraniuk, R. G. (2015). A deep learning approach to structured signal recovery. In Proceedings of IEEE allerton conference on communication, control, and computing, pp. 1336–1343. Mousavi, A., Patel, A. B., & Baraniuk, R. G. (2015). A deep learning approach to structured signal recovery. In Proceedings of IEEE allerton conference on communication, control, and computing, pp. 1336–1343.
go back to reference Mun, S., & Fowler, J. E. (2009). Block compressed sensing of images using directional transforms. In Proceedings of IEEE international conference on image processing (ICIP), pp. 3021–3024. Mun, S., & Fowler, J. E. (2009). Block compressed sensing of images using directional transforms. In Proceedings of IEEE international conference on image processing (ICIP), pp. 3021–3024.
go back to reference Nam, S., Davies, M. E., Elad, M., & Gribonval, R. (2013). The cosparse analysis model and algorithms. Applied and Computational Harmonic Analysis, 34(1), 30–56.MathSciNetMATH Nam, S., Davies, M. E., Elad, M., & Gribonval, R. (2013). The cosparse analysis model and algorithms. Applied and Computational Harmonic Analysis, 34(1), 30–56.MathSciNetMATH
go back to reference Niu, S., Gao, Y., Bian, Z., Huang, J., Chen, W., Gaohang, Yu., Liang, Z., & Ma, J. (2014). Sparse-View X-Ray CT reconstruction via total generalized variation regularization. Physics in Medicine & Biology, 59(12), 2997. Niu, S., Gao, Y., Bian, Z., Huang, J., Chen, W., Gaohang, Yu., Liang, Z., & Ma, J. (2014). Sparse-View X-Ray CT reconstruction via total generalized variation regularization. Physics in Medicine & Biology, 59(12), 2997.
go back to reference Parikh, N., Boyd, S., et al. (2014). Proximal algorithms. Foundations and in Optimization, 1(3), 127–239. Parikh, N., Boyd, S., et al. (2014). Proximal algorithms. Foundations and in Optimization, 1(3), 127–239.
go back to reference Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., & Antiga, L. et al. (2019). PyTorch: An imperative style, high-performance deep learning library. Proceedings of Neural Information Processing Systems (NeurIPS), 32. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., & Antiga, L. et al. (2019). PyTorch: An imperative style, high-performance deep learning library. Proceedings of Neural Information Processing Systems (NeurIPS), 32.
go back to reference Pelt, D. M., Batenburg, K. J., & Sethian, J. A. (2018). Improving tomographic reconstruction from limited data using mixed-scale dense convolutional neural networks. Journal of Imaging, 4(11), 128. Pelt, D. M., Batenburg, K. J., & Sethian, J. A. (2018). Improving tomographic reconstruction from limited data using mixed-scale dense convolutional neural networks. Journal of Imaging, 4(11), 128.
go back to reference Radon, J. (1986). On the determination of functions from their integral values along certain manifolds. IEEE Transactions on Medical Imaging, 5(4), 170–176. Radon, J. (1986). On the determination of functions from their integral values along certain manifolds. IEEE Transactions on Medical Imaging, 5(4), 170–176.
go back to reference Ravishankar, S., Ye, J. C., & Fessler, J. A. (2019). Image reconstruction: From sparsity to data-adaptive methods and machine learning. Proceedings of the IEEE, 108(1), 86–109. Ravishankar, S., Ye, J. C., & Fessler, J. A. (2019). Image reconstruction: From sparsity to data-adaptive methods and machine learning. Proceedings of the IEEE, 108(1), 86–109.
go back to reference Ren, C., He, X., Wang, C., & Zhao, Zhibo. (2021). Adaptive consistency prior based deep network for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 8596–8606. Ren, C., He, X., Wang, C., & Zhao, Zhibo. (2021). Adaptive consistency prior based deep network for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 8596–8606.
go back to reference Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of international conference on medical image computing and computer-assisted intervention (MICCAI), pp. 234–241. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of international conference on medical image computing and computer-assisted intervention (MICCAI), pp. 234–241.
go back to reference Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 1874–1883. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 1874–1883.
go back to reference Shi, W., Jiang, F., Liu, S., & Zhao, D. (2019a). Image compressed sensing using convolutional neural network. IEEE Transactions on Image Processing, 29, 375–388.MathSciNetMATH Shi, W., Jiang, F., Liu, S., & Zhao, D. (2019a). Image compressed sensing using convolutional neural network. IEEE Transactions on Image Processing, 29, 375–388.MathSciNetMATH
go back to reference Shi, W., Jiang, F., Liu, S., & Zhao, D. (2019b). Scalable convolutional neural network for image compressed sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 12290–12299. Shi, W., Jiang, F., Liu, S., & Zhao, D. (2019b). Scalable convolutional neural network for image compressed sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 12290–12299.
go back to reference Song, J., Chen, B., & Zhang, J. (2021). Memory-augmented deep unfolding network for compressive sensing. In Proceedings of ACM international conference on multimedia (ACM MM), pp. 4249–4258. Song, J., Chen, B., & Zhang, J. (2021). Memory-augmented deep unfolding network for compressive sensing. In Proceedings of ACM international conference on multimedia (ACM MM), pp. 4249–4258.
go back to reference Song, J., Chen, B., & Zhang, J. (2023a). Deep memory-augmented proximal unrolling network for compressive sensing. International Journal of Computer Vision, 1–20. Song, J., Chen, B., & Zhang, J. (2023a). Deep memory-augmented proximal unrolling network for compressive sensing. International Journal of Computer Vision, 1–20.
go back to reference Song, J., Chen, B., & Zhang, J. (2023b). Dynamic path-controllable deep unfolding network for compressive sensing. IEEE Transactions on Image Processing, 32, 2202–2214. Song, J., Chen, B., & Zhang, J. (2023b). Dynamic path-controllable deep unfolding network for compressive sensing. IEEE Transactions on Image Processing, 32, 2202–2214.
go back to reference Sun, J., Li, H., Zongben, X., et al. (2016). Deep ADMM-Net for compressive sensing MRI. Proceedings of Neural Information Processing Systems (NeurIPS), 29, 10–18. Sun, J., Li, H., Zongben, X., et al. (2016). Deep ADMM-Net for compressive sensing MRI. Proceedings of Neural Information Processing Systems (NeurIPS), 29, 10–18.
go back to reference Sun, Y., Chen, J., Liu, Q., Liu, B., & Guo, G. (2020). Dual-path attention network for compressed sensing image reconstruction. IEEE Transactions on Image Processing, 29, 9482–9495.MATH Sun, Y., Chen, J., Liu, Q., Liu, B., & Guo, G. (2020). Dual-path attention network for compressed sensing image reconstruction. IEEE Transactions on Image Processing, 29, 9482–9495.MATH
go back to reference Szczykutowicz, T. P., & Chen, G.-H. (2010). Dual energy CT using slow kVp switching acquisition and prior image constrained compressed sensing. Physics in Medicine & Biology, 55(21), 6411. Szczykutowicz, T. P., & Chen, G.-H. (2010). Dual energy CT using slow kVp switching acquisition and prior image constrained compressed sensing. Physics in Medicine & Biology, 55(21), 6411.
go back to reference Tian, C., Yong, X., Li, Z., Zuo, W., Fei, L., & Liu, H. (2020). Attention-guided CNN for image denoising. Neural Networks, 124, 117–129. Tian, C., Yong, X., Li, Z., Zuo, W., Fei, L., & Liu, H. (2020). Attention-guided CNN for image denoising. Neural Networks, 124, 117–129.
go back to reference Van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research,9(11). Van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research,9(11).
go back to reference Wang, H., Zhang, T., Yu, M., Sun, J., Ye, W., Wang, C., & Zhang, S. (2020). Stacking networks dynamically for image restoration based on the plug-and-play framework. In Proceedings of European conference on computer vision (ECCV), pp. 446–462. Wang, H., Zhang, T., Yu, M., Sun, J., Ye, W., Wang, C., & Zhang, S. (2020). Stacking networks dynamically for image restoration based on the plug-and-play framework. In Proceedings of European conference on computer vision (ECCV), pp. 446–462.
go back to reference Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612. Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.
go back to reference Wu, Z., Zhang, J., & Mou, C. (2021). Dense deep unfolding network with 3D-CNN prior for snapshot compressive imaging. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4892–4901. Wu, Z., Zhang, J., & Mou, C. (2021). Dense deep unfolding network with 3D-CNN prior for snapshot compressive imaging. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4892–4901.
go back to reference Xiang, J., Dong, Y., & Yang, Y. (2021). FISTA-Net: Learning a fast iterative shrinkage thresholding network for inverse problems in imaging. IEEE Transactions on Medical Imaging, 40(5), 1329–1339. Xiang, J., Dong, Y., & Yang, Y. (2021). FISTA-Net: Learning a fast iterative shrinkage thresholding network for inverse problems in imaging. IEEE Transactions on Medical Imaging, 40(5), 1329–1339.
go back to reference Yang, J., Wright, J., Huang, T., & Ma, Y. (2008). Image super-resolution as sparse representation of raw image patches. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8. Yang, J., Wright, J., Huang, T., & Ma, Y. (2008). Image super-resolution as sparse representation of raw image patches. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8.
go back to reference You, D., Xie, J., & Zhang, J. (2021a). ISTA-Net\(^{++}\): Flexible deep unfolding network for compressive sensing. In Proceedings of IEEE international conference on multimedia and expo (ICME), pp. 1–6. You, D., Xie, J., & Zhang, J. (2021a). ISTA-Net\(^{++}\): Flexible deep unfolding network for compressive sensing. In Proceedings of IEEE international conference on multimedia and expo (ICME), pp. 1–6.
go back to reference You, D., Zhang, J., Xie, J., Chen, B., & Ma, S. (2021b). COAST: Controllable arbitrary-sampling network for compressive sensing. IEEE Transactions on Image Processing, 30, 6066–6080.MathSciNet You, D., Zhang, J., Xie, J., Chen, B., & Ma, S. (2021b). COAST: Controllable arbitrary-sampling network for compressive sensing. IEEE Transactions on Image Processing, 30, 6066–6080.MathSciNet
go back to reference Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2021). Multi-stage progressive image restoration. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 14821–14831. Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2021). Multi-stage progressive image restoration. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 14821–14831.
go back to reference Zhang, J., & Ghanem, B. (2018). ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 1828–1837. Zhang, J., & Ghanem, B. (2018). ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 1828–1837.
go back to reference Zhang, J., Zhao, C., Zhao, D., & Gao, W. (2014a). Image compressive sensing recovery using adaptively learned sparsifying basis via L0 minimization. Signal Processing, 103, 114-126. Zhang, J., Zhao, C., Zhao, D., & Gao, W. (2014a). Image compressive sensing recovery using adaptively learned sparsifying basis via L0 minimization. Signal Processing, 103, 114-126.
go back to reference Zhang, J., Zhao, D., & Gao, W. (2014b). Group-based sparse representation for image restoration. IEEE Transactions on Image Processing, 23(8), 3336–3351.MathSciNetMATH Zhang, J., Zhao, D., & Gao, W. (2014b). Group-based sparse representation for image restoration. IEEE Transactions on Image Processing, 23(8), 3336–3351.MathSciNetMATH
go back to reference Zhang, J., Zhao, C., & Gao, W. (2020a). Optimization-inspired compact deep compressive sensing. IEEE Journal of Selected Topics in Signal Processing, 14(4), 765–774. Zhang, J., Zhao, C., & Gao, W. (2020a). Optimization-inspired compact deep compressive sensing. IEEE Journal of Selected Topics in Signal Processing, 14(4), 765–774.
go back to reference Zhang, J., Chen, B., Xiong, R., & Zhang, Y. (2023). Physics-inspired compressive sensing: Beyond deep unrolling. IEEE Signal Processing Magazine, 40(1), 58–72. Zhang, J., Chen, B., Xiong, R., & Zhang, Y. (2023). Physics-inspired compressive sensing: Beyond deep unrolling. IEEE Signal Processing Magazine, 40(1), 58–72.
go back to reference Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017a). Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7), 3142–3155.MathSciNetMATH Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017a). Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7), 3142–3155.MathSciNetMATH
go back to reference Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017b). Learning deep CNN denoiser prior for image restoration. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3929–3938. Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017b). Learning deep CNN denoiser prior for image restoration. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3929–3938.
go back to reference Zhang, K., Van Gool, L., & Timofte, R.(2020b). Deep unfolding network for image super-resolution. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3217–3226. Zhang, K., Van Gool, L., & Timofte, R.(2020b). Deep unfolding network for image super-resolution. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3217–3226.
go back to reference Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., & Timofte, R. (2021). Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence. Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., & Timofte, R. (2021). Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence.
go back to reference Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018). Image super-resolution using very deep residual channel attention networks. In Proceedings of European conference on computer vision (ECCV), pp. 286–301. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018). Image super-resolution using very deep residual channel attention networks. In Proceedings of European conference on computer vision (ECCV), pp. 286–301.
go back to reference Zhang, Z., Liu, Y., Liu, J., Wen, F., & Zhu, C. (2020c). AMP-Net: Denoising-based deep unfolding for compressive image sensing. IEEE Transactions on Image Processing, 30, 1487–1500.MathSciNet Zhang, Z., Liu, Y., Liu, J., Wen, F., & Zhu, C. (2020c). AMP-Net: Denoising-based deep unfolding for compressive image sensing. IEEE Transactions on Image Processing, 30, 1487–1500.MathSciNet
go back to reference Zhao, C., Ma, S., Zhang, J., Xiong, R., & Gao, W. (2016). Video compressive sensing reconstruction via reweighted residual sparsity. IEEE Transactions on Circuits and Systems for Video Technology, 27(6), 1182–1195. Zhao, C., Ma, S., Zhang, J., Xiong, R., & Gao, W. (2016). Video compressive sensing reconstruction via reweighted residual sparsity. IEEE Transactions on Circuits and Systems for Video Technology, 27(6), 1182–1195.
go back to reference Zhao, C., Zhang, J., Ma, S., Fan, X., Zhang, Y., & Gao, W. (2016). Reducing image compression artifacts by structural sparse representation and quantization constraint prior. IEEE Transactions on Circuits and Systems for Video Technology, 27(10), 2057–2071. Zhao, C., Zhang, J., Ma, S., Fan, X., Zhang, Y., & Gao, W. (2016). Reducing image compression artifacts by structural sparse representation and quantization constraint prior. IEEE Transactions on Circuits and Systems for Video Technology, 27(10), 2057–2071.
go back to reference Zheng, H., Yong, H., & Zhang, L. (2021). Deep convolutional dictionary learning for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 630–641. Zheng, H., Yong, H., & Zhang, L. (2021). Deep convolutional dictionary learning for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 630–641.
Metadata
Title
Deep Physics-Guided Unrolling Generalization for Compressed Sensing
Authors
Bin Chen
Jiechong Song
Jingfen Xie
Jian Zhang
Publication date
04-07-2023
Publisher
Springer US
Published in
International Journal of Computer Vision / Issue 11/2023
Print ISSN: 0920-5691
Electronic ISSN: 1573-1405
DOI
https://doi.org/10.1007/s11263-023-01814-w

Other articles of this Issue 11/2023

International Journal of Computer Vision 11/2023 Go to the issue

S.I. : Traditional Computer Vision in the Age of Deep Learning

Improving Domain Adaptation Through Class Aware Frequency Transformation

Premium Partner