Skip to main content
Erschienen in: International Journal of Computer Vision 11/2023

04.07.2023 | Manuscript

Deep Physics-Guided Unrolling Generalization for Compressed Sensing

verfasst von: Bin Chen, Jiechong Song, Jingfen Xie, Jian Zhang

Erschienen in: International Journal of Computer Vision | Ausgabe 11/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

By absorbing the merits of both the model- and data-driven methods, deep physics-engaged learning scheme achieves high-accuracy and interpretable image reconstruction. It has attracted growing attention and become the mainstream for inverse imaging tasks. Focusing on the image compressed sensing (CS) problem, we find the intrinsic defect of this emerging paradigm, widely implemented by deep algorithm-unrolled networks, in which more plain iterations involving real physics will bring enormous computation cost and long inference time, hindering their practical application. A novel deep Physics-guided unRolled recovery Learning (RL) framework is proposed by generalizing the traditional iterative recovery model from image domain (ID) to the high-dimensional feature domain (FD). A compact multiscale unrolling architecture is then developed to enhance the network capacity and keep real-time inference speeds. Taking two different perspectives of optimization and range-nullspace decomposition, instead of building an algorithm-specific unrolled network, we provide two implementations: PRL-PGD and PRL-RND. Experiments exhibit the significant performance and efficiency leading of PRL networks over other state-of-the-art methods with a large potential for further improvement and real application to other inverse imaging problems or optimization models.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Fußnoten
1
In the field of CS imaging, the condition of orthogonality \(\textbf{A}\textbf{A}^\top =\textbf{I}_M\) is widely implemented by orthogonalizing an i.i.d. random Gaussian matrix or a Hadamard matrix (Zhang et al., 2014b).
 
2
Note that this phenomenon indicates that our baseline network is “degraded” with large unrolled stage number instead of “overfitted” as the training loss and test PSNR both become worse. It has been previously discovered and studied in (He et al., 2016).
 
3
For reproducible research, the complete source code and pre-trained models of our PRL networks are available at https://​github.​com/​Guaishou74851/​PRL.
 
Literatur
Zurück zum Zitat Adler, J., & Öktem, O. (2018). Learned primal-dual reconstruction. IEEE Transactions on Medical Imaging, 37(6), 1322–1332. Adler, J., & Öktem, O. (2018). Learned primal-dual reconstruction. IEEE Transactions on Medical Imaging, 37(6), 1322–1332.
Zurück zum Zitat Agustsson, E., & Timofte, R. (2017). NTIRE 2017 challenge on single image super-resolution: dataset and study. In Proceedings of IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp. 126–135. Agustsson, E., & Timofte, R. (2017). NTIRE 2017 challenge on single image super-resolution: dataset and study. In Proceedings of IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp. 126–135.
Zurück zum Zitat Björck, Å., Elfving, T., & Strakos, Z. (1998). Stability of Conjugate Gradient and lanczos methods for linear least squares problems. SIAM Journal on Matrix Analysis and Applications, 19(3), 720–736.MathSciNetMATH Björck, Å., Elfving, T., & Strakos, Z. (1998). Stability of Conjugate Gradient and lanczos methods for linear least squares problems. SIAM Journal on Matrix Analysis and Applications, 19(3), 720–736.MathSciNetMATH
Zurück zum Zitat Blumensath, T., & Davies, M. E. (2009). Iterative hard thresholding for compressed sensing. Applied and Computational Harmonic Analysis, 27(3), 265–274.MathSciNetMATH Blumensath, T., & Davies, M. E. (2009). Iterative hard thresholding for compressed sensing. Applied and Computational Harmonic Analysis, 27(3), 265–274.MathSciNetMATH
Zurück zum Zitat Boufounos, Petros. T., & Baraniuk, Richard. G. (2008). 1-bit Compressive sensing. In Proceedings of IEEE conference on information sciences and systems (CISS), pp. 16–21. Boufounos, Petros. T., & Baraniuk, Richard. G. (2008). 1-bit Compressive sensing. In Proceedings of IEEE conference on information sciences and systems (CISS), pp. 16–21.
Zurück zum Zitat Cai, J.-F., Candès, E. J., & Shen, Z. (2010). A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20(4), 1956–1982.MathSciNetMATH Cai, J.-F., Candès, E. J., & Shen, Z. (2010). A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20(4), 1956–1982.MathSciNetMATH
Zurück zum Zitat Candès, E. J., & Wakin, M. B. (2008). An introduction to compressive sampling. IEEE Signal Processing Magazine, 25(2), 21–30. Candès, E. J., & Wakin, M. B. (2008). An introduction to compressive sampling. IEEE Signal Processing Magazine, 25(2), 21–30.
Zurück zum Zitat Chen, B., & Zhang, J. (2022). Content-aware scalable deep compressed sensing. IEEE Transactions on Image Processing, 31, 5412–5426. Chen, B., & Zhang, J. (2022). Content-aware scalable deep compressed sensing. IEEE Transactions on Image Processing, 31, 5412–5426.
Zurück zum Zitat Chen, D., & Davies, Mike. E. (2020). Deep decomposition learning for inverse imaging problems. In Proceedings of European conference on computer vision (ECCV), pp. 510–526. Chen, D., & Davies, Mike. E. (2020). Deep decomposition learning for inverse imaging problems. In Proceedings of European conference on computer vision (ECCV), pp. 510–526.
Zurück zum Zitat Chen, D., Tachella, J., & Davies, M. E. (2021a). Equivariant imaging: learning beyond the range space. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4379–4388. Chen, D., Tachella, J., & Davies, M. E. (2021a). Equivariant imaging: learning beyond the range space. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4379–4388.
Zurück zum Zitat Chen, H., Zhang, Y., Zhang, W., Liao, P., Li, K., Zhou, J., & Wang, G. (2017). Low-dose CT via convolutional neural network. Biomedical Optics Express, 8(2), 679–694. Chen, H., Zhang, Y., Zhang, W., Liao, P., Li, K., Zhou, J., & Wang, G. (2017). Low-dose CT via convolutional neural network. Biomedical Optics Express, 8(2), 679–694.
Zurück zum Zitat Chen, J., Sun, Y., Liu, Q., & Huang, R. (2020). Learning memory augmented cascading network for compressed sensing of images. In Proceedings of European conference on computer vision (ECCV), pp. 513–529. Chen, J., Sun, Y., Liu, Q., & Huang, R. (2020). Learning memory augmented cascading network for compressed sensing of images. In Proceedings of European conference on computer vision (ECCV), pp. 513–529.
Zurück zum Zitat Chen, Y., & Pock, T. (2016). Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1256–1272. Chen, Y., & Pock, T. (2016). Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1256–1272.
Zurück zum Zitat Chen, Z., Guo, W., Feng, Y., Li, Y., Zhao, C., Ren, Y., & Shao, L. (2021). Deep-learned regularization and proximal operator for image compressive sensing. IEEE Transactions on Image Processing, 30, 7112–7126.MathSciNet Chen, Z., Guo, W., Feng, Y., Li, Y., Zhao, C., Ren, Y., & Shao, L. (2021). Deep-learned regularization and proximal operator for image compressive sensing. IEEE Transactions on Image Processing, 30, 7112–7126.MathSciNet
Zurück zum Zitat Coban, S., Andriiashen, V., & Ganguly, P. (2020). Apple CT Data: simulated parallel-beam tomographic datasets. Zenodo. Coban, S., Andriiashen, V., & Ganguly, P. (2020). Apple CT Data: simulated parallel-beam tomographic datasets. Zenodo.
Zurück zum Zitat Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080–2095.MathSciNet Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080–2095.MathSciNet
Zurück zum Zitat Denker, A., Schmidt, M., Leuschner, J., Maass, P., & Behrmann, J. (2020). Conditional normalizing flows for low-dose computed tomography image reconstruction. arXiv preprint arXiv:2006.06270. Denker, A., Schmidt, M., Leuschner, J., Maass, P., & Behrmann, J. (2020). Conditional normalizing flows for low-dose computed tomography image reconstruction. arXiv preprint arXiv:​2006.​06270.
Zurück zum Zitat Dong, C., Loy, C. C., He, K., & Tang, X. (2014a). Learning a deep convolutional network for image super-resolution. In Proceedings of European conference on computer vision (ECCV), pp. 184–199. Dong, C., Loy, C. C., He, K., & Tang, X. (2014a). Learning a deep convolutional network for image super-resolution. In Proceedings of European conference on computer vision (ECCV), pp. 184–199.
Zurück zum Zitat Dong, W., Shi, G., Li, X., Ma, Y., & Huang, F. (2014b). Compressive sensing via nonlocal low-rank regularization. IEEE Transactions on Image Processing, 23(8), 3618–3632.MathSciNetMATH Dong, W., Shi, G., Li, X., Ma, Y., & Huang, F. (2014b). Compressive sensing via nonlocal low-rank regularization. IEEE Transactions on Image Processing, 23(8), 3618–3632.MathSciNetMATH
Zurück zum Zitat Donoho, D. L. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52(4), 1289–1306.MathSciNetMATH Donoho, D. L. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52(4), 1289–1306.MathSciNetMATH
Zurück zum Zitat Elad, M. (2010). Sparse and redundant representations: from theory to applications in signal and image processing (Vol. 2). Springer.MATH Elad, M. (2010). Sparse and redundant representations: from theory to applications in signal and image processing (Vol. 2). Springer.MATH
Zurück zum Zitat Fan, Z.-E., Lian, F., & Quan, J.-N. (2022). Global sensing and measurements reuse for image compressed sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 8954–8963. Fan, Z.-E., Lian, F., & Quan, J.-N. (2022). Global sensing and measurements reuse for image compressed sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 8954–8963.
Zurück zum Zitat Fowler, J. E., Mun, S., Tramel, E. W., et al. (2012). Block-based compressed sensing of images and video. Foundations and Trends in Signal Processing, 4(4), 297–416.MATH Fowler, J. E., Mun, S., Tramel, E. W., et al. (2012). Block-based compressed sensing of images and video. Foundations and Trends in Signal Processing, 4(4), 297–416.MATH
Zurück zum Zitat Gan, L. (2007). Block compressed sensing of natural images. In Proceedings of IEEE international conference on digital signal processing (ICDSP), pp. 403–406. IEEE. Gan, L. (2007). Block compressed sensing of natural images. In Proceedings of IEEE international conference on digital signal processing (ICDSP), pp. 403–406. IEEE.
Zurück zum Zitat Gilton, D., Ongie, G., & Willett, R. (2019). Neumann networks for linear inverse problems in imaging. IEEE Transactions on Computational Imaging, 6, 328–343.MathSciNet Gilton, D., Ongie, G., & Willett, R. (2019). Neumann networks for linear inverse problems in imaging. IEEE Transactions on Computational Imaging, 6, 328–343.MathSciNet
Zurück zum Zitat Gu, J., & Dong, C. (2021). Interpreting super-resolution networks with local attribution maps. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 9199–9208. Gu, J., & Dong, C. (2021). Interpreting super-resolution networks with local attribution maps. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 9199–9208.
Zurück zum Zitat He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 770–778. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 770–778.
Zurück zum Zitat Huang, J. -B., Singh, A., & Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 5197–5206. Huang, J. -B., Singh, A., & Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 5197–5206.
Zurück zum Zitat Huang, Y., Würfl, T., Breininger, K., Liu, L., Lauritsch, G., & Maier, A. (2018). Some investigations on robustness of deep learning in limited angle tomography. In Proceedings of international conference on medical image computing and computer-assisted intervention (MICCAI), pp. 145–153. Huang, Y., Würfl, T., Breininger, K., Liu, L., Lauritsch, G., & Maier, A. (2018). Some investigations on robustness of deep learning in limited angle tomography. In Proceedings of international conference on medical image computing and computer-assisted intervention (MICCAI), pp. 145–153.
Zurück zum Zitat Jacques, L., Laska, J. N., Boufounos, P. T., & Baraniuk, R. G. (2013). Robust 1-bit compressive sensing via binary stable embeddings of sparse vectors. IEEE Transactions on Information Theory, 59(4), 2082–2102.MathSciNetMATH Jacques, L., Laska, J. N., Boufounos, P. T., & Baraniuk, R. G. (2013). Robust 1-bit compressive sensing via binary stable embeddings of sparse vectors. IEEE Transactions on Information Theory, 59(4), 2082–2102.MathSciNetMATH
Zurück zum Zitat Kafle, S., Joseph, G., & Varshney, P. K. (2021). One-bit compressed sensing using untrained network prior. In Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 2875–2879. Kafle, S., Joseph, G., & Varshney, P. K. (2021). One-bit compressed sensing using untrained network prior. In Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 2875–2879.
Zurück zum Zitat Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In Proceedings of international conference on learning representations (ICLR), pp. 1–15. Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In Proceedings of international conference on learning representations (ICLR), pp. 1–15.
Zurück zum Zitat Kokkinos, F., & Lefkimmiatis, S. (2018). Deep image demosaicking using a cascade of convolutional residual denoising networks. In Proceedings of European conference on computer vision (ECCV), pp. 303–319. Kokkinos, F., & Lefkimmiatis, S. (2018). Deep image demosaicking using a cascade of convolutional residual denoising networks. In Proceedings of European conference on computer vision (ECCV), pp. 303–319.
Zurück zum Zitat Kruse, J., Rother, C., & Schmidt, U. (2017). Learning to push the limits of efficient FFT-based image deconvolution. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4586–4594. Kruse, J., Rother, C., & Schmidt, U. (2017). Learning to push the limits of efficient FFT-based image deconvolution. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4586–4594.
Zurück zum Zitat Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., & Ashok, A. (2016). ReconNet: Non-iterative reconstruction of images from compressively sensed measurements. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 449–458. Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., & Ashok, A. (2016). ReconNet: Non-iterative reconstruction of images from compressively sensed measurements. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 449–458.
Zurück zum Zitat Lefkimmiatis, S. (2017). Non-local color image denoising with convolutional neural networks. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3587–3596. Lefkimmiatis, S. (2017). Non-local color image denoising with convolutional neural networks. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3587–3596.
Zurück zum Zitat Lefkimmiatis, S. (2018). Universal denoising networks: A novel CNN architecture for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3204–3213. Lefkimmiatis, S. (2018). Universal denoising networks: A novel CNN architecture for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3204–3213.
Zurück zum Zitat Leuschner, J., Schmidt, M., Ganguly, P. S., Andriiashen, V., Coban, S. B., Denker, A., Bauer, D., Hadjifaradji, A., Batenburg, K. J., Maass, P., et al. (2021). Quantitative comparison of deep learning-based image reconstruction methods for low-dose and sparse-angle CT applications. Journal of Imaging, 7(3), 44. Leuschner, J., Schmidt, M., Ganguly, P. S., Andriiashen, V., Coban, S. B., Denker, A., Bauer, D., Hadjifaradji, A., Batenburg, K. J., Maass, P., et al. (2021). Quantitative comparison of deep learning-based image reconstruction methods for low-dose and sparse-angle CT applications. Journal of Imaging, 7(3), 44.
Zurück zum Zitat Li, Y., Li, K., Zhang, C., Montoya, J., & Chen, G.-H. (2019). Learning to reconstruct computed tomography images directly from sinogram data under a variety of data acquisition conditions. IEEE Transactions on Medical Imaging, 38(10), 2469–2481. Li, Y., Li, K., Zhang, C., Montoya, J., & Chen, G.-H. (2019). Learning to reconstruct computed tomography images directly from sinogram data under a variety of data acquisition conditions. IEEE Transactions on Medical Imaging, 38(10), 2469–2481.
Zurück zum Zitat Liu, T., Chaman, A., Belius, D., & Dokmanic, I. (2020). Interpreting U-nets via task-driven multiscale dictionary learning. arXiv preprint arXiv:2011.12815. Liu, T., Chaman, A., Belius, D., & Dokmanic, I. (2020). Interpreting U-nets via task-driven multiscale dictionary learning. arXiv preprint arXiv:​2011.​12815.
Zurück zum Zitat Liu, Y., Long, Z., & Zhu, C. (2018). Image completion using low tensor tree rank and total variation minimization. IEEE Transactions on Multimedia, 21(2), 338–350. Liu, Y., Long, Z., & Zhu, C. (2018). Image completion using low tensor tree rank and total variation minimization. IEEE Transactions on Multimedia, 21(2), 338–350.
Zurück zum Zitat Liu, Y., Long, Z., Huang, H., & Zhu, C. (2019). Low CP rank and tucker rank tensor completion for estimating missing components in image data. IEEE Transactions on Circuits and Systems for Video Technology, 30(4), 944–954. Liu, Y., Long, Z., Huang, H., & Zhu, C. (2019). Low CP rank and tucker rank tensor completion for estimating missing components in image data. IEEE Transactions on Circuits and Systems for Video Technology, 30(4), 944–954.
Zurück zum Zitat Long, Z., Liu, Y., Chen, L., & Zhu, C. (2019). Low rank tensor completion for multiway visual data. Signal Processing, 155, 301–316. Long, Z., Liu, Y., Chen, L., & Zhu, C. (2019). Low rank tensor completion for multiway visual data. Signal Processing, 155, 301–316.
Zurück zum Zitat Long, Z., Zhu, C., Liu, J., & Liu, Y. (2021). Bayesian low rank tensor ring for image recovery. IEEE Transactions on Image Processing, 30, 3568–3580.MathSciNet Long, Z., Zhu, C., Liu, J., & Liu, Y. (2021). Bayesian low rank tensor ring for image recovery. IEEE Transactions on Image Processing, 30, 3568–3580.MathSciNet
Zurück zum Zitat Long, Z., Zhu, C., Liu, J., Comon, P., & Liu, Y. (2022). Trainable subspaces for low rank tensor completion: Model and analysis. IEEE Transactions on Signal Processing, 70, 2502–2517.MathSciNet Long, Z., Zhu, C., Liu, J., Comon, P., & Liu, Y. (2022). Trainable subspaces for low rank tensor completion: Model and analysis. IEEE Transactions on Signal Processing, 70, 2502–2517.MathSciNet
Zurück zum Zitat Lustig, M., Donoho, D., & Pauly, J. M. (2007). Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine, 58(6), 1182–1195. Lustig, M., Donoho, D., & Pauly, J. M. (2007). Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine, 58(6), 1182–1195.
Zurück zum Zitat Lustig, M., Donoho, D. L., Santos, J. M., & Pauly, J. M. (2008). Compressed sensing MRI. IEEE Signal Processing Magazine, 25(2), 72–82. Lustig, M., Donoho, D. L., Santos, J. M., & Pauly, J. M. (2008). Compressed sensing MRI. IEEE Signal Processing Magazine, 25(2), 72–82.
Zurück zum Zitat Ma, K., Duanmu, Z., Qingbo, W., Wang, Z., Yong, H., Li, H., & Zhang, L. (2016). Waterloo exploration database: New challenges for image quality assessment models. IEEE Transactions on Image Processing, 26(2), 1004–1016.MathSciNetMATH Ma, K., Duanmu, Z., Qingbo, W., Wang, Z., Yong, H., Li, H., & Zhang, L. (2016). Waterloo exploration database: New challenges for image quality assessment models. IEEE Transactions on Image Processing, 26(2), 1004–1016.MathSciNetMATH
Zurück zum Zitat Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of IEEE international conference on computer vision (ICCV), 2, pp. 416–423. Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of IEEE international conference on computer vision (ICCV), 2, pp. 416–423.
Zurück zum Zitat Mousavi, A., & Baraniuk, R. G. (2017). Learning to invert: Signal recovery via deep convolutional networks. In Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 2272–2276. Mousavi, A., & Baraniuk, R. G. (2017). Learning to invert: Signal recovery via deep convolutional networks. In Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 2272–2276.
Zurück zum Zitat Mousavi, A., Patel, A. B., & Baraniuk, R. G. (2015). A deep learning approach to structured signal recovery. In Proceedings of IEEE allerton conference on communication, control, and computing, pp. 1336–1343. Mousavi, A., Patel, A. B., & Baraniuk, R. G. (2015). A deep learning approach to structured signal recovery. In Proceedings of IEEE allerton conference on communication, control, and computing, pp. 1336–1343.
Zurück zum Zitat Mun, S., & Fowler, J. E. (2009). Block compressed sensing of images using directional transforms. In Proceedings of IEEE international conference on image processing (ICIP), pp. 3021–3024. Mun, S., & Fowler, J. E. (2009). Block compressed sensing of images using directional transforms. In Proceedings of IEEE international conference on image processing (ICIP), pp. 3021–3024.
Zurück zum Zitat Nam, S., Davies, M. E., Elad, M., & Gribonval, R. (2013). The cosparse analysis model and algorithms. Applied and Computational Harmonic Analysis, 34(1), 30–56.MathSciNetMATH Nam, S., Davies, M. E., Elad, M., & Gribonval, R. (2013). The cosparse analysis model and algorithms. Applied and Computational Harmonic Analysis, 34(1), 30–56.MathSciNetMATH
Zurück zum Zitat Niu, S., Gao, Y., Bian, Z., Huang, J., Chen, W., Gaohang, Yu., Liang, Z., & Ma, J. (2014). Sparse-View X-Ray CT reconstruction via total generalized variation regularization. Physics in Medicine & Biology, 59(12), 2997. Niu, S., Gao, Y., Bian, Z., Huang, J., Chen, W., Gaohang, Yu., Liang, Z., & Ma, J. (2014). Sparse-View X-Ray CT reconstruction via total generalized variation regularization. Physics in Medicine & Biology, 59(12), 2997.
Zurück zum Zitat Parikh, N., Boyd, S., et al. (2014). Proximal algorithms. Foundations and in Optimization, 1(3), 127–239. Parikh, N., Boyd, S., et al. (2014). Proximal algorithms. Foundations and in Optimization, 1(3), 127–239.
Zurück zum Zitat Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., & Antiga, L. et al. (2019). PyTorch: An imperative style, high-performance deep learning library. Proceedings of Neural Information Processing Systems (NeurIPS), 32. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., & Antiga, L. et al. (2019). PyTorch: An imperative style, high-performance deep learning library. Proceedings of Neural Information Processing Systems (NeurIPS), 32.
Zurück zum Zitat Pelt, D. M., Batenburg, K. J., & Sethian, J. A. (2018). Improving tomographic reconstruction from limited data using mixed-scale dense convolutional neural networks. Journal of Imaging, 4(11), 128. Pelt, D. M., Batenburg, K. J., & Sethian, J. A. (2018). Improving tomographic reconstruction from limited data using mixed-scale dense convolutional neural networks. Journal of Imaging, 4(11), 128.
Zurück zum Zitat Radon, J. (1986). On the determination of functions from their integral values along certain manifolds. IEEE Transactions on Medical Imaging, 5(4), 170–176. Radon, J. (1986). On the determination of functions from their integral values along certain manifolds. IEEE Transactions on Medical Imaging, 5(4), 170–176.
Zurück zum Zitat Ravishankar, S., Ye, J. C., & Fessler, J. A. (2019). Image reconstruction: From sparsity to data-adaptive methods and machine learning. Proceedings of the IEEE, 108(1), 86–109. Ravishankar, S., Ye, J. C., & Fessler, J. A. (2019). Image reconstruction: From sparsity to data-adaptive methods and machine learning. Proceedings of the IEEE, 108(1), 86–109.
Zurück zum Zitat Ren, C., He, X., Wang, C., & Zhao, Zhibo. (2021). Adaptive consistency prior based deep network for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 8596–8606. Ren, C., He, X., Wang, C., & Zhao, Zhibo. (2021). Adaptive consistency prior based deep network for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 8596–8606.
Zurück zum Zitat Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of international conference on medical image computing and computer-assisted intervention (MICCAI), pp. 234–241. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of international conference on medical image computing and computer-assisted intervention (MICCAI), pp. 234–241.
Zurück zum Zitat Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 1874–1883. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 1874–1883.
Zurück zum Zitat Shi, W., Jiang, F., Liu, S., & Zhao, D. (2019a). Image compressed sensing using convolutional neural network. IEEE Transactions on Image Processing, 29, 375–388.MathSciNetMATH Shi, W., Jiang, F., Liu, S., & Zhao, D. (2019a). Image compressed sensing using convolutional neural network. IEEE Transactions on Image Processing, 29, 375–388.MathSciNetMATH
Zurück zum Zitat Shi, W., Jiang, F., Liu, S., & Zhao, D. (2019b). Scalable convolutional neural network for image compressed sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 12290–12299. Shi, W., Jiang, F., Liu, S., & Zhao, D. (2019b). Scalable convolutional neural network for image compressed sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 12290–12299.
Zurück zum Zitat Song, J., Chen, B., & Zhang, J. (2021). Memory-augmented deep unfolding network for compressive sensing. In Proceedings of ACM international conference on multimedia (ACM MM), pp. 4249–4258. Song, J., Chen, B., & Zhang, J. (2021). Memory-augmented deep unfolding network for compressive sensing. In Proceedings of ACM international conference on multimedia (ACM MM), pp. 4249–4258.
Zurück zum Zitat Song, J., Chen, B., & Zhang, J. (2023a). Deep memory-augmented proximal unrolling network for compressive sensing. International Journal of Computer Vision, 1–20. Song, J., Chen, B., & Zhang, J. (2023a). Deep memory-augmented proximal unrolling network for compressive sensing. International Journal of Computer Vision, 1–20.
Zurück zum Zitat Song, J., Chen, B., & Zhang, J. (2023b). Dynamic path-controllable deep unfolding network for compressive sensing. IEEE Transactions on Image Processing, 32, 2202–2214. Song, J., Chen, B., & Zhang, J. (2023b). Dynamic path-controllable deep unfolding network for compressive sensing. IEEE Transactions on Image Processing, 32, 2202–2214.
Zurück zum Zitat Sun, J., Li, H., Zongben, X., et al. (2016). Deep ADMM-Net for compressive sensing MRI. Proceedings of Neural Information Processing Systems (NeurIPS), 29, 10–18. Sun, J., Li, H., Zongben, X., et al. (2016). Deep ADMM-Net for compressive sensing MRI. Proceedings of Neural Information Processing Systems (NeurIPS), 29, 10–18.
Zurück zum Zitat Sun, Y., Chen, J., Liu, Q., Liu, B., & Guo, G. (2020). Dual-path attention network for compressed sensing image reconstruction. IEEE Transactions on Image Processing, 29, 9482–9495.MATH Sun, Y., Chen, J., Liu, Q., Liu, B., & Guo, G. (2020). Dual-path attention network for compressed sensing image reconstruction. IEEE Transactions on Image Processing, 29, 9482–9495.MATH
Zurück zum Zitat Szczykutowicz, T. P., & Chen, G.-H. (2010). Dual energy CT using slow kVp switching acquisition and prior image constrained compressed sensing. Physics in Medicine & Biology, 55(21), 6411. Szczykutowicz, T. P., & Chen, G.-H. (2010). Dual energy CT using slow kVp switching acquisition and prior image constrained compressed sensing. Physics in Medicine & Biology, 55(21), 6411.
Zurück zum Zitat Tian, C., Yong, X., Li, Z., Zuo, W., Fei, L., & Liu, H. (2020). Attention-guided CNN for image denoising. Neural Networks, 124, 117–129. Tian, C., Yong, X., Li, Z., Zuo, W., Fei, L., & Liu, H. (2020). Attention-guided CNN for image denoising. Neural Networks, 124, 117–129.
Zurück zum Zitat Van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research,9(11). Van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research,9(11).
Zurück zum Zitat Wang, H., Zhang, T., Yu, M., Sun, J., Ye, W., Wang, C., & Zhang, S. (2020). Stacking networks dynamically for image restoration based on the plug-and-play framework. In Proceedings of European conference on computer vision (ECCV), pp. 446–462. Wang, H., Zhang, T., Yu, M., Sun, J., Ye, W., Wang, C., & Zhang, S. (2020). Stacking networks dynamically for image restoration based on the plug-and-play framework. In Proceedings of European conference on computer vision (ECCV), pp. 446–462.
Zurück zum Zitat Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612. Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.
Zurück zum Zitat Wu, Z., Zhang, J., & Mou, C. (2021). Dense deep unfolding network with 3D-CNN prior for snapshot compressive imaging. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4892–4901. Wu, Z., Zhang, J., & Mou, C. (2021). Dense deep unfolding network with 3D-CNN prior for snapshot compressive imaging. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4892–4901.
Zurück zum Zitat Xiang, J., Dong, Y., & Yang, Y. (2021). FISTA-Net: Learning a fast iterative shrinkage thresholding network for inverse problems in imaging. IEEE Transactions on Medical Imaging, 40(5), 1329–1339. Xiang, J., Dong, Y., & Yang, Y. (2021). FISTA-Net: Learning a fast iterative shrinkage thresholding network for inverse problems in imaging. IEEE Transactions on Medical Imaging, 40(5), 1329–1339.
Zurück zum Zitat Yang, J., Wright, J., Huang, T., & Ma, Y. (2008). Image super-resolution as sparse representation of raw image patches. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8. Yang, J., Wright, J., Huang, T., & Ma, Y. (2008). Image super-resolution as sparse representation of raw image patches. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8.
Zurück zum Zitat You, D., Xie, J., & Zhang, J. (2021a). ISTA-Net\(^{++}\): Flexible deep unfolding network for compressive sensing. In Proceedings of IEEE international conference on multimedia and expo (ICME), pp. 1–6. You, D., Xie, J., & Zhang, J. (2021a). ISTA-Net\(^{++}\): Flexible deep unfolding network for compressive sensing. In Proceedings of IEEE international conference on multimedia and expo (ICME), pp. 1–6.
Zurück zum Zitat You, D., Zhang, J., Xie, J., Chen, B., & Ma, S. (2021b). COAST: Controllable arbitrary-sampling network for compressive sensing. IEEE Transactions on Image Processing, 30, 6066–6080.MathSciNet You, D., Zhang, J., Xie, J., Chen, B., & Ma, S. (2021b). COAST: Controllable arbitrary-sampling network for compressive sensing. IEEE Transactions on Image Processing, 30, 6066–6080.MathSciNet
Zurück zum Zitat Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2021). Multi-stage progressive image restoration. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 14821–14831. Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2021). Multi-stage progressive image restoration. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 14821–14831.
Zurück zum Zitat Zhang, J., & Ghanem, B. (2018). ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 1828–1837. Zhang, J., & Ghanem, B. (2018). ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 1828–1837.
Zurück zum Zitat Zhang, J., Zhao, C., Zhao, D., & Gao, W. (2014a). Image compressive sensing recovery using adaptively learned sparsifying basis via L0 minimization. Signal Processing, 103, 114-126. Zhang, J., Zhao, C., Zhao, D., & Gao, W. (2014a). Image compressive sensing recovery using adaptively learned sparsifying basis via L0 minimization. Signal Processing, 103, 114-126.
Zurück zum Zitat Zhang, J., Zhao, D., & Gao, W. (2014b). Group-based sparse representation for image restoration. IEEE Transactions on Image Processing, 23(8), 3336–3351.MathSciNetMATH Zhang, J., Zhao, D., & Gao, W. (2014b). Group-based sparse representation for image restoration. IEEE Transactions on Image Processing, 23(8), 3336–3351.MathSciNetMATH
Zurück zum Zitat Zhang, J., Zhao, C., & Gao, W. (2020a). Optimization-inspired compact deep compressive sensing. IEEE Journal of Selected Topics in Signal Processing, 14(4), 765–774. Zhang, J., Zhao, C., & Gao, W. (2020a). Optimization-inspired compact deep compressive sensing. IEEE Journal of Selected Topics in Signal Processing, 14(4), 765–774.
Zurück zum Zitat Zhang, J., Chen, B., Xiong, R., & Zhang, Y. (2023). Physics-inspired compressive sensing: Beyond deep unrolling. IEEE Signal Processing Magazine, 40(1), 58–72. Zhang, J., Chen, B., Xiong, R., & Zhang, Y. (2023). Physics-inspired compressive sensing: Beyond deep unrolling. IEEE Signal Processing Magazine, 40(1), 58–72.
Zurück zum Zitat Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017a). Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7), 3142–3155.MathSciNetMATH Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017a). Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7), 3142–3155.MathSciNetMATH
Zurück zum Zitat Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017b). Learning deep CNN denoiser prior for image restoration. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3929–3938. Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017b). Learning deep CNN denoiser prior for image restoration. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3929–3938.
Zurück zum Zitat Zhang, K., Van Gool, L., & Timofte, R.(2020b). Deep unfolding network for image super-resolution. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3217–3226. Zhang, K., Van Gool, L., & Timofte, R.(2020b). Deep unfolding network for image super-resolution. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3217–3226.
Zurück zum Zitat Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., & Timofte, R. (2021). Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence. Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., & Timofte, R. (2021). Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Zurück zum Zitat Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018). Image super-resolution using very deep residual channel attention networks. In Proceedings of European conference on computer vision (ECCV), pp. 286–301. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018). Image super-resolution using very deep residual channel attention networks. In Proceedings of European conference on computer vision (ECCV), pp. 286–301.
Zurück zum Zitat Zhang, Z., Liu, Y., Liu, J., Wen, F., & Zhu, C. (2020c). AMP-Net: Denoising-based deep unfolding for compressive image sensing. IEEE Transactions on Image Processing, 30, 1487–1500.MathSciNet Zhang, Z., Liu, Y., Liu, J., Wen, F., & Zhu, C. (2020c). AMP-Net: Denoising-based deep unfolding for compressive image sensing. IEEE Transactions on Image Processing, 30, 1487–1500.MathSciNet
Zurück zum Zitat Zhao, C., Ma, S., Zhang, J., Xiong, R., & Gao, W. (2016). Video compressive sensing reconstruction via reweighted residual sparsity. IEEE Transactions on Circuits and Systems for Video Technology, 27(6), 1182–1195. Zhao, C., Ma, S., Zhang, J., Xiong, R., & Gao, W. (2016). Video compressive sensing reconstruction via reweighted residual sparsity. IEEE Transactions on Circuits and Systems for Video Technology, 27(6), 1182–1195.
Zurück zum Zitat Zhao, C., Zhang, J., Ma, S., Fan, X., Zhang, Y., & Gao, W. (2016). Reducing image compression artifacts by structural sparse representation and quantization constraint prior. IEEE Transactions on Circuits and Systems for Video Technology, 27(10), 2057–2071. Zhao, C., Zhang, J., Ma, S., Fan, X., Zhang, Y., & Gao, W. (2016). Reducing image compression artifacts by structural sparse representation and quantization constraint prior. IEEE Transactions on Circuits and Systems for Video Technology, 27(10), 2057–2071.
Zurück zum Zitat Zheng, H., Yong, H., & Zhang, L. (2021). Deep convolutional dictionary learning for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 630–641. Zheng, H., Yong, H., & Zhang, L. (2021). Deep convolutional dictionary learning for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 630–641.
Metadaten
Titel
Deep Physics-Guided Unrolling Generalization for Compressed Sensing
verfasst von
Bin Chen
Jiechong Song
Jingfen Xie
Jian Zhang
Publikationsdatum
04.07.2023
Verlag
Springer US
Erschienen in
International Journal of Computer Vision / Ausgabe 11/2023
Print ISSN: 0920-5691
Elektronische ISSN: 1573-1405
DOI
https://doi.org/10.1007/s11263-023-01814-w

Weitere Artikel der Ausgabe 11/2023

International Journal of Computer Vision 11/2023 Zur Ausgabe

S.I. : Traditional Computer Vision in the Age of Deep Learning

Improving Domain Adaptation Through Class Aware Frequency Transformation

Premium Partner