Skip to main content

2021 | OriginalPaper | Buchkapitel

Image Registration Improved by Generative Adversarial Networks

verfasst von : Shiyan Jiang, Ci Wang, Chang Huang

Erschienen in: MultiMedia Modeling

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The performances of most image registrations will decrease if the quality of the image to be registered is poor, especially contaminated with heavy distortions such as noise, blur, and uneven degradation. To solve this problem, a generative adversarial networks (GANs) based approach and the specified loss functions are proposed to improve image quality for better registration. Specifically, given the paired images, the generator network enhances the distorted image and the discriminator network compares the enhanced image with the ideal image. To efficiently discriminate the enhanced image, the loss function is designed to describe the perceptual loss and the adversarial loss, where the former measures the image similarity and the latter pushes the enhanced solution to natural image manifold. After enhancement, image features are more accurate and the registrations between feature point pairs will be more consistent.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Zitováand, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21, 977–1000 (2003)CrossRef Zitováand, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21, 977–1000 (2003)CrossRef
2.
Zurück zum Zitat Barnea, D.I., Silverman, H.F.: A class of algorithms for fast digital image registration. IEEE Trans. Comput. 21, 179–186 (1972)CrossRef Barnea, D.I., Silverman, H.F.: A class of algorithms for fast digital image registration. IEEE Trans. Comput. 21, 179–186 (1972)CrossRef
3.
Zurück zum Zitat Guo, X., Xu, Z., Lu, Y., Pang, Y.: An application of Fourier–Mellin transform in image registration. In: Proceedings of the 5th International Conference on Computer and Information Technology (CIT), Shanghai, China, September 2005, pp. 619–623 (2005) Guo, X., Xu, Z., Lu, Y., Pang, Y.: An application of Fourier–Mellin transform in image registration. In: Proceedings of the 5th International Conference on Computer and Information Technology (CIT), Shanghai, China, September 2005, pp. 619–623 (2005)
4.
Zurück zum Zitat Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)CrossRef Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)CrossRef
6.
Zurück zum Zitat Dosovitskiy, A., Fischer, P., Springenberg, J.T., Riedmiller, M., Brox, T.: Discriminative unsupervised feature learning with exemplar convolutional neural networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(9), 1734–1747 (2016)CrossRef Dosovitskiy, A., Fischer, P., Springenberg, J.T., Riedmiller, M., Brox, T.: Discriminative unsupervised feature learning with exemplar convolutional neural networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(9), 1734–1747 (2016)CrossRef
7.
Zurück zum Zitat Yang, Z., Dan, T., Yang, Y.: Multi-temporal remote sensing image registration using deep convolutional features. IEEE Access 6, 38544–38555 (2018)CrossRef Yang, Z., Dan, T., Yang, Y.: Multi-temporal remote sensing image registration using deep convolutional features. IEEE Access 6, 38544–38555 (2018)CrossRef
8.
Zurück zum Zitat Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014) Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:​1409.​1556 (2014)
10.
Zurück zum Zitat Liao, R., et al.: An artificial agent for robust image registration. In: AAAI, pp. 4168–4175 (2017) Liao, R., et al.: An artificial agent for robust image registration. In: AAAI, pp. 4168–4175 (2017)
11.
Zurück zum Zitat Nguyen, T., Chen, S.W., Shivakumar, S.S., Taylor, C.J., Kumar, V.: Unsupervised deep homography: a fast and robust homography estimation model. arXiv preprint arXiv:1709.03966 (2017) Nguyen, T., Chen, S.W., Shivakumar, S.S., Taylor, C.J., Kumar, V.: Unsupervised deep homography: a fast and robust homography estimation model. arXiv preprint arXiv:​1709.​03966 (2017)
12.
Zurück zum Zitat Divya, K.A., Roshna, K.I.: A survey on various image enhancement algorithms for naturalness preservation. Int. J. Comput. Sci. Inf. Technol. 6, 2043–2045 (2015) Divya, K.A., Roshna, K.I.: A survey on various image enhancement algorithms for naturalness preservation. Int. J. Comput. Sci. Inf. Technol. 6, 2043–2045 (2015)
13.
Zurück zum Zitat Bedi, S., Khandelwal, R.: Various image enhancement techniques—a critical review. Int. J. Adv. Res. Comput. Commun. Eng. 2(3), 1605–1609 (2013) Bedi, S., Khandelwal, R.: Various image enhancement techniques—a critical review. Int. J. Adv. Res. Comput. Commun. Eng. 2(3), 1605–1609 (2013)
14.
Zurück zum Zitat Yang, F., Wu, J.: An improved image contrast enhancement in multiple-peak images based on histogram equalization. In: 2010 International Conference on Computer Design and Applications, vol. 1, no. 4, pp. 346–349. IEEE (2010) Yang, F., Wu, J.: An improved image contrast enhancement in multiple-peak images based on histogram equalization. In: 2010 International Conference on Computer Design and Applications, vol. 1, no. 4, pp. 346–349. IEEE (2010)
15.
Zurück zum Zitat Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014) Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
16.
Zurück zum Zitat Ignatov, A., Kobyshev, N., Vanhoey, K., Timofte, R., Van Gool, L.: DSLR-quality photos on mobile devices with deep convolutional networks. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), October 2017 Ignatov, A., Kobyshev, N., Vanhoey, K., Timofte, R., Van Gool, L.: DSLR-quality photos on mobile devices with deep convolutional networks. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), October 2017
17.
Zurück zum Zitat Chen, Y.-S., Wang, Y.-C., Kao, M.-H., Chuang, Y.-Y.: Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs. In: Proceedings of the IEEE Conference on Computer Vision Pattern Recognition, pp. 6306–6314 (2018) Chen, Y.-S., Wang, Y.-C., Kao, M.-H., Chuang, Y.-Y.: Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs. In: Proceedings of the IEEE Conference on Computer Vision Pattern Recognition, pp. 6306–6314 (2018)
19.
Zurück zum Zitat Tripathi, S., Lipton, Z.C., Nguyen, T.Q.: Correction by projection: denoising images with generative adversarial networks. arXiv:1803.04477 (2018) Tripathi, S., Lipton, Z.C., Nguyen, T.Q.: Correction by projection: denoising images with generative adversarial networks. arXiv:​1803.​04477 (2018)
20.
Zurück zum Zitat Chen, J., Chen, J., Chao, H., Yang, M.: Image blind denoising with generative adversarial network-based noise modeling. In: Proceedings of the IEEE International Conference Computer Vision Pattern Recognition, pp. 3155–3164, June 2018 Chen, J., Chen, J., Chao, H., Yang, M.: Image blind denoising with generative adversarial network-based noise modeling. In: Proceedings of the IEEE International Conference Computer Vision Pattern Recognition, pp. 3155–3164, June 2018
24.
Zurück zum Zitat Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.: Improved training of Wasserstein GANs.arXiv e-prints arXiv:1704.00028 (2017). Advances in Neural Information Processing Systems 31 (NIPS 2017) Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.: Improved training of Wasserstein GANs.arXiv e-prints arXiv:​1704.​00028 (2017). Advances in Neural Information Processing Systems 31 (NIPS 2017)
25.
Zurück zum Zitat Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (ICLR) (2015) Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (ICLR) (2015)
26.
Zurück zum Zitat Aly, H.A., Dubois, E.: Image up-sampling using total variation regularization with a new observation model. IEEE Trans. Image Process. 14(10), 1647–1659 (2005)CrossRef Aly, H.A., Dubois, E.: Image up-sampling using total variation regularization with a new observation model. IEEE Trans. Image Process. 14(10), 1647–1659 (2005)CrossRef
27.
Zurück zum Zitat Vedaldi, A., Fulkerson, B.: VLFeat: an open and portable library of computer vision algorithms (2008) Vedaldi, A., Fulkerson, B.: VLFeat: an open and portable library of computer vision algorithms (2008)
28.
Zurück zum Zitat Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR, abs/1412.6980 (2014) Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR, abs/1412.6980 (2014)
29.
Zurück zum Zitat Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRef Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRef
30.
Zurück zum Zitat Krishnan, D., Tay, T., Fergus, R.: Blind deconvolution using a normalized sparsity measure. In: CVPR, pp. 233–240 (2011) Krishnan, D., Tay, T., Fergus, R.: Blind deconvolution using a normalized sparsity measure. In: CVPR, pp. 233–240 (2011)
31.
Zurück zum Zitat Pan, J., Hu, Z., Su, Z., Yang, M.-H.: Deblurring text imagesvia L0-regularized intensity and gradient prior. In: CVPR, pp. 2901–2908 (2014) Pan, J., Hu, Z., Su, Z., Yang, M.-H.: Deblurring text imagesvia L0-regularized intensity and gradient prior. In: CVPR, pp. 2901–2908 (2014)
32.
Zurück zum Zitat Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain, November 2011, pp. 2564–2571 (2011) Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain, November 2011, pp. 2564–2571 (2011)
33.
Zurück zum Zitat Calonder, M., Lepetit, V., Özuysal, M., Trzinski, T., Strecha, C., Fua, P.: BRIEF: computing a local binary descriptor very fast. IEEE Trans. Pattern Anal. Machine Intell. 34(7), 1281–1298 (2011)CrossRef Calonder, M., Lepetit, V., Özuysal, M., Trzinski, T., Strecha, C., Fua, P.: BRIEF: computing a local binary descriptor very fast. IEEE Trans. Pattern Anal. Machine Intell. 34(7), 1281–1298 (2011)CrossRef
Metadaten
Titel
Image Registration Improved by Generative Adversarial Networks
verfasst von
Shiyan Jiang
Ci Wang
Chang Huang
Copyright-Jahr
2021
DOI
https://doi.org/10.1007/978-3-030-67835-7_3

Premium Partner