Top

Multimedia Systems

Published in:

05-06-2023 | Regular Paper

HRCutBlur Augment: effectively enhancing data diversity for image super-resolution

Authors: Hong Lin, Xi Wang, Chun Liu, Dewei Peng

Published in: Multimedia Systems | Issue 4/2023

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Data augmentation is a low-cost but effective technique to suppress overfitting due to limited datasets. In this paper, we aim to design efficient data augmentation methods for image super-resolution to expand the number and diversity of data. First, we propose HRCutBlur that mixes a low-resolution image with its corresponding high-resolution image and cut-and-pastes the mixed image patch to the corresponding low-resolution image region. It alleviates the great resolution difference between low-resolution region and high-resolution region in augmented image caused by directly cut-and-pasting the high-resolution image patch to low-resolution image patch. Then, we propose HRCutBlur Augment to solve the insufficient diversity of input images caused by using a certain method alone. The core idea is to design a search space that includes various augmentation methods such as HRCutBlur and Cutout, and use a search algorithm to weighted select methods from the space to augment the input. This design can integrate the advantages of various methods and effectively enrich the diversity of data to adapt to the characteristics of different super-resolution models and various scenarios. Finally, the effectiveness and generality of our methods are verified by designing multi-dimensional experiments on different sizes of image super-resolution models and different benchmark datasets.

previous article FDS_2D: rethinking magnitude-phase features for DeepFake detection

next article Micro-expression spotting network based on attention and one-dimensional convolutional sliding window

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Agustsson, E., Timofte, R.: Ntire 2017 challenge on single image super-resolution: dataset and study. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1122–1131 (2017) https://doi.org/10.1109/CVPRW.2017.150

Ahn, N., Kang, B., Sohn, K.A.: Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European conference on computer vision (ECCV), pp. 252–268 (2018). https://doi.org/10.1007/978-3-030-01249-6_16

Antoniou, A., Storkey, A., Edwards, H. Data augmentation generative adversarial networks. arXiv preprint (2017). https://doi.org/10.48550/arXiv.1711.04340

Bei, Y., Damian, A., Hu, S., et al.: New techniques for preserving global structure and denoising with low information loss in single-image super-resolution. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 987–9877 (2018). https://doi.org/10.1109/CVPRW.2018.00132

Bevilacqua, M., Roumy, A., Guillemot, C., et al.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of the British Machine Vision Conference. BMVA Press, pp. 135.1–135.10 (2012). https://doi.org/10.5244/C.26.135

Cai, J., Zeng, H., Yong, H., et al.: Toward real-world single image super-resolution: a new benchmark and a new model. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3086–3095, (2019). https://doi.org/10.1109/ICCV.2019.00318

Chen, P., Liu, S., Zhao, H., et al.: Gridmask data augmentation. arXiv preprint (2020). https://doi.org/10.48550/arXiv.2001.04086

Chen, C., Xiong, Z., Tian, X., et al.: Camera lens super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2019). https://doi.org/10.1109/CVPR.2019.00175

Cubuk, E.D., Zoph, B., Shlens, J., et al.: Randaugment: practical automated data augmentation with a reduced search space. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 702–703 (2020). https://doi.org/10.1109/CVPRW50498.2020.00359

10.

DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout. arXiv preprint (2017). https://doi.org/10.48550/arXiv.1708.04552

11.

Dong, C., Loy, C.C., He, K., et al.: Learning a deep convolutional network for image super-resolution. In: European Conference on Computer Vision. Springer, pp. 184–199, (2014). https://doi.org/10.1109/ICCVW.2015.17

12.

Feng, R., Gu, J., Qiao, Y., et al.: Suppressing model overfitting for image super-resolution networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), (2019). https://doi.org/10.1109/cvprw.2019.00248

13.

Fujimoto, A., Ogawa, T., Yamamoto, K., et al.: Manga109 dataset and creation of metadata. In: Proceedings of the 1st International Workshop on Comics Analysis, Processing and Understanding, pp. 1–5 (2016). https://doi.org/10.1145/3011549.3011551

14.

Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial nets. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, Vol. 2. nips’14. (2014). https://doi.org/10.48550/arXiv.1406.2661

15.

Ha, V.K., Ren, J., Xu, X., et al.: Deep learning based single image super-resolution: a survey. In: International Conference on Brain Inspired Cognitive Systems, Springer, pp. 106–119, (2018). https://doi.org/10.1007/s11633-019-1183-x

16.

He, K., Sun, J.: Convolutional neural networks at constrained time cost. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5353–5360 (2015). https://doi.org/10.1109/CVPR.2015.7299173

17.

Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5197–5206 (2015). https://doi.org/10.1109/CVPR.2015.7299156

18.

Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016). https://doi.org/10.1109/CVPR.2016.182

19.

LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015). https://doi.org/10.1038/nature14539CrossRef

20.

Lim, B., Son, S., Kim, H., et al.: Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 1132–1140 (2017). https://doi.org/10.1109/CVPRW.2017.151

21.

Liu, J., Liu, B., Zhou, H., et al.: Tokenmix: Rethinking image mixing for data augmentation in vision transformers. In: Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXVI, Springer, pp. 455–471 (2022). https://doi.org/10.48550/arXiv.2207.08409

22.

Lugmayr, A., Danelljan, M., Timofte, R.: Unsupervised learning for real-world super-resolution. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3408–3416 (2019). https://doi.org/10.1109/ICCVW.2019.00423

23.

Mariani, G., Scheidegger, F., Istrate, R., et al.: Bagan: data augmentation with balancing gan. arXiv preprint (2018). https://doi.org/10.48550/arXiv.1803.09655

24.

Mei, Y., Fan, Y., Zhou, Y.: Image super-resolution with non-local sparse attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3517–3526 (2021). https://doi.org/10.1109/CVPR46437.2021.00352

25.

Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 1–48 (2019). https://doi.org/10.1186/s40537-019-0197-0CrossRef

26.

Tong, T., Li, G., Liu, X., et al.: Image super-resolution using dense skip connections. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4809–4817 (2017). https://doi.org/10.1109/ICCV.2017.514

27.

Vu, T., Van Nguyen, C., Pham, T.X., et al.: Fast and efficient image quality enhancement via desubpixel convolutional neural networks. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops (2018). https://doi.org/10.1007/978-3-030-11021-5_16

28.

Wang, Z., Bovik, A.C., Sheikh, H.R., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861CrossRef

29.

Wang, Z., Chen, J., Hoi, S.C.H.: Deep learning for image super-resolution: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 43(10), 3365–3387 (2021). https://doi.org/10.1109/TPAMI.2020.2982166CrossRef

30.

Wu, Y., Guo, H., Chakraborty, C., et al.: Edge computing driven low-light image dynamic enhancement for object detection. IEEE Trans. Netw. Sci. Eng (2022). https://doi.org/10.1109/TNSE.2022.3151502CrossRef

31.

Yoo, J., Ahn, N., Sohn, K.A.: Rethinking data augmentation for image super-resolution: a comprehensive analysis and a new strategy. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8375–8384 (2020). https://doi.org/10.1109/CVPR42600.2020.00840

32.

Yun, S., Han, D., Oh, S.J., et al.: Cutmix: regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6023–6032 (2019). https://doi.org/10.1109/ICCV.2019.00612

33.

Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: International Conference on Curves and Surfaces, Springer, pp. 711–730 (2010). https://doi.org/10.1007/978-3-642-27413-8_47

34.

Zhang, H., Cisse, M., Dauphin, Y.N., et al.: mixup: Beyond empirical risk minimization. arXiv preprint (2017). https://doi.org/10.48550/arXiv.1710.09412

35.

Zhang, Y., Li, K., Li, K., et al.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 286–301 (2018). https://doi.org/10.1007/978-3-030-01234-2_18

Title: HRCutBlur Augment: effectively enhancing data diversity for image super-resolution
Authors: Hong Lin
Xi Wang
Chun Liu
Dewei Peng
Publication date: 05-06-2023
Publisher: Springer Berlin Heidelberg
Published in: Multimedia Systems / Issue 4/2023
Print ISSN: 0942-4962
Electronic ISSN: 1432-1882
DOI: https://doi.org/10.1007/s00530-023-01110-0

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 4/2023

View-aware attribute-guided network for vehicle re-identification

Multi-level network based on transformer encoder for fine-grained image–text matching

Entropy minimization and domain adversarial training guided by label distribution similarity for domain adaptation

Centralized sub-critic based hierarchical-structured reinforcement learning for temporal sentence grounding

E-Cap Net: an efficient-capsule network for shallow and deepfakes forgery detection

Virtual reality in medical emergencies training: benefits, perceived stress, and learning success