Top

International Journal of Computer Vision

Published in:

18-04-2023

Advanced Binary Neural Network for Single Image Super Resolution

Authors: Jingwei Xin, Nannan Wang, Xinrui Jiang, Jie Li, Xinbo Gao

Published in: International Journal of Computer Vision | Issue 7/2023

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Binary neural network (BNN) is an effective approach to accelerate the model inference and has been initially applied in the field of single image super resolution (SISR). However, the optimization of efficiency and accuracy remains a major challenge for achieving further improvements. While existing BNN-based SR methods solve the SISR problems by proposing a residual block-oriented quantization mechanism, the quantization process in the up-sampling stage and the representation tendency of binary super resolution networks are ignored. In this paper, we propose an Advanced Binary Super Resolution (ABSR) method to optimize the binary generator in terms of quantization mechanism and up-sampling strategy. Specifically, we first design an excitation-selection mechanism for binary inference, which could distinctively implement self-adjustment of activation and significantly reduce inference errors. Furthermore, we construct a binary up-sampling strategy that achieves performance almost equal to that of real-valued up-sampling modules, and fully frees up the inference speed of the binary network. Extensive experiments show that the ABSR not only reaches state-of-the-art BNN-based SR performance in terms of objective metrics and visual quality, but also reduces computational consumption drastically.

previous article Neural Architecture Search for Dense Prediction Tasks in Computer Vision

next article Trust-Region Adaptive Frequency for Online Continual Learning

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Ahn, N., Kang, B., & Sohn, KA .(2018). Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European conference on computer vision (ECCV).

Antoni, B., Joan, D., & Julia, N. (2019). Motion-compensated Spatio-temporal filtering for multi-image and multimodal super-resolution. International Journal of Computer Vision, 127(10), 1474–1500.CrossRef

Bevilacqua, M., Roumy, A., Guillemot, C., & Alberi-Morel, ML. (2012). Low-complexity single-image super-resolution based on nonnegative neighbor embedding pp. 1–10.

Bulat, A., & Tzimiropoulos, G. (2019). Xnor-net++: Improved binary neural networks. arXiv preprint arXiv:1909.13863.

Chen, TQ., Rubanova, Y., Bettencourt, J., & Duvenaud, DK. (2018). Neural ordinary differential equations. In: Advances in neural information processing systems, pp. 6571–6583.

Courbariaux, M., Bengio, Y., & David, JP. (2015). Binaryconnect: Training deep neural networks with binary weights during propagations. In: Advances in .neural information processing systems, pp. 3123–3131

Dong, C., Loy, C. C., He, K., & Tang, X. (2015). Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2), 295–307.CrossRef

Greenspan, H. (2009). Super-resolution in medical imaging. The Computer Journal, 52(1), 43–63.CrossRef

Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., & Sun, J. (2020). Single path one-shot neural architecture search with uniform sampling. In: Proceedings of the European conference on computer vision (ECCV), Springer, vol 12361, pp. 544–560.

He, X., Mo, Z., Wang, P., Liu, Y., Yang, M., & Cheng, J. (2019). Ode-inspired network design for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

Huang, JB., Singh, A., & Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 5197–5206.

Huang, Y., Shao, L., & Frangi, AF. (2017). Simultaneous super-resolution and cross-modality synthesis of 3d medical images using weakly-supervised joint convolutional sparse coding. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016). Binarized neural networks. In: Advances in neural information processing systems, pp 4107–4115.

Hui, Z., Gao, X., Yang, Y., & Wang, X. (2019). Lightweight image super-resolution with information multi-distillation network. In: Proceedings of the 27th ACM international conference on multimedia (ACM MM).

Jiang, X., Wang, N., Xin, J., Li, K., Yang, X., & Gao, X. (2021). Training binary neural network without batch normalization for image super-resolution. In: Proceedings of the thirty-fifth AAAI conference on artificial intelligence (AAAI).

Jiang, X., Wang, N., Xin, J., Li, K., Yang, X., Li, X., Wang, Jie, & Gao, X. (2022). Fabnet: Frequency-aware binarized network for single image super-resolution. In: IEEE Transactions on neural networks and learning systems. pp. 1–11.

Kim, J., Kwon, Lee, J., & Mu, Lee, K. (2016). Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

Lai, WS., Huang, JB., Ahuja, N,. & Yang, MH. (2017). Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., & Wang, Z. et al (2017). Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

Lei, Z., Peng, W., Chunhua, S., Lingqiao, L., Wei, W., Yanning, Z., & van den Hengel, A. (2020). Adaptive importance learning for improving lightweight image super-resolution network. International Journal of Computer Vision, 128(2), 479–499.MathSciNetCrossRef

Li, K., Wang, N., Xin, J., Jiang, X., Li, J., Gao, X., Han, K., & Wang, Y. (2022). Local means binary networks for image super-resolution. IEEE Transactions on Neural Networks and Learning Systems pp 1–11, 10.1109/TNNLS.2022.3212827.

Li, Y., Dong, X., Zhang, SQ., Bai, H., Chen, Y., & Wang, W. (2020). Rtn: Reparameterized ternary network. Proceedings of the thirty-fourth AAAI conference on artificial intelligence (AAAI).

Lim, B., Son, S., Kim, H., Nah, S., & Mu, Lee, K. (2017). Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops (CVPRW).

Lin, F., Fookes, C., Chandran, V., & Sridharan, S. (2007). Super-resolved faces for improved face recognition from surveillance video. In: Proceedings of the international conference on advances in biometrics.

Liu, J., Zhang, W., Tang, Y., Tang, J., & Wu, G. (2020a). Residual feature aggregation network for image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2359–2368.

Liu, P., Zhang, H., Wei, L., & Zuo, W. (2019). Multi-level wavelet convolutional neural networks. IEEE Access, 7, 74973–74985.CrossRef

Liu, Z., Wu, B., Luo, W., Yang, X., Liu, W., & Cheng, KT. (2018). Bi-real net: Enhancing the performance of 1-bit cnns with improved representational capability and advanced training algorithm. In: Proceedings of the European conference on computer vision (ECCV).

Liu, Z., Shen, Z., Savvides, M., & Cheng, KT. (2020b). Reactnet: Towards precise binary neural network with generalized activation functions. arXiv preprint arXiv:2003.03488,

Ma, Y., Xiong, H., Hu, Z., & Ma, L. (2019). Efficient super resolution using binarized neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops (CVPRW).

Maas, AL., Hannun, AY., & Ng, AY. (2013). Rectifier nonlinearities improve neural network acoustic models. In: Proceeding ICML, vol 30, p. 3.

Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. ICCV, IEEE, 2, 416–423.

Qin, H., Gong, R., Liu, X., Wei, Z., Yu, F., & Song, J. (2020). Ir-net: Forward and backward information retention for highly accurate binary neural networks. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016). Xnor-net: Imagenet classification using binary convolutional neural networks. In: Proceedings of the European conference on computer vision (ECCV).

Rasti, P., Uiboupin, T., Escalera, S., & Anbarjafari, G. (2016). Convolutional neural network super resolution for face recognition in surveillance monitoring. In: International conference on articulated motion and deformable objects.

Timofte, R., Lee, KM., Wang, X,, Tian, Y., Ke, Y., Zhang, Y., Wu, S., Chao, D., Liang, L., & Yu, Q. (2017). Ntire 2017 challenge on single image super-resolution: Methods and results. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

Wang, Y., Lu, Y., & Blankevoort, T. (2020). Differentiable joint pruning and quantization for hardware efficiency. In: Proceedings of the European conference on computer vision (ECCV), Springer, vol 12374, pp 259–277

Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.

Xin, J., Li, J., Jiang, X., Wang, N., Huang, H., & Gao, X. (2020a). Wavelet-based dual recursive network for image super-resolution. IEEE Transactions on Neural Networks and Learning Systems.

Xin, J., Wang, N., Jiang, X., Li, J., Huang, H., & Gao, X. (2020b). Binarized neural network for single image super resolution. In: Proceedings of the European conference on computer vision (ECCV).

Xu, Z., & Cheung, RC. (2019). Accurate and compact convolutional neural networks with trained binarization. arXiv preprint arXiv:1909.11366

Zeyde, R., Elad, M., & Protter, M. (2012). On single image scale-up using sparse-representations. In: Proceedings of the international conference on curves and surfaces.

Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018a). Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision (ECCV).

Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2018b). Residual dense network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

Title: Advanced Binary Neural Network for Single Image Super Resolution
Authors: Jingwei Xin
Nannan Wang
Xinrui Jiang
Jie Li
Xinbo Gao
Publication date: 18-04-2023
Publisher: Springer US
Published in: International Journal of Computer Vision / Issue 7/2023
Print ISSN: 0920-5691
Electronic ISSN: 1573-1405
DOI: https://doi.org/10.1007/s11263-023-01789-8

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 7/2023

Adaptive Deep PnP Algorithm for Video Snapshot Compressive Imaging

Disassembling Convolutional Segmentation Network

Adversarial Learning Domain-Invariant Conditional Features for Robust Face Anti-spoofing

Multi-target Knowledge Distillation via Student Self-reflection

Trust-Region Adaptive Frequency for Online Continual Learning

Neural Architecture Search for Dense Prediction Tasks in Computer Vision

Premium Partner