Skip to main content
Erschienen in: International Journal of Machine Learning and Cybernetics 5/2024

28.10.2023 | Original Article

GCAM: lightweight image inpainting via group convolution and attention mechanism

verfasst von: Yuantao Chen, Runlong Xia, Kai Yang, Ke Zou

Erschienen in: International Journal of Machine Learning and Cybernetics | Ausgabe 5/2024

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Recently, image inpainting techniques tend to be more concerned with how to enhance the quality of restoration than with how to function on various platforms with limited processing power. In this paper, we propose a lightweight method that combines group convolution and attention mechanism to improve or replace the traditional convolution module. Group convolution was used to achieve multi-level image inpainting, and the authors proposed the rotating attention mechanism for allocation to deal with the issue of information mobility between channels in traditional convolution processing. The parallel discriminator structure was utilized throughout the network's overall design phase to guarantee both local and global consistency of the image inpainting process. The experimental results can demonstrate that, while the quality of image inpainting has been ensured, the proposed image inpainting network's inference time and resource usage are significantly lower than those of comparable lightweight approaches.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Weitere Produktempfehlungen anzeigen
Literatur
1.
Zurück zum Zitat Harald G (2004) A combined PDE and texture synthesis approach to in-painting. In: Proceedings of 8th European Conference on Computer Vision, Prague, Czech Republic, pp 214–224. Harald G (2004) A combined PDE and texture synthesis approach to in-painting. In: Proceedings of 8th European Conference on Computer Vision, Prague, Czech Republic, pp 214–224.
2.
Zurück zum Zitat Rane SD, Sapiro G, Bertalmio M (2003) Structure and texture filling-in of missing image blocks in wireless transmission and compression applications. IEEE Trans Image Process 12(3):296–303MathSciNetCrossRef Rane SD, Sapiro G, Bertalmio M (2003) Structure and texture filling-in of missing image blocks in wireless transmission and compression applications. IEEE Trans Image Process 12(3):296–303MathSciNetCrossRef
3.
Zurück zum Zitat Criminisi A, Pérez P, Toyama K (2004) Region filling and object removal by exemplar-based image inpainting. IEEE Trans Image Process 13(9):1200–1212CrossRef Criminisi A, Pérez P, Toyama K (2004) Region filling and object removal by exemplar-based image inpainting. IEEE Trans Image Process 13(9):1200–1212CrossRef
4.
Zurück zum Zitat Shen J, Chan TF (2002) Mathematical models for local nontexture inpaintings. J SIAM Appl Math 62(3):1019–1043MathSciNetCrossRef Shen J, Chan TF (2002) Mathematical models for local nontexture inpaintings. J SIAM Appl Math 62(3):1019–1043MathSciNetCrossRef
5.
Zurück zum Zitat Hays J, Efros AA (2007) Scene completion using millions of photographs. ACM Trans Graph 26(3):4CrossRef Hays J, Efros AA (2007) Scene completion using millions of photographs. ACM Trans Graph 26(3):4CrossRef
6.
Zurück zum Zitat Whyte O, Sivic J, Zisserman A (2009) Get out of my picture! Internet-based inpainting. In: British Machine Vision Conference (BMVC). London, UK, pp 1–11. Whyte O, Sivic J, Zisserman A (2009) Get out of my picture! Internet-based inpainting. In: British Machine Vision Conference (BMVC). London, UK, pp 1–11.
7.
Zurück zum Zitat Wadhwa G, Dhall A, Murala S, Tariq U (2021) Hyperrealistic image inpainting with hypergraphs. In: Proceedings of 2021 IEEE Winter Conference on Applications of Computer Vision, Waikoloa, USA, pp 3912–3921. Wadhwa G, Dhall A, Murala S, Tariq U (2021) Hyperrealistic image inpainting with hypergraphs. In: Proceedings of 2021 IEEE Winter Conference on Applications of Computer Vision, Waikoloa, USA, pp 3912–3921.
8.
Zurück zum Zitat Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:​1511.​06434.
9.
Zurück zum Zitat Yu J H, Lin Z, Yang J M, Shen X H, Lu X, Huang T S (2019) Free-form image inpainting with gated convolution. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul, Korea (South), pp 4470–4479. Yu J H, Lin Z, Yang J M, Shen X H, Lu X, Huang T S (2019) Free-form image inpainting with gated convolution. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul, Korea (South), pp 4470–4479.
10.
Zurück zum Zitat Zeng Y H, Fu J L, Chao H Y, Guo B N (2019) Learning pyramid-context encoder network for high-quality image inpainting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA, pp 1486–1494. Zeng Y H, Fu J L, Chao H Y, Guo B N (2019) Learning pyramid-context encoder network for high-quality image inpainting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA, pp 1486–1494.
11.
Zurück zum Zitat Pathak D, Krähenbühl P, Donahue J, Darrell T, Efros A A (2016) Context encoders: Feature learning by inpainting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NA, USA, pp 2536–2544. Pathak D, Krähenbühl P, Donahue J, Darrell T, Efros A A (2016) Context encoders: Feature learning by inpainting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NA, USA, pp 2536–2544.
12.
Zurück zum Zitat Yu J H, Lin Z, Yang J M, Shen X H, Huang T S (2018) Generative image inpainting with contextual attention. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA, pp 5505–5514. Yu J H, Lin Z, Yang J M, Shen X H, Huang T S (2018) Generative image inpainting with contextual attention. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA, pp 5505–5514.
13.
Zurück zum Zitat Broad T, Grierson M (2016) Light field completion using focal stack propagation. In: SIGGRAPH’16: Special Interest Group on Computer Graphics and Interactive Techniques Conference. Anaheim, CA, USA, pp 54: 1–54:2. Broad T, Grierson M (2016) Light field completion using focal stack propagation. In: SIGGRAPH’16: Special Interest Group on Computer Graphics and Interactive Techniques Conference. Anaheim, CA, USA, pp 54: 1–54:2.
14.
Zurück zum Zitat Sagong M, Shin Y G, Kim S W, Park S, Ko S J. PEPSI: Fast image inpainting with parallel decoding network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA, 2019: 11360–11368. Sagong M, Shin Y G, Kim S W, Park S, Ko S J. PEPSI: Fast image inpainting with parallel decoding network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA, 2019: 11360–11368.
15.
Zurück zum Zitat Khan S H, Naseer M, Hayat M, Zamir S W, Khan F S, Shah M (2022) Transformers in vision: A survey. ACM Computing Surveys, 54(10s): 200: 1–200: 41. Khan S H, Naseer M, Hayat M, Zamir S W, Khan F S, Shah M (2022) Transformers in vision: A survey. ACM Computing Surveys, 54(10s): 200: 1–200: 41.
16.
Zurück zum Zitat Lin TY, Wang YX, Liu XY, Qiu XP (2022) A Survey of Transformers AI Open 3:111–132CrossRef Lin TY, Wang YX, Liu XY, Qiu XP (2022) A Survey of Transformers AI Open 3:111–132CrossRef
17.
Zurück zum Zitat Chen Q W, Zhao H, Li W, Huang P P, Ou W W (2019) Behavior sequence transformer for e-commerce recommendation in Alibaba. In: Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data. Anchorage, arXiv: 1905.06874. Chen Q W, Zhao H, Li W, Huang P P, Ou W W (2019) Behavior sequence transformer for e-commerce recommendation in Alibaba. In: Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data. Anchorage, arXiv: 1905.06874.
18.
Zurück zum Zitat Howard A G, Zhu M, Chen B, Kalenichenko D, Wang W J, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv: 1704.04861. Howard A G, Zhu M, Chen B, Kalenichenko D, Wang W J, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv: 1704.04861.
19.
Zurück zum Zitat Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S E, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA, pp 1–9. Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S E, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA, pp 1–9.
20.
Zurück zum Zitat Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90CrossRef Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90CrossRef
21.
Zurück zum Zitat Ma N N, Zhang X Y, Zheng H T, Sun J (2018) ShuffleNet V2: Practical guidelines for efficient cnn architecture design. In: Proceedings of 15th European Conference on Computer, Vision, Munich, Germany, pp 122–138. Ma N N, Zhang X Y, Zheng H T, Sun J (2018) ShuffleNet V2: Practical guidelines for efficient cnn architecture design. In: Proceedings of 15th European Conference on Computer, Vision, Munich, Germany, pp 122–138.
22.
Zurück zum Zitat Hu J, Shen L, Sun G, Wu EH (2020) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42(8):2011–2023CrossRef Hu J, Shen L, Sun G, Wu EH (2020) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42(8):2011–2023CrossRef
23.
Zurück zum Zitat Liu Z W, Luo P, Wang X G, Tang X O (2015) Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile, pp 3730–3738. Liu Z W, Luo P, Wang X G, Tang X O (2015) Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile, pp 3730–3738.
24.
Zurück zum Zitat Zhou BL, Lapedriza A, Khosla A, Oliva A, Torralba A (2017) Places: A 10 million image database for scene recognition. IEEE Trans Pattern Anal Mach Intell 40(6):1452–1464CrossRef Zhou BL, Lapedriza A, Khosla A, Oliva A, Torralba A (2017) Places: A 10 million image database for scene recognition. IEEE Trans Pattern Anal Mach Intell 40(6):1452–1464CrossRef
25.
Zurück zum Zitat Shin YG, Sagong MC, Yeo YJ, Kim SW, Ko SJ (2021) PEPSI++ : fast and lightweight network for image inpainting. IEEE Trans Neural Netw Learn Syst 32(1):252–265CrossRef Shin YG, Sagong MC, Yeo YJ, Kim SW, Ko SJ (2021) PEPSI++ : fast and lightweight network for image inpainting. IEEE Trans Neural Netw Learn Syst 32(1):252–265CrossRef
26.
Zurück zum Zitat Yang J, Qi Z, Shi Y (2020) Learning to incorporate structure knowledge for image inpainting. Proc AAAI Conf Artif Intell 34:12605–12612 Yang J, Qi Z, Shi Y (2020) Learning to incorporate structure knowledge for image inpainting. Proc AAAI Conf Artif Intell 34:12605–12612
27.
Zurück zum Zitat Movassagh AA, Alzubi JA, Gheisari M, Rahimi M, Mohan SK, Abbasi AA, Nabipour N (2023) Artificial neural networks training algorithm integrating invasive weed optimization with differential evolutionary model. J Amb Intell Human Comput 13:6017–6025CrossRef Movassagh AA, Alzubi JA, Gheisari M, Rahimi M, Mohan SK, Abbasi AA, Nabipour N (2023) Artificial neural networks training algorithm integrating invasive weed optimization with differential evolutionary model. J Amb Intell Human Comput 13:6017–6025CrossRef
28.
Zurück zum Zitat Alzubi OA, Alzubi JA, Alweshah M, Qiqieh I, Shami SA, Ramachandran M (2020) An optimal pruning algorithm of classifier ensembles: dynamic programming approach. Neural Comput Appl 32(20):16091–16107CrossRef Alzubi OA, Alzubi JA, Alweshah M, Qiqieh I, Shami SA, Ramachandran M (2020) An optimal pruning algorithm of classifier ensembles: dynamic programming approach. Neural Comput Appl 32(20):16091–16107CrossRef
29.
Zurück zum Zitat Alzubi JA, Jain R, Alzubi O, Thareja A, Upadhyay Y (2022) Distracted driver detection using compressed energy efficient convolutional neural network. J Intell Fuzzy Syst 42(2):1253–1265CrossRef Alzubi JA, Jain R, Alzubi O, Thareja A, Upadhyay Y (2022) Distracted driver detection using compressed energy efficient convolutional neural network. J Intell Fuzzy Syst 42(2):1253–1265CrossRef
30.
Zurück zum Zitat Alzubi JA (2016) Diversity-based boosting algorithm. Int J Adv Comput Sci Appl 7(5):524–529 Alzubi JA (2016) Diversity-based boosting algorithm. Int J Adv Comput Sci Appl 7(5):524–529
31.
Zurück zum Zitat Doersch C, Singh S, Gupta A, Sivic J, Efros AA (2012) What makes Paris look like Paris? ACM Trans Graph 31(4):1–9CrossRef Doersch C, Singh S, Gupta A, Sivic J, Efros AA (2012) What makes Paris look like Paris? ACM Trans Graph 31(4):1–9CrossRef
Metadaten
Titel
GCAM: lightweight image inpainting via group convolution and attention mechanism
verfasst von
Yuantao Chen
Runlong Xia
Kai Yang
Ke Zou
Publikationsdatum
28.10.2023
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal of Machine Learning and Cybernetics / Ausgabe 5/2024
Print ISSN: 1868-8071
Elektronische ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-023-01999-z

Weitere Artikel der Ausgabe 5/2024

International Journal of Machine Learning and Cybernetics 5/2024 Zur Ausgabe

Neuer Inhalt