Skip to main content
Top
Published in: Neural Computing and Applications 11/2023

31-01-2022 | S.I. : TAM-LHR

MDVA-GAN: multi-domain visual attribution generative adversarial networks

Authors: Muhammad Nawaz, Feras Al-Obeidat, Abdallah Tubaishat, Tehseen Zia, Fahad Maqbool, Alvaro Rocha

Published in: Neural Computing and Applications | Issue 11/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Some pixels of an input image have thick information and give insights about a particular category during classification decisions. Visualization of these pixels is a well-studied problem in computer vision, called visual attribution (VA), which helps radiologists to recognize abnormalities and identify a particular disease in the medical image. In recent years, several classification-based techniques for domain-specific attribute visualization have been proposed, but these techniques can only highlight a small subset of most discriminative features. Therefore, their generated VA maps are inadequate to visualize all effects in an input image. Due to recent advancements in generative models, generative model-based VA techniques are introduced which generate efficient VA maps and visualize all affected regions. To deal the issue, generative adversarial network-based VA techniques are recently proposed, where the researchers leverage the advances in domain adaption techniques to learn a map for abnormal-to-normal medical image translation. As these approaches rely on a two-domain translation model, it would require training as many models as number of diseases in a medical dataset, which is a tedious and compute-intensive task. In this work, we introduce a unified multi-domain VA model that generates a VA map of more than one disease at a time. The proposed unified model gets images from a particular domain and its domain label as input to generate VA map and visualize all the affected regions by that particular disease. Experiments on the CheXpert dataset, which is a publicly available multi-disease chest radiograph dataset, and the TBX11K dataset show that the proposed model generates identical results.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 2921–2929) Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 2921–2929)
2.
go back to reference Feng X, Yang J, Laine AF, Angelini ED (2017, September) Discriminative localization in CNNs for weakly-supervised segmentation of pulmonary nodules. In: International conference on medical image computing and computer-assisted intervention (pp 568–576). Springer, Cham Feng X, Yang J, Laine AF, Angelini ED (2017, September) Discriminative localization in CNNs for weakly-supervised segmentation of pulmonary nodules. In: International conference on medical image computing and computer-assisted intervention (pp 568–576). Springer, Cham
3.
go back to reference Fong RC, Vedaldi A (2017) Interpretable explanations of black boxes by meaningful perturbation. In: Proceedings of the IEEE International Conference on Computer Vision (pp 3429–3437) Fong RC, Vedaldi A (2017) Interpretable explanations of black boxes by meaningful perturbation. In: Proceedings of the IEEE International Conference on Computer Vision (pp 3429–3437)
4.
go back to reference Ge Z, Demyanov S, Chakravorty R, Bowling A, Garnavi R (2017, September) Skin disease recognition using deep saliency features and multimodal learning of dermoscopy and clinical images. In: International conference on medical image computing and computer-assisted intervention (pp 250–258). Springer, Cham. (year) Ge Z, Demyanov S, Chakravorty R, Bowling A, Garnavi R (2017, September) Skin disease recognition using deep saliency features and multimodal learning of dermoscopy and clinical images. In: International conference on medical image computing and computer-assisted intervention (pp 250–258). Springer, Cham. (year)
5.
go back to reference Baumgartner CF, Kamnitsas K, Matthew J, Fletcher TP, Smith S, Koch LM, Rueckert D (2017) SonoNet: real-time detection and localisation of fetal standard scan planes in freehand ultrasound. IEEE Trans Med Imaging 36(11):2204–2215CrossRef Baumgartner CF, Kamnitsas K, Matthew J, Fletcher TP, Smith S, Koch LM, Rueckert D (2017) SonoNet: real-time detection and localisation of fetal standard scan planes in freehand ultrasound. IEEE Trans Med Imaging 36(11):2204–2215CrossRef
6.
go back to reference Sundararajan M, Taly A, Yan Q (2017, July) Axiomatic attribution for deep networks. In International Conference on Machine Learning (pp. 3319-3328). PMLR Sundararajan M, Taly A, Yan Q (2017, July) Axiomatic attribution for deep networks. In International Conference on Machine Learning (pp. 3319-3328). PMLR
8.
go back to reference Sun L, Wang J, Huang Y, Ding X, Greenspan H, Paisley J (2020) An adversarial learning approach to medical image synthesis for lesion detection. IEEE J Biomed Health Informatics 24(8):2303–2314CrossRef Sun L, Wang J, Huang Y, Ding X, Greenspan H, Paisley J (2020) An adversarial learning approach to medical image synthesis for lesion detection. IEEE J Biomed Health Informatics 24(8):2303–2314CrossRef
9.
go back to reference Baumgartner CF, Koch LM, Tezcan KC, Ang JX, Konukoglu E (2018) Visual feature attribution using wasserstein gans. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 8309–8319) Baumgartner CF, Koch LM, Tezcan KC, Ang JX, Konukoglu E (2018) Visual feature attribution using wasserstein gans. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 8309–8319)
10.
go back to reference Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Bengio Y (2014) Generative adversarial networks. arXiv:1406.2661 Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Bengio Y (2014) Generative adversarial networks. arXiv:​1406.​2661
11.
go back to reference Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision (pp 2223–2232) Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision (pp 2223–2232)
12.
go back to reference Arjovsky M, Chintala S, Bottou L (2017, July) Wasserstein generative adversarial networks. In: International conference on machine learning (pp 214–223). PMLR Arjovsky M, Chintala S, Bottou L (2017, July) Wasserstein generative adversarial networks. In: International conference on machine learning (pp 214–223). PMLR
13.
go back to reference Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 8789–8797) Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 8789–8797)
14.
go back to reference Choi Y, Uh Y, Yoo J, Ha JW (2020) Stargan v2: Diverse image synthesis for multiple domains. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp 8188–8197) Choi Y, Uh Y, Yoo J, Ha JW (2020) Stargan v2: Diverse image synthesis for multiple domains. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp 8188–8197)
15.
go back to reference He Z, Zuo W, Kan M, Shan S, Chen X (2019) Attgan: facial attribute editing by only changing what you want. IEEE Trans Image Process 28(11):5464–5478MathSciNetCrossRefMATH He Z, Zuo W, Kan M, Shan S, Chen X (2019) Attgan: facial attribute editing by only changing what you want. IEEE Trans Image Process 28(11):5464–5478MathSciNetCrossRefMATH
16.
go back to reference Ping Q, Wu B, Ding W, Yuan J (2019) Fashion-AttGAN: Attribute-aware fashion editing with multi-objective GAN. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops (pp 0-0) Ping Q, Wu B, Ding W, Yuan J (2019) Fashion-AttGAN: Attribute-aware fashion editing with multi-objective GAN. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops (pp 0-0)
17.
go back to reference Liu M, Ding Y, Xia M, Liu X, Ding E, Zuo W, Wen S (2019) STGAN: A unified selective transfer network for arbitrary image attribute editing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp 3673–3682) Liu M, Ding Y, Xia M, Liu X, Ding E, Zuo W, Wen S (2019) STGAN: A unified selective transfer network for arbitrary image attribute editing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp 3673–3682)
18.
go back to reference Huang X, Li Y, Poursaeed O, Hopcroft J, Belongie S (2017) Stacked generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 5077–5086) Huang X, Li Y, Poursaeed O, Hopcroft J, Belongie S (2017) Stacked generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 5077–5086)
19.
go back to reference Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of gans for improved quality, stability, and variation. arXiv:1710.10196 Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of gans for improved quality, stability, and variation. arXiv:​1710.​10196
20.
go back to reference Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 1125–1134) Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 1125–1134)
21.
go back to reference Kim T, Cha M, Kim H, Lee JK, Kim J (2017, July) Learning to discover cross-domain relations with generative adversarial networks. In: International conference on machine learning (pp 1857–1865). PMLR Kim T, Cha M, Kim H, Lee JK, Kim J (2017, July) Learning to discover cross-domain relations with generative adversarial networks. In: International conference on machine learning (pp 1857–1865). PMLR
22.
go back to reference Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 4681–4690) Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 4681–4690)
23.
go back to reference Kim T, Kim B, Cha M, Kim J (2017) Unsupervised visual attribute transfer with reconfigurable generative adversarial networks. arXiv:1707.09798 Kim T, Kim B, Cha M, Kim J (2017) Unsupervised visual attribute transfer with reconfigurable generative adversarial networks. arXiv:​1707.​09798
24.
go back to reference Shen W, Liu R (2017) Learning residual images for face attribute manipulation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 4030–4038) Shen W, Liu R (2017) Learning residual images for face attribute manipulation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 4030–4038)
25.
go back to reference Nie D, Trullo R, Lian J, Petitjean C, Ruan S, Wang Q, Shen D (2017, September) Medical image synthesis with context-aware generative adversarial networks. In: International conference on medical image computing and computer-assisted intervention (pp 417–425). Springer, Cham Nie D, Trullo R, Lian J, Petitjean C, Ruan S, Wang Q, Shen D (2017, September) Medical image synthesis with context-aware generative adversarial networks. In: International conference on medical image computing and computer-assisted intervention (pp 417–425). Springer, Cham
26.
go back to reference Zhang Z, Yang L, Zheng Y (2018) Translating and segmenting multimodal medical volumes with cycle-and shape-consistency generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern Recognition (pp 9242–9251) Zhang Z, Yang L, Zheng Y (2018) Translating and segmenting multimodal medical volumes with cycle-and shape-consistency generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern Recognition (pp 9242–9251)
27.
go back to reference Chartsias A, Joyce T, Giuffrida MV, Tsaftaris SA (2017) Multimodal MR synthesis via modality-invariant latent representation. IEEE Trans Med Imaging 37(3):803–814CrossRef Chartsias A, Joyce T, Giuffrida MV, Tsaftaris SA (2017) Multimodal MR synthesis via modality-invariant latent representation. IEEE Trans Med Imaging 37(3):803–814CrossRef
28.
go back to reference Cao B, Zhang H, Wang N, Gao X, Shen D (2020, April) Auto-GAN: self-supervised collaborative learning for medical image synthesis. In: Proceedings of the AAAI conference on artificial intelligence (Vol 34, No 07, pp 10486-10493) Cao B, Zhang H, Wang N, Gao X, Shen D (2020, April) Auto-GAN: self-supervised collaborative learning for medical image synthesis. In: Proceedings of the AAAI conference on artificial intelligence (Vol 34, No 07, pp 10486-10493)
29.
go back to reference Zhang Y, Yang L, Chen J, Fredericksen M, Hughes DP, Chen DZ (2017, September) Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In: International conference on medical image computing and computer-assisted intervention (pp 408–416). Springer, Cham Zhang Y, Yang L, Chen J, Fredericksen M, Hughes DP, Chen DZ (2017, September) Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In: International conference on medical image computing and computer-assisted intervention (pp 408–416). Springer, Cham
30.
go back to reference Yang D, Xu D, Zhou SK, Georgescu B, Chen M, Grbic S, Comaniciu D (2017, September) Automatic liver segmentation using an adversarial image-to-image network. In: International conference on medical image computing and computer-assisted intervention (pp 507–515). Springer, Cham Yang D, Xu D, Zhou SK, Georgescu B, Chen M, Grbic S, Comaniciu D (2017, September) Automatic liver segmentation using an adversarial image-to-image network. In: International conference on medical image computing and computer-assisted intervention (pp 507–515). Springer, Cham
31.
go back to reference Yu K, Wang Y, Cai Y, Xiao C, Zhao E, Glass L, Sun J (2019) Rare disease detection by sequence modeling with generative adversarial networks. arXiv:1907.01022 Yu K, Wang Y, Cai Y, Xiao C, Zhao E, Glass L, Sun J (2019) Rare disease detection by sequence modeling with generative adversarial networks. arXiv:​1907.​01022
32.
go back to reference Park KB, Choi SH, Lee JY (2020) M-gan: Retinal blood vessel segmentation by balancing losses through stacked deep fully convolutional networks. IEEE Access 8:146308–146322CrossRef Park KB, Choi SH, Lee JY (2020) M-gan: Retinal blood vessel segmentation by balancing losses through stacked deep fully convolutional networks. IEEE Access 8:146308–146322CrossRef
33.
go back to reference Oquab M, Bottou L, Laptev I, Sivic J (2015) Is object localization for free?-weakly-supervised learning with convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 685–694) Oquab M, Bottou L, Laptev I, Sivic J (2015) Is object localization for free?-weakly-supervised learning with convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 685–694)
34.
go back to reference Gondal WM, Köhler JM, Grzeszick R, Fink GA, Hirsch M (2017, September) Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images. In: 2017 IEEE international conference on image processing (ICIP) (pp 2069–2073). IEEE Gondal WM, Köhler JM, Grzeszick R, Fink GA, Hirsch M (2017, September) Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images. In: 2017 IEEE international conference on image processing (ICIP) (pp 2069–2073). IEEE
36.
go back to reference Zhang J, Bargal SA, Lin Z, Brandt J, Shen X, Sclaroff S (2018) Top-down neural attention by excitation backprop. Int J Comput Vis 126(10):1084–1102CrossRef Zhang J, Bargal SA, Lin Z, Brandt J, Shen X, Sclaroff S (2018) Top-down neural attention by excitation backprop. Int J Comput Vis 126(10):1084–1102CrossRef
37.
go back to reference Gao Y, Noble JA (2017, September) Detection and characterization of the fetal heartbeat in free-hand ultrasound sweeps with weakly-supervised two-streams convolutional networks. In: International conference on medical image computing and computer-assisted intervention (pp 305–313). Springer, Cham Gao Y, Noble JA (2017, September) Detection and characterization of the fetal heartbeat in free-hand ultrasound sweeps with weakly-supervised two-streams convolutional networks. In: International conference on medical image computing and computer-assisted intervention (pp 305–313). Springer, Cham
38.
go back to reference He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 770–778) He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 770–778)
39.
40.
go back to reference Irvin J, Rajpurkar P, Ko M, Yu Y, Ciurea-Ilcus S, Chute C, Ng AY (2019, July) Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI conference on artificial intelligence (Vol 33, No 01, pp 590–597) Irvin J, Rajpurkar P, Ko M, Yu Y, Ciurea-Ilcus S, Chute C, Ng AY (2019, July) Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI conference on artificial intelligence (Vol 33, No 01, pp 590–597)
41.
go back to reference Liu Y, Wu YH, Ban Y, Wang H, Cheng MM (2020) Rethinking computer-aided tuberculosis diagnosis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp 2646–2655) Liu Y, Wu YH, Ban Y, Wang H, Cheng MM (2020) Rethinking computer-aided tuberculosis diagnosis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp 2646–2655)
42.
go back to reference LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRef LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRef
43.
go back to reference Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision (pp 618–626) Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision (pp 618–626)
44.
go back to reference Rudin C (2019) Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Mach Intell 1(5):206–215CrossRef Rudin C (2019) Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Mach Intell 1(5):206–215CrossRef
45.
go back to reference Zintgraf LM, Cohen TS, Adel T, Welling M (2017) Visualizing deep neural network decisions: Prediction difference analysis. arXiv:1702.04595 Zintgraf LM, Cohen TS, Adel T, Welling M (2017) Visualizing deep neural network decisions: Prediction difference analysis. arXiv:​1702.​04595
Metadata
Title
MDVA-GAN: multi-domain visual attribution generative adversarial networks
Authors
Muhammad Nawaz
Feras Al-Obeidat
Abdallah Tubaishat
Tehseen Zia
Fahad Maqbool
Alvaro Rocha
Publication date
31-01-2022
Publisher
Springer London
Published in
Neural Computing and Applications / Issue 11/2023
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-022-06969-0

Other articles of this Issue 11/2023

Neural Computing and Applications 11/2023 Go to the issue

Premium Partner