Skip to main content
Top

2020 | OriginalPaper | Chapter

Overlap Training to Mitigate Inconsistencies Caused by Image Tiling in CNNs

Authors : Yu An, Qing Ye, Jiulin Guo, Ruihai Dong

Published in: Artificial Intelligence XXXVII

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This paper focuses on the problem of inconsistent predictions of modern convolutional neural networks (CNN) at patch (i.e. sub-image) boundaries. Limited by the graphics processing unit (GPU) resources, image tiling and stitching countermeasure have been applied for most megapixel images, that is, cutting images into overlapping tiles as CNN input, and then stitching CNN outputs together. However, we found that stitched (i.e. recovered) predictions have discontinuous grid-like noise. We propose a simple yet efficient overlap training framework to mitigate the inconsistent prediction at patch boundaries without changing the model architecture while improving the stability, robustness of the model. We have applied our solution to various CNNs (such as U-Net, DeepLab, RCF) and tested them on two real-world datasets. Extensive experiments suggest that the new framework is sufficient in reducing inconsistency and outperform these countermeasures. The source code and coloured figures are made publicly available online at: https://​github.​com/​anyuzoey/​Overlap-Training.​git.

To get access to this content you need the following product:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
3.
go back to reference Azulay, A., Weiss, Y.: Why do deep convolutional networks generalize so poorly to small image transformations? J. Mach. Learn. Res. 20, 1–25 (2019)MathSciNetMATH Azulay, A., Weiss, Y.: Why do deep convolutional networks generalize so poorly to small image transformations? J. Mach. Learn. Res. 20, 1–25 (2019)MathSciNetMATH
4.
go back to reference Buslaev, A., Parinov, A., Khvedchenya, E., Iglovikov, V.I., Kalinin, A.A.: Albumentations: fast and flexible image augmentations (2018) Buslaev, A., Parinov, A., Khvedchenya, E., Iglovikov, V.I., Kalinin, A.A.: Albumentations: fast and flexible image augmentations (2018)
5.
go back to reference Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected CRFs (2014) Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected CRFs (2014)
6.
go back to reference Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs (2016) Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs (2016)
7.
go back to reference Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation (2017) Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation (2017)
10.
go back to reference Cohen, T.S., Welling, M.: Group equivariant convolutional networks (2016) Cohen, T.S., Welling, M.: Group equivariant convolutional networks (2016)
12.
go back to reference He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015) He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015)
13.
go back to reference Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications (2017) Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications (2017)
14.
go back to reference Huang, B., Reichman, D., Collins, L.M., Bradbury, K., Malof, J.M.: Tiling and stitching segmentation output for remote sensing: basic challenges and recommendations (2018) Huang, B., Reichman, D., Collins, L.M., Bradbury, K., Malof, J.M.: Tiling and stitching segmentation output for remote sensing: basic challenges and recommendations (2018)
15.
go back to reference Islam, M.A., Jia, S., Bruce, N.D.B.: How much position information do convolutional neural networks encode? (2020) Islam, M.A., Jia, S., Bruce, N.D.B.: How much position information do convolutional neural networks encode? (2020)
16.
go back to reference Kauderer-Abrams, E.: Quantifying translation-invariance in convolutional neural networks (2017) Kauderer-Abrams, E.: Quantifying translation-invariance in convolutional neural networks (2017)
17.
go back to reference Kayhan, O.S., van Gemert, J.C.: On translation invariance in CNNs: convolutional layers can exploit absolute spatial location (2020) Kayhan, O.S., van Gemert, J.C.: On translation invariance in CNNs: convolutional layers can exploit absolute spatial location (2020)
18.
go back to reference Kondor, R., Trivedi, S.: On the generalization of equivariance and convolution in neural networks to the action of compact groups (2018) Kondor, R., Trivedi, S.: On the generalization of equivariance and convolution in neural networks to the action of compact groups (2018)
19.
go back to reference Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, vol. 1, pp. 1097–1105. NIPS-12, Curran Associates Inc., Red Hook, NY, USA (2012) Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, vol. 1, pp. 1097–1105. NIPS-12, Curran Associates Inc., Red Hook, NY, USA (2012)
20.
go back to reference Liu, Y., Cheng, M.M., Hu, X., Wang, K., Bai, X.: Richer convolutional features for edge detection (2016) Liu, Y., Cheng, M.M., Hu, X., Wang, K., Bai, X.: Richer convolutional features for edge detection (2016)
21.
go back to reference Luo, Z., Zhang, Y., Zhou, L., Zhang, B., Luo, J., Wu, H.: Micro-vessel image segmentation based on the AD-UNet model. IEEE Access 7, 143402–143411 (2019)CrossRef Luo, Z., Zhang, Y., Zhou, L., Zhang, B., Luo, J., Wu, H.: Micro-vessel image segmentation based on the AD-UNet model. IEEE Access 7, 143402–143411 (2019)CrossRef
24.
go back to reference Roth, H.R., et al.: An application of cascaded 3D fully convolutional networks for medical image segmentation (2018) Roth, H.R., et al.: An application of cascaded 3D fully convolutional networks for medical image segmentation (2018)
25.
go back to reference Saito, S., Yamashita, T., Aoki, Y.: Multiple object extraction from aerial imagery with convolutional neural networks. Electron. Imaging 2016(10), 1–9 (2016)CrossRef Saito, S., Yamashita, T., Aoki, Y.: Multiple object extraction from aerial imagery with convolutional neural networks. Electron. Imaging 2016(10), 1–9 (2016)CrossRef
26.
go back to reference Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks (2018) Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks (2018)
28.
go back to reference Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014) Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014)
29.
go back to reference Siu, K., Stuart, D.M., Mahmoud, M., Moshovos, A.: Memory requirements for convolutional neural network hardware accelerators. In: 2018 IEEE International Symposium on Workload Characterization (IISWC), pp. 111–121. IEEE, Raleigh, NC (2018) Siu, K., Stuart, D.M., Mahmoud, M., Moshovos, A.: Memory requirements for convolutional neural network hardware accelerators. In: 2018 IEEE International Symposium on Workload Characterization (IISWC), pp. 111–121. IEEE, Raleigh, NC (2018)
31.
go back to reference Tan, M., Le, Q.V.: EfficientNet: rethinking model scaling for convolutional neural networks (2019) Tan, M., Le, Q.V.: EfficientNet: rethinking model scaling for convolutional neural networks (2019)
32.
go back to reference Xie, S., Tu, Z.: Holistically-nested edge detection (2015) Xie, S., Tu, Z.: Holistically-nested edge detection (2015)
Metadata
Title
Overlap Training to Mitigate Inconsistencies Caused by Image Tiling in CNNs
Authors
Yu An
Qing Ye
Jiulin Guo
Ruihai Dong
Copyright Year
2020
DOI
https://doi.org/10.1007/978-3-030-63799-6_3

Premium Partner