Skip to main content
Top

Hint

Swipe to navigate through the chapters of this book

2018 | OriginalPaper | Chapter

CocoNet: A Deep Neural Network for Mapping Pixel Coordinates to Color Values

Authors : Paul Andrei Bricman, Radu Tudor Ionescu

Published in: Neural Information Processing

Publisher: Springer International Publishing

share
SHARE

Abstract

We propose a deep neural network for mapping the 2D pixel coordinates in an image to the corresponding RGB color values. The neural network is termed CocoNet, i.e. coordinates-to-color network. During the training process, the neural network learns to encode the input image within its layers, i.e. it learns a continuous function that approximates the discrete RGB values sampled over the discrete 2D pixel locations. At test time, given a 2D pixel coordinate, the neural network will output the RGB values of the corresponding pixel. By considering every 2D pixel location, the network can actually reconstruct the entire learned image. We note that we have to train an individual neural network for each input image, i.e. one network encodes a single image. Our neural image encoding approach has various low-level image processing applications ranging from image denoising to image resampling and image completion. Our code is available at https://​github.​com/​paubric/​python-fuse-coconet.
Literature
1.
go back to reference Baig, M.H., Koltun, V., Torresani, L.: Learning to inpaint for image compression. In: Proceedings of NIPS, pp. 1246–1255 (2017) Baig, M.H., Koltun, V., Torresani, L.: Learning to inpaint for image compression. In: Proceedings of NIPS, pp. 1246–1255 (2017)
2.
go back to reference Ballé, J., Laparra, V., Simoncelli, E.P.: End-to-end optimized image compression. In: Proceedings of ICLR (2017) Ballé, J., Laparra, V., Simoncelli, E.P.: End-to-end optimized image compression. In: Proceedings of ICLR (2017)
3.
go back to reference Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of BMVC, pp. 135.1–135.10 (2012) Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of BMVC, pp. 135.1–135.10 (2012)
4.
go back to reference Cavigelli, L., Hager, P., Benini, L.: CAS-CNN: A deep convolutional neural network for image compression artifact suppression. In: Proceedings of IJCNN, pp. 752–759 (2017) Cavigelli, L., Hager, P., Benini, L.: CAS-CNN: A deep convolutional neural network for image compression artifact suppression. In: Proceedings of IJCNN, pp. 752–759 (2017)
6.
go back to reference Dumas, T., Roumy, A., Guillemot, C.: Image compression with stochastic winner-take-all auto-encoder. In: Proceedings of ICASSP, pp. 1512–1516 (2017) Dumas, T., Roumy, A., Guillemot, C.: Image compression with stochastic winner-take-all auto-encoder. In: Proceedings of ICASSP, pp. 1512–1516 (2017)
7.
go back to reference Girshick, R.: Fast R-CNN. In: Proceedings of ICCV, pp. 1440–1448 (2015) Girshick, R.: Fast R-CNN. In: Proceedings of ICCV, pp. 1440–1448 (2015)
10.
go back to reference He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of CVPR, pp. 770–778, June 2016 He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of CVPR, pp. 770–778, June 2016
11.
go back to reference Huang, X., Li, Y., Poursaeed, O., Hopcroft, J., Belongie, S.: Stacked generative adversarial networks. In: Proceedings of CVPR, pp. 5077–5086 (2017) Huang, X., Li, Y., Poursaeed, O., Hopcroft, J., Belongie, S.: Stacked generative adversarial networks. In: Proceedings of CVPR, pp. 5077–5086 (2017)
12.
go back to reference Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. 36(4), 107 (2017) CrossRef Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. 36(4), 107 (2017) CrossRef
13.
go back to reference Ionescu, R.T., Alexe, B., Leordeanu, M., Popescu, M., Papadopoulos, D., Ferrari, V.: How hard can it be? Estimating the difficulty of visual search in an image. In: Proceedings of CVPR, pp. 2157–2166, June 2016 Ionescu, R.T., Alexe, B., Leordeanu, M., Popescu, M., Papadopoulos, D., Ferrari, V.: How hard can it be? Estimating the difficulty of visual search in an image. In: Proceedings of CVPR, pp. 2157–2166, June 2016
14.
go back to reference Kim, J., Kwon Lee, J., Mu Lee, K.: Deeply-recursive convolutional network for image super-resolution. In: Proceedings of CVPR, pp. 1637–1645 (2016) Kim, J., Kwon Lee, J., Mu Lee, K.: Deeply-recursive convolutional network for image super-resolution. In: Proceedings of CVPR, pp. 1637–1645 (2016)
15.
go back to reference Krizhevsky, A.: Learning Multiple Layers of Features from Tiny Images. Technical report, University of Toronto (2009) Krizhevsky, A.: Learning Multiple Layers of Features from Tiny Images. Technical report, University of Toronto (2009)
16.
go back to reference Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Bartlett, P., Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Proceedings of NIPS, pp. 1106–1114 (2012) Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Bartlett, P., Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Proceedings of NIPS, pp. 1106–1114 (2012)
17.
go back to reference LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998) CrossRef LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998) CrossRef
18.
go back to reference Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of CVPR, pp. 4681–4690 (2016) Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of CVPR, pp. 4681–4690 (2016)
19.
go back to reference Liu, W., et al.: SSD: single shot MultiBox detector. In: Proceedings of ECCV, pp. 21–37 (2016) CrossRef Liu, W., et al.: SSD: single shot MultiBox detector. In: Proceedings of ECCV, pp. 21–37 (2016) CrossRef
20.
go back to reference Mao, X., Shen, C., Yang, Y.B.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Proceedings of NIPS, pp. 2802–2810 (2016) Mao, X., Shen, C., Yang, Y.B.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Proceedings of NIPS, pp. 2802–2810 (2016)
21.
go back to reference Minnen, D., et al.: Spatially adaptive image compression using a tiled deep network. In: Proceedings of ICIP, pp. 2796–2800 (2017) Minnen, D., et al.: Spatially adaptive image compression using a tiled deep network. In: Proceedings of ICIP, pp. 2796–2800 (2017)
23.
go back to reference Parkhi, O.M., Vedaldi, A., Zisserman, A., et al.: Deep face recognition. In: Proceedings of BMVC, pp. 6–17 (2015) Parkhi, O.M., Vedaldi, A., Zisserman, A., et al.: Deep face recognition. In: Proceedings of BMVC, pp. 6–17 (2015)
24.
go back to reference Prakash, A., Moran, N., Garber, S., DiLillo, A., Storer, J.: Semantic perceptual image compression using deep convolution networks. In: Proceedings of DCC, pp. 250–259 (2017) Prakash, A., Moran, N., Garber, S., DiLillo, A., Storer, J.: Semantic perceptual image compression using deep convolution networks. In: Proceedings of DCC, pp. 250–259 (2017)
25.
go back to reference Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of CVPR, pp. 779–788 (2016) Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of CVPR, pp. 779–788 (2016)
26.
go back to reference Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of NIPS, pp. 91–99 (2015) Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of NIPS, pp. 91–99 (2015)
27.
go back to reference Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015) MathSciNetCrossRef Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015) MathSciNetCrossRef
28.
go back to reference Sajjadi, M.S., Schölkopf, B., Hirsch, M.: Enhancenet: single image super-resolution through automated texture synthesis. In: Proceedings of ICCV, pp. 4501–4510 (2017) Sajjadi, M.S., Schölkopf, B., Hirsch, M.: Enhancenet: single image super-resolution through automated texture synthesis. In: Proceedings of ICCV, pp. 4501–4510 (2017)
29.
go back to reference Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of CVPR, pp. 1874–1883 (2016) Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of CVPR, pp. 1874–1883 (2016)
30.
go back to reference Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of ICLR (2014) Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of ICLR (2014)
31.
go back to reference Soviany, P., Ionescu, R.T.: Optimizing the trade-off between single-stage and two-stage deep object detectors using image difficulty prediction. In: Proceedings of SYNASC, pp. 1–6 (2018) Soviany, P., Ionescu, R.T.: Optimizing the trade-off between single-stage and two-stage deep object detectors using image difficulty prediction. In: Proceedings of SYNASC, pp. 1–6 (2018)
32.
go back to reference Sukhbaatar, S., Weston, J., Fergus, R., et al.: End-to-end memory networks. In: Proceedings of CVPR, pp. 2440–2448 (2015) Sukhbaatar, S., Weston, J., Fergus, R., et al.: End-to-end memory networks. In: Proceedings of CVPR, pp. 2440–2448 (2015)
33.
go back to reference Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of CVPR, pp. 1–9, June 2015 Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of CVPR, pp. 1–9, June 2015
34.
go back to reference Tang, Y., Yan, P., Yuan, Y., Li, X.: Single-image super-resolution via local learning. Int. J. Mach. Learn. Cybern. 2(1), 15–23 (2011) CrossRef Tang, Y., Yan, P., Yuan, Y., Li, X.: Single-image super-resolution via local learning. Int. J. Mach. Learn. Cybern. 2(1), 15–23 (2011) CrossRef
35.
go back to reference Toderici, G., et al.: Full resolution image compression with recurrent neural networks. In: Proceedings of CVPR, pp. 5306–5314 (2017) Toderici, G., et al.: Full resolution image compression with recurrent neural networks. In: Proceedings of CVPR, pp. 5306–5314 (2017)
36.
go back to reference Van Oord, A., Kalchbrenner, N., Kavukcuoglu, K.: Pixel recurrent neural networks. In: Proceedings of ICML, pp. 1747–1756 (2016) Van Oord, A., Kalchbrenner, N., Kavukcuoglu, K.: Pixel recurrent neural networks. In: Proceedings of ICML, pp. 1747–1756 (2016)
37.
go back to reference Wang, Z., Bovik, A.C.: Mean squared error: Love it or leave it? A new look at signal fidelity measures. Signal Process. Mag. 26(1), 98–117 (2009) CrossRef Wang, Z., Bovik, A.C.: Mean squared error: Love it or leave it? A new look at signal fidelity measures. Signal Process. Mag. 26(1), 98–117 (2009) CrossRef
38.
go back to reference Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004) CrossRef Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004) CrossRef
40.
go back to reference Yamanaka, J., Kuwashima, S., Kurita, T.: Fast and accurate image super resolution by deep CNN with skip connection and network in network. In: Proceedings of ICONIP, pp. 217–225 (2017) CrossRef Yamanaka, J., Kuwashima, S., Kurita, T.: Fast and accurate image super resolution by deep CNN with skip connection and network in network. In: Proceedings of ICONIP, pp. 217–225 (2017) CrossRef
41.
go back to reference Yang, C., Lu, X., Lin, Z., Shechtman, E., Wang, O., Li, H.: High-resolution image inpainting using multi-scale neural patch synthesis. In: Proceedings of CVPR, pp. 6721–6729 (2017) Yang, C., Lu, X., Lin, Z., Shechtman, E., Wang, O., Li, H.: High-resolution image inpainting using multi-scale neural patch synthesis. In: Proceedings of CVPR, pp. 6721–6729 (2017)
42.
go back to reference Zhao, G., Liu, J., Jiang, J., Wang, W.: A deep cascade of neural networks for image inpainting, deblurring and denoising. Multimed. Tools Appl. 77(22), 29589–29604 (2018) CrossRef Zhao, G., Liu, J., Jiang, J., Wang, W.: A deep cascade of neural networks for image inpainting, deblurring and denoising. Multimed. Tools Appl. 77(22), 29589–29604 (2018) CrossRef
Metadata
Title
CocoNet: A Deep Neural Network for Mapping Pixel Coordinates to Color Values
Authors
Paul Andrei Bricman
Radu Tudor Ionescu
Copyright Year
2018
DOI
https://doi.org/10.1007/978-3-030-04179-3_6

Premium Partner