Skip to main content
Top
Published in: Cluster Computing 4/2019

17-03-2018

Selfie image super-resolution using an implicit prior learned from self-examples

Authors: W. Jino Hans, N. Venkateswaran

Published in: Cluster Computing | Special Issue 4/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Selfie image is a self-captured photograph of oneself using the front camera of a smartphone. The use of selfie images in social media is enormous and growing day by day. Most of the modern day smartphones are equipped with two cameras, viz. a high-resolution primary camera in the rear and a low-resolution secondary camera in the front of the smartphone. Typically selfie images are captured using low-resolution front camera and hence the spatial resolution of these images will be very much limited. In this paper, we propose an efficient approach to improve the spatial resolution of selfie images by exploiting the self-similarity across various scales of selfie images using a self-example based super-resolution algorithm. The super-resolution algorithm is formulated by learning a local regression from in-place self-example patches extracted from various scales of the given selfie image. In-place matching ensures that image patches extracted from different scales are spatially close and hence will have the same high-frequency details. A local regression is learned by approximating Taylors series and it serves as an efficient implicit prior to learn the relation between low-resolution and its corresponding high-resolution patch. The algorithm is evaluated both qualitatively and quantitatively and the results validates its efficiency.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20, 21–36 (2003)CrossRef Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20, 21–36 (2003)CrossRef
2.
go back to reference Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based super-resolution. IEEE Comput. Graph. Appl. 22, 56–65 (2002)CrossRef Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based super-resolution. IEEE Comput. Graph. Appl. 22, 56–65 (2002)CrossRef
3.
go back to reference Yu, J., Gao, X., Tao, D., Li, X., Zhang, K.: A unified learning framework for single image super-resolution. IEEE Trans. Neural Netw. Learn. Syst. 25(4), 780–792 (2014)CrossRef Yu, J., Gao, X., Tao, D., Li, X., Zhang, K.: A unified learning framework for single image super-resolution. IEEE Trans. Neural Netw. Learn. Syst. 25(4), 780–792 (2014)CrossRef
4.
go back to reference Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: Computer Vision, 2009 IEEE 12th International Conference on, pp. 349–356 (2009) Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: Computer Vision, 2009 IEEE 12th International Conference on, pp. 349–356 (2009)
5.
go back to reference Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Fast and robust multi-frame super-resolution. IEEE Trans. Image Process. 13, 1327–1344 (2003)CrossRef Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Fast and robust multi-frame super-resolution. IEEE Trans. Image Process. 13, 1327–1344 (2003)CrossRef
6.
go back to reference Gao, X., Zhang, K., Tao, D., Li, X.: Joint learning for single-image super-resolution via a coupled constraint. IEEE Trans. Image Process. 21(2), 469–480 (2012)MathSciNetCrossRef Gao, X., Zhang, K., Tao, D., Li, X.: Joint learning for single-image super-resolution via a coupled constraint. IEEE Trans. Image Process. 21(2), 469–480 (2012)MathSciNetCrossRef
7.
go back to reference Wang, S., Yue, B., Liang, X., Jiao, L.: How does the low-rank matrix decomposition help internal and external learnings for super-resolution. IEEE Trans. Image Process. 27(3), 1086–1099 (2018)MathSciNetCrossRef Wang, S., Yue, B., Liang, X., Jiao, L.: How does the low-rank matrix decomposition help internal and external learnings for super-resolution. IEEE Trans. Image Process. 27(3), 1086–1099 (2018)MathSciNetCrossRef
8.
go back to reference Tang, Y., Li, L., Li, X.: Learning similarity with multikernel method. IEEE Trans. Syst. Man Cybern. 41(1), 131–138 (2011)CrossRef Tang, Y., Li, L., Li, X.: Learning similarity with multikernel method. IEEE Trans. Syst. Man Cybern. 41(1), 131–138 (2011)CrossRef
9.
go back to reference Aharon, M., Elad, M., Bruckstein, A.: k -svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54, 4311–4322 (2006)CrossRef Aharon, M., Elad, M., Bruckstein, A.: k -svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54, 4311–4322 (2006)CrossRef
10.
go back to reference Mairal, J., Ponce, J., Sapiro, G., Zisserman, A., Bach, F.R.: Supervised dictionary learning. In: Advances in Neural Information Processing Systems 21, pp. 1033–1040. Curran Associates, Inc., Red Hook (2009) Mairal, J., Ponce, J., Sapiro, G., Zisserman, A., Bach, F.R.: Supervised dictionary learning. In: Advances in Neural Information Processing Systems 21, pp. 1033–1040. Curran Associates, Inc., Red Hook (2009)
11.
go back to reference Bertelli, L., Yu, T., Vu, D., Gokturk, B.: Kernelized structural SVM learning for supervised object segmentation. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp. 2153–2160 (2011) Bertelli, L., Yu, T., Vu, D., Gokturk, B.: Kernelized structural SVM learning for supervised object segmentation. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp. 2153–2160 (2011)
12.
go back to reference Ni, K., Nguyen, T.: Image superresolution using support vector regression. IEEE Trans. Image Process. 16, 1596–1610 (2007)MathSciNetCrossRef Ni, K., Nguyen, T.: Image superresolution using support vector regression. IEEE Trans. Image Process. 16, 1596–1610 (2007)MathSciNetCrossRef
13.
go back to reference Chang, H., Yeung, D.-Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 1, p. 1 (2004) Chang, H., Yeung, D.-Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 1, p. 1 (2004)
14.
go back to reference Tang, Y., Yuan, Y., Yan, P., Li, L., Pan, X., Li, L.: Single-image super-resolution based on semi-supervised learning. In: 2011 First Asian Conference on Pattern Recognition (ACPR), pp. 52–56 (2011) Tang, Y., Yuan, Y., Yan, P., Li, L., Pan, X., Li, L.: Single-image super-resolution based on semi-supervised learning. In: 2011 First Asian Conference on Pattern Recognition (ACPR), pp. 52–56 (2011)
15.
go back to reference Gao, X., Zhang, K., Tao, D., Li, X.: Image super-resolution with sparse neighbor embedding. IEEE Trans. Image Process. 21, 3194–3205 (2012)MathSciNetCrossRef Gao, X., Zhang, K., Tao, D., Li, X.: Image super-resolution with sparse neighbor embedding. IEEE Trans. Image Process. 21, 3194–3205 (2012)MathSciNetCrossRef
16.
go back to reference Yang, J., Lin, Z., Cohen, S.: Fast image super-resolution based on in-place example regression. In: Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pp. 1059–1066 (2013) Yang, J., Lin, Z., Cohen, S.: Fast image super-resolution based on in-place example regression. In: Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pp. 1059–1066 (2013)
17.
go back to reference Hu, J., Luo, Y.: Single-image superresolution based on local regression and nonlocal self-similarity. J. Electron. Imaging 23, 033014 (2014)CrossRef Hu, J., Luo, Y.: Single-image superresolution based on local regression and nonlocal self-similarity. J. Electron. Imaging 23, 033014 (2014)CrossRef
18.
go back to reference Suetake, N., Sakano, M., Uchino, E.: Image super-resolution based on local self-similarity. Opt. Rev. 15(1), 26–30 (2008)CrossRef Suetake, N., Sakano, M., Uchino, E.: Image super-resolution based on local self-similarity. Opt. Rev. 15(1), 26–30 (2008)CrossRef
19.
go back to reference Zhu, Z., Guo, F., Yu, H., Chen, C.: Fast single image super-resolution via self-example learning and sparse representation. IEEE Trans. Multim. 16, 2178–2190 (2014)CrossRef Zhu, Z., Guo, F., Yu, H., Chen, C.: Fast single image super-resolution via self-example learning and sparse representation. IEEE Trans. Multim. 16, 2178–2190 (2014)CrossRef
20.
go back to reference Freedman, G., Fattal, R.: Image and video upscaling from local self-examples. ACM Trans. Graph. 30, 12:1–12:11 (2011)CrossRef Freedman, G., Fattal, R.: Image and video upscaling from local self-examples. ACM Trans. Graph. 30, 12:1–12:11 (2011)CrossRef
21.
go back to reference Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, pp. 60–65. IEEE Computer Society (2005) Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, pp. 60–65. IEEE Computer Society (2005)
22.
go back to reference Zhang, K., Gao, X., Tao, D., Li, X.: Single image super-resolution with multiscale similarity learning. IEEE Trans. Neural Netw. Learn. Syst. 24(10), 1648–1659 (2013)CrossRef Zhang, K., Gao, X., Tao, D., Li, X.: Single image super-resolution with multiscale similarity learning. IEEE Trans. Neural Netw. Learn. Syst. 24(10), 1648–1659 (2013)CrossRef
23.
go back to reference Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. Trans. Image Proc. 19, 2861–2873 (2010)MathSciNetCrossRef Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. Trans. Image Proc. 19, 2861–2873 (2010)MathSciNetCrossRef
24.
go back to reference Wang, Z., Bovik, A.C.: Mean squared error: love it or leave it? A new look at signal fidelity measures. IEEE Signal Process. Mag. 26(1), 98–117 (2009)CrossRef Wang, Z., Bovik, A.C.: Mean squared error: love it or leave it? A new look at signal fidelity measures. IEEE Signal Process. Mag. 26(1), 98–117 (2009)CrossRef
25.
go back to reference Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRef Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRef
26.
go back to reference Leisti, T., Radun, J., Virtanen, T., Halonen, R., Nyman, G.: Subjective experience of image quality: attributes, definitions, and decision making of subjective image quality. In: Proc. SPIE, vol. 7242, p. 72420D (2009) Leisti, T., Radun, J., Virtanen, T., Halonen, R., Nyman, G.: Subjective experience of image quality: attributes, definitions, and decision making of subjective image quality. In: Proc. SPIE, vol. 7242, p. 72420D (2009)
27.
go back to reference Dong, W., Zhang, L., Lukac, R., Shi, G.: Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Trans. Image Process. 22(4), 1382–1394 (2013)MathSciNetCrossRef Dong, W., Zhang, L., Lukac, R., Shi, G.: Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Trans. Image Process. 22(4), 1382–1394 (2013)MathSciNetCrossRef
Metadata
Title
Selfie image super-resolution using an implicit prior learned from self-examples
Authors
W. Jino Hans
N. Venkateswaran
Publication date
17-03-2018
Publisher
Springer US
Published in
Cluster Computing / Issue Special Issue 4/2019
Print ISSN: 1386-7857
Electronic ISSN: 1573-7543
DOI
https://doi.org/10.1007/s10586-018-2466-7

Other articles of this Special Issue 4/2019

Cluster Computing 4/2019 Go to the issue

Premium Partner