Skip to main content
Erschienen in: Machine Vision and Applications 1/2019

06.10.2018 | Original paper

Two-stage local details restoration framework for face hallucination

verfasst von: Di Zhao, Zhenxue Chen, Q. M. Jonathan Wu, Chengyun Liu

Erschienen in: Machine Vision and Applications | Ausgabe 1/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Face hallucination is of great importance in many applications. In this paper, a novel two-stage framework is proposed for hallucinating high-resolution (HR) face image from the given low-resolution (LR) one. In contrast to the existing methods, where the finer details are ignored, our framework pays more attention to the further local details enhancement. In the first stage, the local position-patch-based method with locality constraint is introduced to obtain the initial estimate image. In order to generate more reasonable face image and reduce noise, our method only represents the input LR patches over the similar training patches in the same position. In the second stage, the initial estimate image rather than residual image is directly used as the input to obtain the final HR image via local position-patch-based method. Besides, contextual information of position-patch is taken into consideration to generate more precise details in the second stage. Extensive experiments on the open face database illustrate that the proposed method achieves superior performance in comparison with state-of-the-art methods.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Baker, S., Kanade, T.: Hallucinating faces. In: Proceedings 4th IEEE International Conference on Automatic Face and Gesture Recognition, Grenoble, France, pp. 83–88 (2000) Baker, S., Kanade, T.: Hallucinating faces. In: Proceedings 4th IEEE International Conference on Automatic Face and Gesture Recognition, Grenoble, France, pp. 83–88 (2000)
2.
Zurück zum Zitat Wang, Z., Miao, Z., Wu, Q.M.J., Wan, Y., Tang, Z.: Low-resolution face recognition: a review. Vis. Comput. 30(4), 359–386 (2014)CrossRef Wang, Z., Miao, Z., Wu, Q.M.J., Wan, Y., Tang, Z.: Low-resolution face recognition: a review. Vis. Comput. 30(4), 359–386 (2014)CrossRef
3.
Zurück zum Zitat Liu, C., Shum, H., Freeman, W.: Face hallucination: theory and practice. Int. J. Comput. Vis. 7(1), 115–134 (2007)CrossRef Liu, C., Shum, H., Freeman, W.: Face hallucination: theory and practice. Int. J. Comput. Vis. 7(1), 115–134 (2007)CrossRef
4.
Zurück zum Zitat Liu, C., Shum, H.Y., Zhang, C.S.: A two-step approach to hallucinating faces: global parametric model and local non parametric model. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 192–198 (2001) Liu, C., Shum, H.Y., Zhang, C.S.: A two-step approach to hallucinating faces: global parametric model and local non parametric model. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 192–198 (2001)
5.
Zurück zum Zitat Zhuang, Y.T., Zhang, J., Wu, F.: Hallucinating faces: LPH super-resolution and neighbor reconstruction for residue compensation. Pattern Recognit. 40(40), 3178–3194 (2007)CrossRefMATH Zhuang, Y.T., Zhang, J., Wu, F.: Hallucinating faces: LPH super-resolution and neighbor reconstruction for residue compensation. Pattern Recognit. 40(40), 3178–3194 (2007)CrossRefMATH
6.
Zurück zum Zitat Hui, Z., Liu, W., Lam, K.M.: A novel correspondence-based face-hallucination method. Image Vis. Comput. 60, 171–184 (2017)CrossRef Hui, Z., Liu, W., Lam, K.M.: A novel correspondence-based face-hallucination method. Image Vis. Comput. 60, 171–184 (2017)CrossRef
7.
Zurück zum Zitat Wang, X., Tang, X.: Hallucinating face by eigentransformation. IEEE Trans. Syst. Man Cybern. Part C 35(3), 425–434 (2005)CrossRef Wang, X., Tang, X.: Hallucinating face by eigentransformation. IEEE Trans. Syst. Man Cybern. Part C 35(3), 425–434 (2005)CrossRef
8.
Zurück zum Zitat Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 1, 275–282 (2004) Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 1, 275–282 (2004)
9.
Zurück zum Zitat Wei, F., Yeung, D.: Image hallucination using neighbor embedding over visual primitive manifolds. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–7 (2007) Wei, F., Yeung, D.: Image hallucination using neighbor embedding over visual primitive manifolds. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–7 (2007)
10.
Zurück zum Zitat Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: Proceedings Computer Vision and Pattern Recognition (2008) Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: Proceedings Computer Vision and Pattern Recognition (2008)
11.
Zurück zum Zitat Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)MathSciNetCrossRefMATH Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)MathSciNetCrossRefMATH
12.
Zurück zum Zitat Ma, X., Zhang, J., Qi, C.: Position-based face hallucination method. In: Proc. IEEE Int. Conf. Multimedia and Expo. (ICME), pp. 290–293 (2009) Ma, X., Zhang, J., Qi, C.: Position-based face hallucination method. In: Proc. IEEE Int. Conf. Multimedia and Expo. (ICME), pp. 290–293 (2009)
13.
Zurück zum Zitat Ma, X., Zhang, J., Qi, C.: Hallucinating face by position-patch. Pattern Recognit. 43(6), 3178–3194 (2010)CrossRef Ma, X., Zhang, J., Qi, C.: Hallucinating face by position-patch. Pattern Recognit. 43(6), 3178–3194 (2010)CrossRef
14.
Zurück zum Zitat Jung, C., Jiao, L., Liu, B., Gong, M.: Position-patch based face hallucination using convex optimization. IEEE Signal Process. Lett. 18(6), 367–370 (2011)CrossRef Jung, C., Jiao, L., Liu, B., Gong, M.: Position-patch based face hallucination using convex optimization. IEEE Signal Process. Lett. 18(6), 367–370 (2011)CrossRef
15.
Zurück zum Zitat Jiang, J., Hu, R., Han, Z., Lu, T., Huang, K.: Position-patch based face hallucination via locality-constrained representation. In: Proc. IEEE Int. Conf. Multimedia and Expo (ICME), pp. 212–217 (2012) Jiang, J., Hu, R., Han, Z., Lu, T., Huang, K.: Position-patch based face hallucination via locality-constrained representation. In: Proc. IEEE Int. Conf. Multimedia and Expo (ICME), pp. 212–217 (2012)
16.
Zurück zum Zitat Jiang, J., Hu, R., Wang, Z., Han, Z.: Noise robust face hallucination via locality-constrained representation. IEEE Trans. Multimed. 16(5), 1268–1281 (2014)CrossRef Jiang, J., Hu, R., Wang, Z., Han, Z.: Noise robust face hallucination via locality-constrained representation. IEEE Trans. Multimed. 16(5), 1268–1281 (2014)CrossRef
17.
Zurück zum Zitat Chen, H-Y., Chien, S-Y.: Eigen-patch: position-patch based face hallucination using eigen transformation. In: IEEE Int. Conf. on Multimedia and Expo, pp. 1–6 (2014) Chen, H-Y., Chien, S-Y.: Eigen-patch: position-patch based face hallucination using eigen transformation. In: IEEE Int. Conf. on Multimedia and Expo, pp. 1–6 (2014)
18.
Zurück zum Zitat Gao, G., Yang, J., Lai, Z., Huang, P.: Nuclear norm regularized coding with local position-patch and nonlocal similarity for face hallucination. Digital Signal Process. 64, 107–120 (2017)MathSciNetCrossRef Gao, G., Yang, J., Lai, Z., Huang, P.: Nuclear norm regularized coding with local position-patch and nonlocal similarity for face hallucination. Digital Signal Process. 64, 107–120 (2017)MathSciNetCrossRef
19.
Zurück zum Zitat Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)CrossRef Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)CrossRef
20.
Zurück zum Zitat Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016) Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
21.
Zurück zum Zitat Yu, K., Zhang, T., Gong, Y.: Nonlinear learning using local coordinate coding. In: Proc. Advances in Neural Information Processing Systems (NIPS), pp. 2223–2231 (2009) Yu, K., Zhang, T., Gong, Y.: Nonlinear learning using local coordinate coding. In: Proc. Advances in Neural Information Processing Systems (NIPS), pp. 2223–2231 (2009)
22.
Zurück zum Zitat Wang, J., Lu, C.Y., Wang, M., Li, P.P., Yan, S.C., Hu, X.G.: Robust face recognition via adaptive sparse representation. IEEE Trans. Cybern. 44(12), 2368–2378 (2014)CrossRef Wang, J., Lu, C.Y., Wang, M., Li, P.P., Yan, S.C., Hu, X.G.: Robust face recognition via adaptive sparse representation. IEEE Trans. Cybern. 44(12), 2368–2378 (2014)CrossRef
23.
Zurück zum Zitat Wang, Z., Bovik, A.C.: A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)CrossRef Wang, Z., Bovik, A.C.: A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)CrossRef
24.
Zurück zum Zitat Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRef Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRef
25.
Zurück zum Zitat Jiang, J., Hu, R., Wang, Z., Han, Z.: Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE TIP 23(10), 4220–4231 (2014)MathSciNetMATH Jiang, J., Hu, R., Wang, Z., Han, Z.: Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE TIP 23(10), 4220–4231 (2014)MathSciNetMATH
26.
Zurück zum Zitat Jiang, J., Ma, J., Chen, C., Jiang, X., Wang, Z.: Noise robust face image super-resolution through smooth sparse representation. In: CYB, vol. PP, no. 99, pp. 1–12 (2016) Jiang, J., Ma, J., Chen, C., Jiang, X., Wang, Z.: Noise robust face image super-resolution through smooth sparse representation. In: CYB, vol. PP, no. 99, pp. 1–12 (2016)
27.
Zurück zum Zitat Jiang, J., Chen, C., Huang, K., Cai, Z., Hu, R.: Noise robust position-patch based face super-resolution via Tikhonov regularized neighbor representation. Inf. Sci. 367–368, 354–372 (2016)CrossRef Jiang, J., Chen, C., Huang, K., Cai, Z., Hu, R.: Noise robust position-patch based face super-resolution via Tikhonov regularized neighbor representation. Inf. Sci. 367–368, 354–372 (2016)CrossRef
Metadaten
Titel
Two-stage local details restoration framework for face hallucination
verfasst von
Di Zhao
Zhenxue Chen
Q. M. Jonathan Wu
Chengyun Liu
Publikationsdatum
06.10.2018
Verlag
Springer Berlin Heidelberg
Erschienen in
Machine Vision and Applications / Ausgabe 1/2019
Print ISSN: 0932-8092
Elektronische ISSN: 1432-1769
DOI
https://doi.org/10.1007/s00138-018-0983-2

Weitere Artikel der Ausgabe 1/2019

Machine Vision and Applications 1/2019 Zur Ausgabe