Skip to main content
SpringerLink
Log in

HRM graph constrained dictionary learning for face image super-resolution

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Sparse coding based face image Super-Resolution (SR) approaches have received increasing amount of interest recently. However, most of the existing sparse coding based approaches fail to consider the geometrical structure of the face space, as a result, artificial effects on reconstructed High Resolution (HR) face images come into being. In this paper, a novel sparse coding based face image SR method is proposed to reconstruct a HR face image from a Low Resolution (LR) observation. In training stage, it aims to get a more expressive HR-LR dictionary pair for certain input LR patch. The intrinsic geometric structure of training samples is incorporated into the sparse coding procedure for dictionary learning. Unlike the existing SR methods which use the graph constructed in LR Manifold (LRM) as regularization term, the proposed method uses graph constructed in HR Manifold (HRM) as regularization term. In reconstruction stage, K selection mean constrains is used in l 1 convex optimization, aiming at finding an optimal weight for HR face image patch reconstruction. Experimental results on both simulation and real world images suggest that our proposed one achieves better quality when compared with other state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. An L, Bhanu B (2014) Face image super-resolution using 2D CCA. Signal Process 103:184–194

    Article  Google Scholar 

  2. Baker S, Kanade T (2002) Limits on super-resolution and how to break them. IEEE Trans Pattern Analysis Machine Intell 24(9):1167–1183

    Article  Google Scholar 

  3. Belkin M, Niyogi P (2002) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396

    Article  MATH  Google Scholar 

  4. Capel D, Zisserman A (2001) Super- resolution from multiple views using learnt image models, IEEE Int Conf Comput Vision Pattern Recognition, pp.627–634

  5. Chang H, Yeung D, Xiong Y (2004) Super-resolution through neighbor embedding. IEEE Int Conf Comput Vision Pattern Recognition, pp. 275–282

  6. Chang L, Zhou M (2010) Face sketch synthesis via sparse representation, Int Conf Pattern Recognization, pp.2146–2149

  7. Cheng B, Yang J, Yan S, Yun F, Huang TS (2010) Learning with L1-graph for image analysis. IEEE Trans Image Process 19(4):858–866

    Article  MathSciNet  MATH  Google Scholar 

  8. CMU+ MIT Face Database. [Online]. Available: http://vasc.ri.cmu.edu/idb/images/face/frontal images/images.html

  9. Gao W, Cao B (2008) The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE TSMC-A 38(1):149–161

    Google Scholar 

  10. He H, Siu W-C (2011) Single image super-resolution using Gaussian process regression, in CVPR, pp. 449–456

  11. Huang H, He H, Fan X, Zhang J (2010) Super-resolution of human face image using canonical correlation analysis. Pattern Recogn 43(7):2532–2543

    Article  MATH  Google Scholar 

  12. Huang H, Wu N (2011) Fast facial image super-resolution via local linear transformations for resource-limited applications. IEEE Trans Image Process 21(10):1363–1377

    Google Scholar 

  13. Jiang J, Hu R, Han Z, Lu T (2014) Efficient single image super-resolution via graph-constrained least squares regression. Multimedia Tools Appl 72(3):2573–2596

    Article  Google Scholar 

  14. Jiang J, Hu R, Wang Z, Han Z, Jiayi M (2015) Facial Image Hallucination through Coupled-Layer Neighbor Embedding. IEEE Trans Circ Syst Video Technol. doi:10.1109/TCSVT.2015.2433538

    Google Scholar 

  15. Jiang J, Hu R, Wang Z, Han Z (2014) Noise robust face hallucination via locality-constrained representation, IEEE Trans Multimedia, vol. PP, no. 99, pp. 1–1

  16. Jiang J, Hu R, Wang Z, Han Z (2014) Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE Trans Image Process 23(10):4220–4231

    Article  MathSciNet  MATH  Google Scholar 

  17. Jiang J, Ma X, Hu R, Cai Z (2015) Sparse support regression for image super-resolution. IEEE Photonics J 7(5):1–11

    Article  Google Scholar 

  18. Jung C, Jiao L, Liu B, Gong M (2011) Position-patch based face hallucination using convex optimization. IEEE Signal Proc Letters 18(6):367–370

    Article  Google Scholar 

  19. Kim KI, Kwon Y (2010) Single-image super-resolution using sparse regression and natural image prior. IEEE Trans Pattern Anal Mach Intell 32(6):1127–1133

    Article  Google Scholar 

  20. Li X, He H, Wang R, Cheng J (2015) Superpixel-guided nonlocal means for image denoising and super-resolution, Signal Processing

  21. Li X, He H, Wang R, Tao D (2015) Single image superresolution via directional group sparsity and directional Features. IEEE Trans Image Process 24(9):2874–2888

    Article  MathSciNet  Google Scholar 

  22. Li X, He H, Yin Z, Chen F, Cheng J (2015) KPLS-based image super-resolution using clustering and weighted boosting. Neurocomputing 149:940–948

    Article  Google Scholar 

  23. Liu C, Shum HY, Zhang CS (2001) A two-step approach to hallucinating faces: global parametric model and local nonparametric model. IEEE Int Conf Comput Vision Pattern Recognition, pp. 192–198

  24. Lu X, Yan P, Yuan Y (2014) Alternatively constrained dictionary learning for image superresolution. IEEE Trans Cybernetics 44(3):366–377

    Article  Google Scholar 

  25. Ma X, Luong HQ, Philips W, Song H, Cui H (2012) Sparse representation and position prior based face hallucination upon classified over-complete dictionaries. Signal Process 92(9):2066–2074

    Article  Google Scholar 

  26. Ma J, Qiu W, Zhao J, Ma Y, Yuille AL, Zhuowen T (2015) Robust L2E estimation of transformation for non-rigid registration. IEEE Trans Signal Process 63(5):1115–1129

    Article  MathSciNet  Google Scholar 

  27. Ma X, Zhang J, Qi C (2010) Hallucinating face by position-patch. Pattern Recogn 43(1):2224–2236

    Article  Google Scholar 

  28. Ma J, Zhao J, Tian J, Yuille A, Zhuowen T (2014) Robust point matching via vector field consensus. IEEE Trans Image Process 23(4):1706–1721

    Article  MathSciNet  MATH  Google Scholar 

  29. Ma J, Zhao J, Yuille AL (2016) Non-rigid point set registration by preserving global and local structures. IEEE Trans Image Process 25(1):53–64

    Article  MathSciNet  Google Scholar 

  30. Ma J, Zhou H, Zhao J, Gao Y, Jiang J, Tian J (2015) Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Trans Geosci Remote Sens 53(12):6469–6481

    Article  Google Scholar 

  31. Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  32. Su C, Zhuang Y, Huang L et al (2005) Steerable pyramid based face hallucination. Pattern Recogn 38(6):813–824

    Article  Google Scholar 

  33. Sung Won Park, Savvides M (2007) Breaking the limitation of manifold analysis for super-resolution of facial images, IEEE Int Conf Acoustics, Speech Signal Proc, Page(s): 573–576

  34. Timofte R, De V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution, in ICCV, pp. 1920–1927

  35. Wang Z, Hu R, Liang C, Yu Y, Jiang J, Ye M, Chen J, Leng Q. Zero-shot person re-identification via cross-view consistency, IEEE Trans Multimedia, vol. PP, no.99, pp.1–1, doi: 10.1109/TMM.2015.2505083

  36. Wang Z, Bovik AC (2004) Image quality assessment: from error measurement to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  37. Wang Z, Hu R, Jiang J, Han Z, Shao Z (2015) Heteroskedasticity tuned mixed-norm sparse regularization for face hallucination, Multimedia Tools Appl, doi: 10.1007/s11042-015-2996-4

  38. Wang Z, Hu R, Wang S, Jiang J (2014) Face hallucination via weighted adaptive sparse regularization. IEEE Trans Circuits Syst Video Technol 24(5):802–813

    Article  Google Scholar 

  39. Wang X, Tang X (2005) Hallucinating face by eigentransform. IEEE TSMC-C 34(3):425–434

    Google Scholar 

  40. Xiang M, Huansheng S, Xueming Q (2015) Robust framework of single-frame face super-resolution across head pose, facial expression, and illumination variations. IEEE Trans Human-Machine Syst 45(2):238–250

    Article  Google Scholar 

  41. Yang J, Wright J, Huang T, Ma Y (2008) Image super resolution as sparse representation of raw image patches. IEEE Conf Comput Vision Pattern Recognition, pp. 1–8

  42. Zhang W, Cham W-K (2009) Learning-based face hallucination in DCT domain, in Proc. IEEE Int Conf Comput Vision Pattern Recognition (CVPR), pp. 1–8

  43. Zhang W, Cham W-K (2011) Hallucinating face in the DCT domain. IEEE Trans Image Process 20(10):2769–2779

    Article  MathSciNet  MATH  Google Scholar 

  44. Zhang X, Silong P, Jing J (2008) An adaptive learning method for face hallucination using locality preserving projections, 8th IEEE Int Conf Auto Face Gesture Recognition, pp.1–8

  45. Zhu Z, Guo F, Yu H, Chen C (2014) Fast single image super-resolution via self-example learning and sparse representation. IEEE Trans Multimedia 16(8):2178–2190

    Article  Google Scholar 

Download references

Acknowledgments

The research was supported by the National High Technology Research and Development Program of China (863 Program) (2015AA016306, 2013AA014602), the National Natural Science Foundation of China (61231015, 61172173, 61303114, 61501413), the EU FP7 QUICK project under Grant Agreement No. PIRSES-GA-2013-612652, the Internet of Things Development Funding Project of Ministry of industry in 2013 (No.25), the Technology Research Program of Ministry of Public Security (2014JSYJA016), the China Postdoctoral Science Foundation funded project (2013M530350, 2014M562058), and the Specialized Research Fund for the Doctoral Program of Higher Education (20130141120024), the Fundamental Research Funds for the Central Universities (2042014kf0025), Major Science and Technology Innovation Plan of Hubei Province (No. 2013AAA020).

Author information

Authors and Affiliations

  1. National Engineering Research Center on Multimedia Software, School of Computer, Wuhan University, Wuhan, 430072, China

    Kebin Huang, Ruimin Hu & Zhen Han

  2. Collaborative Innovation Center of Geospatial Technology, Wuhan, 430072, China

    Kebin Huang & Ruimin Hu

  3. Department of Digital Media Technology, Huanggang Normal University, Huangzhou, 438000, China

    Kebin Huang & Feng Wang

  4. School of Computer Science, China University of Geosciences, Wuhan, 430074, China

    Junjun Jiang

Authors
  1. Kebin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruimin Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junjun Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhen Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Feng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ruimin Hu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, K., Hu, R., Jiang, J. et al. HRM graph constrained dictionary learning for face image super-resolution. Multimed Tools Appl 76, 3139–3162 (2017). https://doi.org/10.1007/s11042-015-3215-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-3215-z

Keywords

Search

Navigation