Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Comparing Physical and Immersive VR Prototypes for Evaluation of an Industrial System User Interface Read chapter Read first chapter

2020 | OriginalPaper | Chapter

An Unsupervised Approach for 3D Face Reconstruction from a Single Depth Image

Authors : Peixin Li, Yuru Pei, Yicheng Zhong, Yuke Guo, Gengyu Ma, Meng Liu, Wei Bai, Wenhai Wu, Hongbin Zha

Published in: Advances in Computer Graphics

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In this paper, we propose a convolutional encoder network to learn a mapping function from a noisy depth image to a 3D expressive facial model. We formulate the task as an embedding problem and train the network in an unsupervised manner by exploiting the consistent fitting of the 3D mesh and the depth image. We use the 3DMM-based representation and embed depth images to code vectors concerning facial identities, expressions, and poses. Without semantic textural cues from RGB images, we exploit geometric and contextual constraints in both the depth image and the 3D surface for reliable mapping. We combine the multi-level filtered point cloud pyramid and semantic adaptive weighting for fitting. The proposed system enables the 3D expressive face completion and reconstruction in poor illuminations by leveraging a single noisy depth image. The system realizes a full correspondence between the depth image and the 3D statistical deformable mesh, facilitating landmark location and feature segmentation of depth images.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Illumination Harmonization with Gray Mean Scale
next chapter Fusing IMU Data into SfM for Image-Based 3D Reconstruction
Literature
1.
Amberg, B., Romdhani, S., Vetter, T.: Optimal step nonrigid ICP algorithms for surface registration. In: IEEE CVPR, pp. 1–8 (2007)
2.
Baltrušaitis, T., Robinson, P., Morency, L.P.: 3D constrained local model for rigid and non-rigid facial tracking. In: IEEE CVPR, pp. 2610–2617 (2012)
3.
Bas, A., Huber, P., Smith, W.A., Awais, M., Kittler, J.: 3D morphable models as spatial transformer networks. In: ICCV Workshop on Geometry Meets Deep Learning, pp. 904–912 (2017)
4.
Blanz, V., Basso, C., Poggio, T., Vetter, T.: Reanimating faces in images and video. Comput. Graph. Forum 22, 641–650 (2003)CrossRef
5.
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: SIGGRAPH 1999, pp. 187–194 (1999)
6.
Borghi, G., Venturelli, M., Vezzani, R., Cucchiara, R.: Poseidon: face-from-depth for driver pose estimation. In: IEEE CVPR, pp. 5494–5503 (2017)
7.
Bulat, A., Tzimiropoulos, G.: How far are we from solving the 2D 3D face alignment problem? (and a dataset of 230,000 3D facial landmarks). In: IEEE ICCV, pp. 1021–1030 (2017)
8.
Cao, C., Weng, Y., Zhou, S., Tong, Y., Zhou, K.: FaceWarehouse: a 3D facial expression database for visual computing. IEEE Trans. VCG 20(3), 413–425 (2014)
9.
Chang, F.J., Tran, A.T., Hassner, T., Masi, I., Nevatia, R., Medioni, G.: ExpNet: landmark-free, deep, 3D facial expressions. In: IEEE FG 2018, pp. 122–129 (2018)
10.
Deng, Y., Yang, J., Xu, S., Chen, D., Jia, Y., Tong, X.: Accurate 3D face reconstruction with weakly-supervised learning: from single image to image set. In: IEEE CVPR Workshops (2019)
11.
12.
Fanelli, G., Gall, J., Van Gool, L.: Real time head pose estimation with random regression forests. In: CVPR, pp. 617–624 (2011)
13.
14.
Ghiass, R.S., Arandjelović, O., Laurendeau, D.: Highly accurate and fully automatic head pose estimation from a low quality consumer-level RGB-D sensor. In: Proceedings of the 2nd Workshop on Computational Models of Social Interactions: Human-Computer-Media Communication, pp. 25–34 (2015)
15.
16.
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE CVPR, pp. 770–778 (2016)
17.
Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. In: NIPS, pp. 2017–2025 (2015)
18.
Jin, X., Tan, X.: Face alignment in-the-wild: a survey. Comput. Vis. Image Underst. 162, 1–22 (2017)CrossRef
19.
Kundu, A., Li, Y., Rehg, J.M.: 3D-RCNN: instance-level 3D object reconstruction via render-and-compare. In: IEEE CVPR, pp. 3559–3568 (2018)
20.
Li, S., Ngan, K.N., Paramesran, R., Sheng, L.: Real-time head pose tracking with online face template reconstruction. IEEE Trans. PAMI 38(9), 1922–1928 (2016)CrossRef
21.
Lu, S., Cai, J., Cham, T.J., Pavlovic, V., Ngan, K.N.: A generative model for depth-based robust 3D facial pose tracking. In: IEEE CVPR (2017)
22.
Martin, M., Camp, F.V.D., Stiefelhagen, R.: Real time head model creation and head pose estimation on consumer depth cameras. In: 3DV (2015)
23.
Meyer, G.P., Gupta, S., Frosio, I., Reddy, D., Kautz, J.: Robust model-based 3D head pose estimation. In: ICCV, pp. 3649–3657 (2015)
24.
Morency, L.P.: 3D constrained local model for rigid and non-rigid facial tracking. In: IEEE CVPR (2012)
25.
Padeleris, P., Zabulis, X., Argyros, A.A.: Head pose estimation on depth data based on particle swarm optimization. In: IEEE CVPR Workshops, pp. 42–49 (2012)
26.
Paysan, P., Knothe, R., Amberg, B., Romdhani, S., Vetter, T.: A 3D face model for pose and illumination invariant face recognition. In: IEEE International Conference on Advanced Video and Signal Based Surveillance, pp. 296–301 (2009)
27.
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: IEEE CVPR, pp. 652–660 (2017)
28.
Richardson, E., Sela, M., Kimmel, R.: 3D face reconstruction by learning from synthetic data. In: 3DV, pp. 460–469 (2016)
29.
Richardson, E., Sela, M., Or-El, R., Kimmel, R.: Learning detailed face reconstruction from a single image. In: IEEE CVPR, pp. 5553–5562 (2017)
30.
Shin, D., Fowlkes, C.C., Hoiem, D.: Pixels, voxels, and views: a study of shape representations for single view 3D object shape prediction. In: IEEE CVPR, pp. 3061–3069 (2018)
31.
Tewari, A., et al.: Self-supervised multi-level face model learning for monocular reconstruction at over 250 Hz. In: IEEE CVPR, pp. 2549–2559 (2018)
32.
Tewari, A., et al.: MoFA: model-based deep convolutional face autoencoder for unsupervised monocular reconstruction. In: IEEE ICCV, vol. 2, p. 5 (2017)
33.
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., Nießner, M.: Face2Face: real-time face capture and reenactment of RGB videos. In: IEEE CVPR, pp. 2387–2395 (2016)
34.
Tran, A.T., Hassner, T., Masi, I., Medioni, G.: Regressing robust and discriminative 3D morphable models with a very deep neural network. In: IEEE CVPR, pp. 1493–1502 (2017)
35.
36.
Wang, W., Ceylan, D., Mech, R., Neumann, U.: 3DN: 3D deformation network. In: IEEE CVPR, pp. 1038–1046 (2019)
37.
Weise, T., Bouaziz, S., Li, H., Pauly, M.: Realtime performance-based facial animation. ACM Trans. Graph. 30, 77 (2011)CrossRef
38.
Yan, X., Yang, J., Yumer, E., Guo, Y., Lee, H.: Perspective transformer nets: learning single-view 3D object reconstruction without 3D supervision. In: NIPS, pp. 1696–1704 (2016)
39.
Zhu, X., Liu, X., Lei, Z., Li, S.Z.: Face alignment in full pose range: a 3D total solution. IEEE Trans. PAMI 41(1), 78–92 (2017)CrossRef
Metadata
Title
An Unsupervised Approach for 3D Face Reconstruction from a Single Depth Image
Authors
Peixin Li
Yuru Pei
Yicheng Zhong
Yuke Guo
Gengyu Ma
Meng Liu
Wei Bai
Wenhai Wu
Hongbin Zha
Copyright Year
2020
Book
Advances in Computer Graphics

Print ISBN: 978-3-030-61863-6

Electronic ISBN: 978-3-030-61864-3

Copyright Year: 2020

DOI
https://doi.org/10.1007/978-3-030-61864-3_18

Premium Partner

    Image Credits