Skip to main content
Top

01-06-2015 | 3DR Express

A Low-cost System for Generating Near-realistic Virtual Actors

Authors: Mahmoud Afifi, Khaled F. Hussain, Hosny M. Ibrahim, Nagwa M. Omar

Published in: 3D Research | Issue 2/2015

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Generating virtual actors is one of the most challenging fields in computer graphics. The reconstruction of a realistic virtual actor has been paid attention by the academic research and the film industry to generate human-like virtual actors. Many movies were acted by human-like virtual actors, where the audience cannot distinguish between real and virtual actors. The synthesis of realistic virtual actors is considered a complex process. Many techniques are used to generate a realistic virtual actor; however they usually require expensive hardware equipment. In this paper, a low-cost system that generates near-realistic virtual actors is presented. The facial features of the real actor are blended with a virtual head that is attached to the actor’s body. Comparing with other techniques that generate virtual actors, the proposed system is considered a low-cost system that requires only one camera that records the scene without using any expensive hardware equipment. The results of our system show that the system generates good near-realistic virtual actors that can be used on many applications.

Graphical Abstract

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Literature
1.
go back to reference Magnenat-Thalmann, N., & Thalmann, D. (2005). Handbook of virtual humans. Hoboken: Wiley. Magnenat-Thalmann, N., & Thalmann, D. (2005). Handbook of virtual humans. Hoboken: Wiley.
2.
go back to reference Kurtz, L. A. (2005). Digital actors and copyright-from the polar express to simone. Santa Clara Computer and High Technology Law Journal, 21, 783. Kurtz, L. A. (2005). Digital actors and copyright-from the polar express to simone. Santa Clara Computer and High Technology Law Journal, 21, 783.
3.
go back to reference Jőrg, S. (2011). Perception of body and hand animations for realistic virtual characters. PhD thesis, Trinity College Dublin. Jőrg, S. (2011). Perception of body and hand animations for realistic virtual characters. PhD thesis, Trinity College Dublin.
4.
go back to reference Debevec, P. (2012). The light stages and their applications to photoreal digital actors. Singapore: SIGGRAPH Asia Technical Briefs. Debevec, P. (2012). The light stages and their applications to photoreal digital actors. Singapore: SIGGRAPH Asia Technical Briefs.
5.
go back to reference Parke, F. I. (1972). Computer generated animation of faces. In Proceedings of the ACM annual conference (Vol. 1, pp. 451–457). Parke, F. I. (1972). Computer generated animation of faces. In Proceedings of the ACM annual conference (Vol. 1, pp. 451–457).
6.
go back to reference Alexander, O., Rogers, M., Lambeth, W., Chiang, J.-Y., Ma, W.-C., Wang, C.-C., & Debevec, P. (2010). The digital emily project: Achieving a photorealistic digital actor. Computer Graphics and Applications, 30(4), 20–31.CrossRef Alexander, O., Rogers, M., Lambeth, W., Chiang, J.-Y., Ma, W.-C., Wang, C.-C., & Debevec, P. (2010). The digital emily project: Achieving a photorealistic digital actor. Computer Graphics and Applications, 30(4), 20–31.CrossRef
7.
go back to reference Mori, M., MacDorman, K. F., & Kageki, N. (2012). The uncanny valley [from the field]. IEEE Robotics and Automation Magazine, 19(2), 98–100.CrossRef Mori, M., MacDorman, K. F., & Kageki, N. (2012). The uncanny valley [from the field]. IEEE Robotics and Automation Magazine, 19(2), 98–100.CrossRef
8.
go back to reference Geller, T. (2008). Overcoming the uncanny valley. IEEE Computer Graphics and Applications, 28(4), 11–17.CrossRef Geller, T. (2008). Overcoming the uncanny valley. IEEE Computer Graphics and Applications, 28(4), 11–17.CrossRef
9.
go back to reference Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., & Sagar, M. (2000). Acquiring the reflectance field of a human face. In Proceedings of the 27th annual conference on computer graphics and interactive techniques (pp. 145–156). Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., & Sagar, M. (2000). Acquiring the reflectance field of a human face. In Proceedings of the 27th annual conference on computer graphics and interactive techniques (pp. 145–156).
10.
go back to reference Robertson, B. (2009). What’s old is new again. Computer Graphics World. Robertson, B. (2009). What’s old is new again. Computer Graphics World.
11.
go back to reference Cosker, D., Eisert, P., Grau, O., Hancock, P. J. B., McKinnell, J., & Ong, E.-J. (2013). Applications of face analysis and modeling in media production. MultiMedia IEEE, 20(4), 18–27.CrossRef Cosker, D., Eisert, P., Grau, O., Hancock, P. J. B., McKinnell, J., & Ong, E.-J. (2013). Applications of face analysis and modeling in media production. MultiMedia IEEE, 20(4), 18–27.CrossRef
12.
go back to reference Alexander, O., Fyffe, G., Busch, J., Yu, X., Ichikari, R., Jones, A., Debevec, P., Jimenez, J., Danvoye, E., & Antionazzi, B., et al. (2013) Digital ira: Creating a real-time photoreal digital actor. ACM SIGGRAPH 2013 Posters 1. Alexander, O., Fyffe, G., Busch, J., Yu, X., Ichikari, R., Jones, A., Debevec, P., Jimenez, J., Danvoye, E., & Antionazzi, B., et al. (2013) Digital ira: Creating a real-time photoreal digital actor. ACM SIGGRAPH 2013 Posters 1.
13.
go back to reference Bhat, K. S., Goldenthal, R., Ye, Y., Mallet, R., Koperwas, M. (2013). High fidelity facial animation capture and retargeting with contours. In Proceedings of the 12th ACM SIGGRAPH/eurographics symposium on computer animation (pp. 7–14). Bhat, K. S., Goldenthal, R., Ye, Y., Mallet, R., Koperwas, M. (2013). High fidelity facial animation capture and retargeting with contours. In Proceedings of the 12th ACM SIGGRAPH/eurographics symposium on computer animation (pp. 7–14).
14.
go back to reference Afifi, M., Hussain, K. F., Ibrahim, H. M., & Omar, N. M. (2014). Fast video completion using patch-based synthesis and image registration. In IEEE international symposium on intelligent signal processing and communication systems (ISPACS) 2000–2004. Afifi, M., Hussain, K. F., Ibrahim, H. M., & Omar, N. M. (2014). Fast video completion using patch-based synthesis and image registration. In IEEE international symposium on intelligent signal processing and communication systems (ISPACS) 2000–2004.
15.
go back to reference Pérez, P., Gangnet, M., & Blake, A. (2003). Poisson image editing. ACM Transactions on Graphics (TOG), 22, 313–318.CrossRef Pérez, P., Gangnet, M., & Blake, A. (2003). Poisson image editing. ACM Transactions on Graphics (TOG), 22, 313–318.CrossRef
16.
go back to reference Liu, Z., Zhang, Z., Jacobs, C., & Cohen, M. (2001). Rapid modeling of animated faces from video. The Journal of Visualization and Computer Animation, 12(4), 227–240.MATHCrossRef Liu, Z., Zhang, Z., Jacobs, C., & Cohen, M. (2001). Rapid modeling of animated faces from video. The Journal of Visualization and Computer Animation, 12(4), 227–240.MATHCrossRef
17.
go back to reference Wang, C., Yan, S., Zhang, H., & Ma, W. (2005). Realistic 3d face modeling by fusing multiple 2d images. In Proceedings of the 11th international multimedia modelling conference (MMM) (pp. 139–146). IEEE. Wang, C., Yan, S., Zhang, H., & Ma, W. (2005). Realistic 3d face modeling by fusing multiple 2d images. In Proceedings of the 11th international multimedia modelling conference (MMM) (pp. 139–146). IEEE.
18.
go back to reference Blanz, V., & Vetter, T. (1999). A morphable model for the synthesis of 3d faces. In Proceedings of the 26th annual conference on computer graphics and interactive techniques (pp. 187–194). Blanz, V., & Vetter, T. (1999). A morphable model for the synthesis of 3d faces. In Proceedings of the 26th annual conference on computer graphics and interactive techniques (pp. 187–194).
19.
go back to reference Jain, A., Thormählen, T., Seidel, H.-P., & Theobalt, C. (2010). Moviereshape: Tracking and reshaping of humans in videos. ACM Transactions on Graphics (TOG), 29(6), 148:1–148:10.CrossRef Jain, A., Thormählen, T., Seidel, H.-P., & Theobalt, C. (2010). Moviereshape: Tracking and reshaping of humans in videos. ACM Transactions on Graphics (TOG), 29(6), 148:1–148:10.CrossRef
20.
go back to reference Abate, A. F., Nappi, M., Riccio, D., & Sabatino, G. (2007). 2d and 3d face recognition: A survey. Pattern Recognition Letters, 28(14), 1885–1906.CrossRef Abate, A. F., Nappi, M., Riccio, D., & Sabatino, G. (2007). 2d and 3d face recognition: A survey. Pattern Recognition Letters, 28(14), 1885–1906.CrossRef
21.
go back to reference Tong, J., Zhang, M., Xiang, X., Shen, H., Yan, H., & Chen, Z. (2011). 3d body scanning with hairstyle using one time-of-flight camera. Computer Animation and Virtual Worlds, 22(2–3), 203–211.CrossRef Tong, J., Zhang, M., Xiang, X., Shen, H., Yan, H., & Chen, Z. (2011). 3d body scanning with hairstyle using one time-of-flight camera. Computer Animation and Virtual Worlds, 22(2–3), 203–211.CrossRef
22.
go back to reference Lee, Y., Terzopoulos, D., Waters, K. (1995). Realistic modeling for facial animation. In Proceedings of the 22nd annual conference on computer graphics and interactive techniques (pp. 55–62). Lee, Y., Terzopoulos, D., Waters, K. (1995). Realistic modeling for facial animation. In Proceedings of the 22nd annual conference on computer graphics and interactive techniques (pp. 55–62).
23.
go back to reference Beeler, T., Bickel, B., Beardsley, P., Sumner, B., & Gross, M. (2010). High-quality single-shot capture of facial geometry. ACM Transactions on Graphics (TOG), 29(4), 40.CrossRef Beeler, T., Bickel, B., Beardsley, P., Sumner, B., & Gross, M. (2010). High-quality single-shot capture of facial geometry. ACM Transactions on Graphics (TOG), 29(4), 40.CrossRef
24.
go back to reference Bermano, A. H., Bradley, D., Beeler, T., Zund, F., Nowrouzezahrai, D., Baran, I., et al. (2014). Facial performance enhancement using dynamic shape space analysis. ACM Transactions on Graphics (TOG), 33(2), 13.CrossRef Bermano, A. H., Bradley, D., Beeler, T., Zund, F., Nowrouzezahrai, D., Baran, I., et al. (2014). Facial performance enhancement using dynamic shape space analysis. ACM Transactions on Graphics (TOG), 33(2), 13.CrossRef
25.
go back to reference Sagar, M. (2004). Reflectance field rendering of human faces for spider-man 2 (p. 118). ACM SIGGRAPH 2004 Sketches. Sagar, M. (2004). Reflectance field rendering of human faces for spider-man 2 (p. 118). ACM SIGGRAPH 2004 Sketches.
26.
go back to reference Ghosh, A., Fyffe, G., Tunwattanapong, B., Busch, J., Xueming, Yu., & Debevec, P. (2011). Multiview face capture using polarized spherical gradient illumination. ACM Transactions on Graphics (TOG), 30(6), 129.CrossRef Ghosh, A., Fyffe, G., Tunwattanapong, B., Busch, J., Xueming, Yu., & Debevec, P. (2011). Multiview face capture using polarized spherical gradient illumination. ACM Transactions on Graphics (TOG), 30(6), 129.CrossRef
27.
go back to reference Gleicher, M. (1999). Animation from observation: Motion capture and motion editing. ACM SIGGRAPH Computer Graphics, 33(4), 51–54.CrossRef Gleicher, M. (1999). Animation from observation: Motion capture and motion editing. ACM SIGGRAPH Computer Graphics, 33(4), 51–54.CrossRef
28.
go back to reference Brown, W. (2009). Beowulf: The digital monster movie. Animation, 4(2), 153–168.CrossRef Brown, W. (2009). Beowulf: The digital monster movie. Animation, 4(2), 153–168.CrossRef
29.
go back to reference Wyatt, R., & Klaw, R. (2013). The apes of wrath. San Francisco: Tachyon Publications. Wyatt, R., & Klaw, R. (2013). The apes of wrath. San Francisco: Tachyon Publications.
30.
go back to reference Stoll, C., Gall, J., de Aguiar, E., Thrun, S., & Theobalt, C. (2010). Video-based reconstruction of animatable human characters. ACM Transactions on Graphics (TOG), 29(6), 139:1–139:10.CrossRef Stoll, C., Gall, J., de Aguiar, E., Thrun, S., & Theobalt, C. (2010). Video-based reconstruction of animatable human characters. ACM Transactions on Graphics (TOG), 29(6), 139:1–139:10.CrossRef
31.
go back to reference Xu, F., Liu, Y., Stoll, C., Tompkin, J., Bharaj, G., Dai, Q., et al. (2011). Video-based characters: Creating new human performances from a multi-view video database. ACM Transactions on Graphics (TOG), 30(4), 32:1–32:10. Xu, F., Liu, Y., Stoll, C., Tompkin, J., Bharaj, G., Dai, Q., et al. (2011). Video-based characters: Creating new human performances from a multi-view video database. ACM Transactions on Graphics (TOG), 30(4), 32:1–32:10.
32.
go back to reference Deng, Z., Noh, J. (2007). Computer facial animation: A survey. In Data-driven 3D facial animation (pp. 1–28). Deng, Z., Noh, J. (2007). Computer facial animation: A survey. In Data-driven 3D facial animation (pp. 1–28).
33.
go back to reference Ekman, P., & Friesen, W. V. (1977). Facial action coding system. Palo Alto: Consulting Psychologists Press, Stanford University. Ekman, P., & Friesen, W. V. (1977). Facial action coding system. Palo Alto: Consulting Psychologists Press, Stanford University.
34.
go back to reference Terzopoulos, D., & Waters, K. (1990). Physically-based facial modelling, analysis, and animation. The Journal of Visualization and Computer Animation, 1(2), 73–80.CrossRef Terzopoulos, D., & Waters, K. (1990). Physically-based facial modelling, analysis, and animation. The Journal of Visualization and Computer Animation, 1(2), 73–80.CrossRef
35.
go back to reference Hyneman, W., Itokazu, H., Williams, L., & Zhao, X. (2005). Human face project. ACM SIGGRAPH 2005 Courses 5. Hyneman, W., Itokazu, H., Williams, L., & Zhao, X. (2005). Human face project. ACM SIGGRAPH 2005 Courses 5.
36.
go back to reference Wolff, E. (2003). Creating virtual performers: Disneys human face project. Millimeter magazine. Wolff, E. (2003). Creating virtual performers: Disneys human face project. Millimeter magazine.
37.
go back to reference Yuksel, C., Schaefer, S., & Keyser, J. (2009). Hair meshes. ACM Transactions on Graphics (TOG), 28(5), 166:1–166:7.CrossRef Yuksel, C., Schaefer, S., & Keyser, J. (2009). Hair meshes. ACM Transactions on Graphics (TOG), 28(5), 166:1–166:7.CrossRef
38.
go back to reference Lucas, B. D., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In Proceedings of the 7th international joint conference on Artificial intelligence (Vol. 2, pp. 674–679). Lucas, B. D., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In Proceedings of the 7th international joint conference on Artificial intelligence (Vol. 2, pp. 674–679).
39.
go back to reference Tomasi, C., & Kanade, T. (1991). Detection and tracking of point features. Pittsburgh: School of Computer Science, Carnegie Mellon Univ. Tomasi, C., & Kanade, T. (1991). Detection and tracking of point features. Pittsburgh: School of Computer Science, Carnegie Mellon Univ.
40.
go back to reference Shi, J., & Tomasi, C. (1994). Good features to track. In Proceedings of the IEEE computer society conference on computer vision and pattern recognition (pp. 593–600). Shi, J., & Tomasi, C. (1994). Good features to track. In Proceedings of the IEEE computer society conference on computer vision and pattern recognition (pp. 593–600).
41.
go back to reference Black, M. J., & Jepson, A. D. (1998). Eigentracking: Robust matching and tracking of articulated objects using a view-based representation. International Journal of Computer Vision, 26(1), 63–84.CrossRef Black, M. J., & Jepson, A. D. (1998). Eigentracking: Robust matching and tracking of articulated objects using a view-based representation. International Journal of Computer Vision, 26(1), 63–84.CrossRef
42.
go back to reference Comaniciu, D., Ramesh, V., & Meer, P. (2003). Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(5), 564–577.CrossRef Comaniciu, D., Ramesh, V., & Meer, P. (2003). Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(5), 564–577.CrossRef
43.
go back to reference Jepson, A. D., Fleet, D. J., & El-Maraghi, T. F. (2003). Robust online appearance models for visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10), 1296–1311.CrossRef Jepson, A. D., Fleet, D. J., & El-Maraghi, T. F. (2003). Robust online appearance models for visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10), 1296–1311.CrossRef
44.
go back to reference Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking: A survey. ACM computing surveys (CSUR), 38(4), 13.CrossRef Yilmaz, A., Javed, O., & Shah, M. (2006). Object tracking: A survey. ACM computing surveys (CSUR), 38(4), 13.CrossRef
45.
go back to reference Torr, P. H. S., & Murray, D. W. (1997). The development and comparison of robust methodsfor estimating the fundamental matrix. International Journal of Computer Vision, 24(3), 271–300.CrossRef Torr, P. H. S., & Murray, D. W. (1997). The development and comparison of robust methodsfor estimating the fundamental matrix. International Journal of Computer Vision, 24(3), 271–300.CrossRef
46.
go back to reference Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.MathSciNetCrossRef Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.MathSciNetCrossRef
47.
go back to reference Cevher, V., Shah, F., Velmurugan, R., & McClellan, J. H. (2007). A multi target bearing tracking system using random sampling consensus. IEEE Aerospace Conference, 1–15. Cevher, V., Shah, F., Velmurugan, R., & McClellan, J. H. (2007). A multi target bearing tracking system using random sampling consensus. IEEE Aerospace Conference, 1–15.
48.
go back to reference Torr, P. H., & Zisserman, A. (2000). MLESAC: A new robust estimator with application to estimating image geometry. Computer Vision and Image Understanding, 78(1), 138–156.CrossRef Torr, P. H., & Zisserman, A. (2000). MLESAC: A new robust estimator with application to estimating image geometry. Computer Vision and Image Understanding, 78(1), 138–156.CrossRef
49.
go back to reference Kato, H., & Billinghurst, M. (1999). Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In Proceedings of 2nd IEEE and ACM international workshop on augmented reality (IWAR’99) (pp. 85–94). Kato, H., & Billinghurst, M. (1999). Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In Proceedings of 2nd IEEE and ACM international workshop on augmented reality (IWAR’99) (pp. 85–94).
50.
go back to reference Wright, S. (2013). Digital compositing for film and video. Boca Raton: CRC Press. Wright, S. (2013). Digital compositing for film and video. Boca Raton: CRC Press.
51.
go back to reference Afifi, M., Hussain, K. F., Ibrahim, H. M., & Omar, N. M. (2014). Video face replacement system using a modified Poisson blending technique. In IEEE international symposium on intelligent signal processing and communication systems (ISPACS) 2005–2010. Afifi, M., Hussain, K. F., Ibrahim, H. M., & Omar, N. M. (2014). Video face replacement system using a modified Poisson blending technique. In IEEE international symposium on intelligent signal processing and communication systems (ISPACS) 2005–2010.
Metadata
Title
A Low-cost System for Generating Near-realistic Virtual Actors
Authors
Mahmoud Afifi
Khaled F. Hussain
Hosny M. Ibrahim
Nagwa M. Omar
Publication date
01-06-2015
Publisher
3D Display Research Center
Published in
3D Research / Issue 2/2015
Electronic ISSN: 2092-6731
DOI
https://doi.org/10.1007/s13319-015-0050-y

Premium Partner