nach oben

Erschienen in:

2020 | OriginalPaper | Buchkapitel

Capture, Reconstruction, and Representation of the Visual Real World for Virtual Reality

verfasst von : Christian Richardt, James Tompkin, Gordon Wetzstein

Erschienen in: Real VR – Immersive Digital Reality

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

We provide an overview of the concerns, current practice, and limitations for capturing, reconstructing, and representing the real world visually within virtual reality. Given that our goals are to capture, transmit, and depict complex real-world phenomena to humans, these challenges cover the opto-electro-mechanical, computational, informational, and perceptual fields. Practically producing a system for real-world VR capture requires navigating a complex design space and pushing the state of the art in each of these areas. As such, we outline several promising directions for future work to improve the quality and flexibility of real-world VR capture systems.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nächstes Kapitel Light Field Video for Immersive Content Production

Aggarwal, R., Vohra, A., Namboodiri, A.M.: Panoramic stereo videos with a single camera. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3755–3763, June 2016. https://doi.org/10.1109/CVPR.2016.408

Aliev, K.A., Ulyanov, D., Lempitsky, V.: Neural point-based graphics (2019). arXiv:1906.08240

Anderson, R., et al.: Jump: virtual reality video. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 35(6), 198:1–198:13 (2016). https://doi.org/10.1145/2980179.2980257CrossRef

Bau, D., et al.: Seeing what a GAN cannot generate. In: Proceedings of the International Conference on Computer Vision (ICCV) (2019)

Bertel, T., Campbell, N.D.F., Richardt, C.: MegaParallax: casual 360\(^\circ \) panoramas with motion parallax. IEEE Trans. Visual Comput. Graphics 25(5), 1828–1835 (2019). https://doi.org/10.1109/TVCG.2019.2898799CrossRef

Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1) (2007). https://doi.org/10.1007/s11263-006-0002-3

Buehler, C., Bosse, M., McMillan, L., Gortler, S., Cohen, M.: Unstructured lumigraph rendering. In: Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pp. 425–432 (2001). https://doi.org/10.1145/383259.383309

Bussone, W.: Linear and angular head accelerations in daily life. Ph.D. thesis, Virginia Tech (2005)

Cabral, B.: VR capture: designing and building an open source 3D-360 video camera. In: SIGGRAPH Asia Keynote, December 2016

10.

Chai, J.X., Tong, X., Chan, S.C., Shum, H.Y.: Plenoptic sampling. In: Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pp. 307–318 (2000). https://doi.org/10.1145/344779.344932

11.

Chaurasia, G., Duchêne, S., Sorkine-Hornung, O., Drettakis, G.: Depth synthesis and local warps for plausible image-based navigation. ACM Trans. Graph. 32(3), 30:1–30:12 (2013). https://doi.org/10.1145/2487228.2487238CrossRef

12.

Chaurasia, G., Sorkine-Hornung, O., Drettakis, G.: Silhouette-aware warping for image-based rendering. Comput. Graph. Forum (Proc. Eurographics Symp. Rendering) 30(4), 1223–1232 (2011). https://doi.org/10.1111/j.1467-8659.2011.01981.xCrossRef

13.

Chen, S.E., Williams, L.: View interpolation for image synthesis. In: Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pp. 279–288 (1993). https://doi.org/10.1145/166117.166153

14.

Cohen, T.S., Welling, M.: Transformation properties of learned visual representations. In: Proceedings of the International Conference on Learning Representations (ICLR) (2015)

15.

Collet, A., et al.: High-quality streamable free-viewpoint video. ACM Trans. Graph. (Proc. SIGGRAPH) 34(4), 69:1–69:13 (2015). https://doi.org/10.1145/2766945CrossRef

16.

Curless, B., Seitz, S., Bouguet, J.Y., Debevec, P., Levoy, M., Nayar, S.K.: 3D photography. In: SIGGRAPH Courses (2000). http://www.cs.cmu.edu/~seitz/course/3DPhoto.html

17.

Dai, A., Nießner, M., Zollhöfer, M., Izadi, S., Theobalt, C.: BundleFusion: real-time globally consistent 3D reconstruction using on-the-fly surface reintegration. ACM Trans. Graph. 36(3), 24:1–24:18 (2017). https://doi.org/10.1145/3054739CrossRef

18.

Davis, A., Levoy, M., Durand, F.: Unstructured light fields. Comput. Graph. Forum (Proc. Eurographics) 31(2), 305–314 (2012). https://doi.org/10.1111/j.1467-8659.2012.03009.xCrossRef

19.

Debevec, P.: The light stages and their applications to photoreal digital actors. In: SIGGRAPH Asia Technical Briefs (2012)

20.

Debevec, P., Bregler, C., Cohen, M.F., McMillan, L., Sillion, F., Szeliski, R.: Image-based modeling, rendering, and lighting. In: SIGGRAPH Courses (2000). https://www.pauldebevec.com/IBMR99/

21.

Debevec, P.E., Taylor, C.J., Malik, J.: Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach. In: Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pp. 11–20, August 1996. https://doi.org/10.1145/237170.237191

22.

Dosovitskiy, A., Springenberg, J.T., Tatarchenko, M., Brox, T.: Learning to generate chairs, tables and cars with convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 692–705 (2017). https://doi.org/10.1109/TPAMI.2016.2567384CrossRef

23.

DXOMARK: RED Helium 8K DxOMark sensor score: 108—a new all-time-high score! https://www.dxomark.com/red-helium-8k-dxomark-sensor-score-108-a-new-all-time-high-score2/. Accessed 30 Oct 2019

24.

Eslami, S.M.A., et al.: Neural scene representation and rendering. Science 360(6394), 1204–1210 (2018). https://doi.org/10.1126/science.aar6170CrossRef

25.

Flynn, J., et al.: DeepView: view synthesis with learned gradient descent. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2367–2376, June 2019

26.

Flynn, J., Neulander, I., Philbin, J., Snavely, N.: DeepStereo: learning to predict new views from the world’s imagery. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5515–5524, June 2016. https://doi.org/10.1109/CVPR.2016.595

27.

Fuhrmann, S., Langguth, F., Goesele, M.: MVE: a multi-view reconstruction environment. In: Proceedings of the Eurographics Workshop on Graphics and Cultural Heritage, pp. 11–18 (2014). https://doi.org/10.2312/gch.20141299

28.

Galliani, S., Lasinger, K., Schindler, K.: Massively parallel multiview stereopsis by surface normal diffusion. In: Proceedings of the International Conference on Computer Vision (ICCV), pp. 873–881, December 2015. https://doi.org/10.1109/ICCV.2015.106

29.

Garon, M., Sunkavalli, K., Hadap, S., Carr, N., Lalonde, J.F.: Fast spatially-varying indoor lighting estimation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

30.

Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The lumigraph. In: Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pp. 43–54, August 1996. https://doi.org/10.1145/237170.237200

31.

Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2004). https://doi.org/10.1017/CBO9780511811685

32.

Hedman, P., Alsisan, S., Szeliski, R., Kopf, J.: Casual 3D photography. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 36(6), 234:1–234:15 (2017). https://doi.org/10.1145/3130800.3130828CrossRef

33.

Hedman, P., Kopf, J.: Instant 3D photography. ACM Trans. Graph. (Proc. SIGGRAPH) 37(4), 101:1–101:12 (2018). https://doi.org/10.1145/3197517.3201384CrossRef

34.

Hedman, P., Philip, J., Price, T., Frahm, J.M., Drettakis, G.: Deep blending for free-viewpoint image-based rendering. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37(6), 257:1–257:15 (2018). https://doi.org/10.1145/3272127.3275084CrossRef

35.

Hedman, P., Ritschel, T., Drettakis, G., Brostow, G.: Scalable inside-out image-based rendering. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 35(6), 231:1–231:11 (2016). https://doi.org/10.1145/2980179.2982420CrossRef

36.

Huang, J., Chen, Z., Ceylan, D., Jin, H.: 6-DOF VR videos with a single 360-camera. In: Proceedings of IEEE Virtual Reality (VR), pp. 37–44, March 2017. https://doi.org/10.1109/VR.2017.7892229

37.

Huang, P.H., Matzen, K., Kopf, J., Ahuja, N., Huang, J.B.: DeepMVS: learning multi-view stereopsis. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

38.

Hukkelås, H., Mester, R., Lindseth, F.: DeepPrivacy: a generative adversarial network for face anonymization. In: Bebis, G., et al. (eds.) ISVC 2019. LNCS, vol. 11844, pp. 565–578. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33720-9_44CrossRef

39.

Ishiguro, H., Yamamoto, M., Tsuji, S.: Omni-directional stereo. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 257–262 (1992). https://doi.org/10.1109/34.121792CrossRef

40.

Jahanian, A., Chai, L., Isola, P.: On the “steerability” of generative adversarial networks (2019). arXiv:1907.07171

41.

Jancosek, M., Pajdla, T.: Multi-view reconstruction preserving weakly-supported surfaces. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3121–3128, June 2011. https://doi.org/10.1109/CVPR.2011.5995693

42.

Ji, D., Kwon, J., McFarland, M., Savarese, S.: Deep view morphing. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7092–7100, July 2017. https://doi.org/10.1109/CVPR.2017.750

43.

Kalantari, N.K., Wang, T.C., Ramamoorthi, R.: Learning-based view synthesis for light field cameras. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 35(6), 193:1–193:10 (2016). https://doi.org/10.1145/2980179.2980251CrossRef

44.

Keysers, C., Xiao, D.K., Földiák, P., Perrett, D.I.: The speed of sight. J. Cogn. Neurosci. 13(1), 90–101 (2001). https://doi.org/10.1162/089892901564199CrossRef

45.

Kim, C., Zimmer, H., Pritch, Y., Sorkine-Hornung, A., Gross, M.: Scene reconstruction from high spatio-angular resolution light fields. ACM Trans. Graph. (Proc. SIGGRAPH) 32(4), 73:1–73:12 (2013). https://doi.org/10.1145/2461912.2461926CrossRefMATH

46.

Kim, H., et al.: Deep video portraits. ACM Trans. Graph. (Proc. SIGGRAPH) 37(4), 163:1–163:14 (2018). https://doi.org/10.1145/3197517.3201283CrossRef

47.

Kirillov, A., He, K., Girshick, R., Rother, C., Dollár, P.: Panoptic segmentation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

48.

Konrad, R., Dansereau, D.G., Masood, A., Wetzstein, G.: SpinVR: towards live-streaming 3D virtual reality video. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 36(6), 209:1–209:12 (2017). https://doi.org/10.1145/3130800.3130836CrossRef

49.

Kopf, J., et al.: Practical 3D photography. In: Proceedings of CVPR Workshops (2019)

50.

Koulieris, G.A., Akşit, K., Stengel, M., Mantiuk, R.K., Mania, K., Richardt, C.: Near-eye display and tracking technologies for virtual and augmented reality. Comput. Graph. Forum 38(2), 493–519 (2019). https://doi.org/10.1111/cgf.13654CrossRef

51.

Kulkarni, T.D., Whitney, W., Kohli, P., Tenenbaum, J.B.: Deep convolutional inverse graphics network. In: Advances in Neural Information Processing Systems (NIPS) (2015)

52.

Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., Raskar, R.: Polarization fields: dynamic light field display using multi-layer LCDs. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 30(6), 186:1–186:10 (2011). https://doi.org/10.1145/2070781.2024220CrossRef

53.

Lee, J., Kim, B., Kim, K., Kim, Y., Noh, J.: Rich360: optimized spherical representation from structured panoramic camera arrays. ACM Trans. Graph. (Proc. SIGGRAPH) 35(4), 63:1–63:11 (2016). https://doi.org/10.1145/2897824.2925983CrossRef

54.

LeGendre, C., et al.: DeepLight: learning illumination for unconstrained mobile mixed reality. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

55.

Levoy, M., Hanrahan, P.: Light field rendering. In: Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pp. 31–42, August 1996. https://doi.org/10.1145/237170.237199

56.

Lipski, C., Linz, C., Berger, K., Sellent, A., Magnor, M.: Virtual video camera: image-based viewpoint navigation through space and time. Comput. Graph. Forum 29(8), 2555–2568 (2010). https://doi.org/10.1111/j.1467-8659.2010.01824.xCrossRef

57.

Lombardi, S., Simon, T., Saragih, J., Schwartz, G., Lehrmann, A., Sheikh, Y.: Neural volumes: learning dynamic renderable volumes from images. ACM Trans. Graph. (Proc. SIGGRAPH) 38(4), 65:1–65:14 (2019). https://doi.org/10.1145/3306346.3323020CrossRef

58.

Luo, B., Xu, F., Richardt, C., Yong, J.H.: Parallax360: stereoscopic 360\(^\circ \) scene representation for head-motion parallax. IEEE Trans. Vis. Comput. Graph. 24(4), 1545–1553 (2018). https://doi.org/10.1109/TVCG.2018.2794071CrossRef

59.

Magnor, M., Grau, O., Sorkine-Hornung, O., Theobalt, C. (eds.): Digital Representations of the Real World: How to Capture, Model, and Render Visual Reality. A K Peters/CRC Press, New York (2015)MATH

60.

Martin-Brualla, R., et al.: LookinGood: enhancing performance capture with real-time neural re-rendering. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37(6), 255:1–255:14 (2018). https://doi.org/10.1145/3272127.3275099CrossRef

61.

Meka, A., et al.: Deep reflectance fields: high-quality facial reflectance field inference from color gradient illumination. ACM Trans. Graph. (Proc. SIGGRAPH) 38(4), 77:1–77:12 (2019). https://doi.org/10.1145/3306346.3323027CrossRef

62.

Meshry, M., et al.: Neural rerendering in the wild. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

63.

Mildenhall, B., et al.: Local light field fusion: practical view synthesis with prescriptive sampling guidelines. ACM Trans. Graph. (Proc. SIGGRAPH) 38(4), 29:1–29:14 (2019). https://doi.org/10.1145/3306346.3322980CrossRef

64.

Mori, M.: The uncanny valley. Energy 7(4), 33–35 (1970). (in Japanese)

65.

Moulon, P., Monasse, P., Marlet, R.: Adaptive structure from motion with a Contrario model estimation. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7727, pp. 257–270. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-37447-0_20CrossRef

66.

Mustafa, A., Volino, M., Guillemaut, J.Y., Hilton, A.: 4D temporally coherent light-field video. In: Proceedings of International Conference on 3D Vision (3DV) (2017)

67.

Mustafa, A., Volino, M., Kim, H., Guillemaut, J.Y., Hilton, A.: Temporally coherent general dynamic scene reconstruction (2019). arXiv:1907.08195

68.

Nam, G., Lee, J.H., Gutierrez, D., Kim, M.H.: Practical SVBRDF acquisition of 3D objects with unstructured flash photography. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37(6), 267:1–267:12 (2018). https://doi.org/10.1145/3272127.3275017CrossRef

69.

Newcombe, R.A., et al.: KinectFusion: real-time dense surface mapping and tracking. In: Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR), pp. 127–136, October 2011. https://doi.org/10.1109/ISMAR.2011.6092378

70.

Nguyen-Phuoc, T., Li, C., Theis, L., Richardt, C., Yang, Y.L.: HoloGAN: unsupervised learning of 3D representations from natural images. In: Proceedings of the International Conference on Computer Vision (ICCV) (2019)

71.

Nießner, M., Zollhöfer, M., Izadi, S., Stamminger, M.: Real-time 3D reconstruction at scale using voxel hashing. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 32(6), 169:1–169:11 (2013). https://doi.org/10.1145/2508363.2508374CrossRef

72.

Niklaus, S., Mai, L., Yang, J., Liu, F.: 3D Ken Burns effect from a single image. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 38(6), 184:1–184:15 (2019). https://doi.org/10.1145/3355089.3356528CrossRef

73.

Oculus: From the lab to the living room: the story behind Facebook’s Oculus Insight technology and a new era of consumer VR. https://tech.fb.com/the-story-behind-oculus-insight-technology/. Accessed 30 Oct 2019

74.

Olszewski, K., Tulyakov, S., Woodford, O., Li, H., Luo, L.: Transformable bottleneck networks. In: Proceedings of the International Conference on Computer Vision (ICCV) (2019)

75.

Overbeck, R.S., Erickson, D., Evangelakos, D., Pharr, M., Debevec, P.: A system for acquiring, compressing, and rendering panoramic light field stills for virtual reality. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37(6), 197:1–197:15 (2018). https://doi.org/10.1145/3272127.3275031CrossRef

76.

Park, E., Yang, J., Yumer, E., Ceylan, D., Berg, A.C.: Transformation-grounded image generation network for novel 3D view synthesis. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 702–711, July 2017. https://doi.org/10.1109/CVPR.2017.82

77.

Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: DeepSDF: learning continuous signed distance functions for shape representation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

78.

Parra Pozo, A., et al.: An integrated 6DoF video camera and system design. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 38(6), 216:1–216:16 (2019). https://doi.org/10.1145/3355089.3356555. https://github.com/facebook/facebook360depCrossRef

79.

Peleg, S., Ben-Ezra, M., Pritch, Y.: Omnistereo: panoramic stereo imaging. IEEE Trans. Pattern Anal. Mach. Intell. 23(3), 279–290 (2001). https://doi.org/10.1109/34.910880CrossRefMATH

80.

Penner, E., Zhang, L.: Soft 3D reconstruction for view synthesis. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 36(6), 235:1–235:11 (2017). https://doi.org/10.1145/3130800.3130855CrossRef

81.

Perazzi, F., et al.: Panoramic video from unstructured camera arrays. Comput. Graph. Forum (Proc. Eurographics) 34(2), 57–68 (2015). https://doi.org/10.1111/cgf.12541CrossRef

82.

Prada, F., Kazhdan, M., Chuang, M., Collet, A., Hoppe, H.: Spatiotemporal atlas parameterization for evolving meshes. ACM Trans. Graph. (Proc. SIGGRAPH) 36(4), 58:1–58:12 (2017). https://doi.org/10.1145/3072959.3073679CrossRef

83.

Qi, M., Li, W., Yang, Z., Wang, Y., Luo, J.: Attentive relational networks for mapping images to scene graphs. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

84.

Rhodin, H., Salzmann, M., Fua, P.: Unsupervised geometry-aware representation for 3D human pose estimation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 765–782. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_46CrossRef

85.

Richardt, C., Hedman, P., Overbeck, R.S., Cabral, B., Konrad, R., Sullivan, S.: Capture4VR: from VR photography to VR video. In: SIGGRAPH Courses (2019). https://doi.org/10.1145/3305366.3328028

86.

Richardt, C., Pritch, Y., Zimmer, H., Sorkine-Hornung, A.: Megastereo: constructing high-resolution stereo panoramas. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1256–1263, June 2013. https://doi.org/10.1109/CVPR.2013.166

87.

Schroers, C., Bazin, J.C., Sorkine-Hornung, A.: An omnistereoscopic video pipeline for capture and display of real-world VR. ACM Trans. Graph. 37(3), 37:1–37:13 (2018). https://doi.org/10.1145/3225150CrossRef

88.

Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4104–4113 (2016). https://doi.org/10.1109/CVPR.2016.445

89.

Schönberger, J.L., Zheng, E., Frahm, J.-M., Pollefeys, M.: Pixelwise view selection for unstructured multi-view stereo. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 501–518. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_31CrossRef

90.

Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 519–528 (2006). https://doi.org/10.1109/CVPR.2006.19

91.

Serrano, A., et al.: Motion parallax for 360\(^\circ \) RGBD video. IEEE Trans. Vis. Comput. Graph. 25(5), 1817–1827 (2019). https://doi.org/10.1109/TVCG.2019.2898757CrossRef

92.

Shum, H., Kang, S.B.: Review of image-based rendering techniques. In: Proceedings of the SPIE Visual Communications and Image Processing, vol. 4067 (2000). https://doi.org/10.1117/12.386541

93.

Shum, H.Y., Chan, S.C., Kang, S.B.: Image-Based Rendering. Springer, Boston (2007). https://doi.org/10.1007/978-0-387-32668-9CrossRefMATH

94.

Sitzmann, V., Thies, J., Heide, F., Nießner, M., Wetzstein, G., Zollhöfer, M.: DeepVoxels: learning persistent 3D feature embeddings. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2437–2446 (2019)

95.

Sitzmann, V., Zollhöfer, M., Wetzstein, G.: Scene representation networks: continuous 3D-structure-aware neural scene representations. In: Advances in Neural Information Processing Systems (NeurIPS) (2019)

96.

Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring photo collections in 3D. ACM Trans. Graph. (Proc. SIGGRAPH) 25(3), 835–846 (2006). https://doi.org/10.1145/1141911.1141964CrossRef

97.

Speciale, P., Schönberger, J.L., Kang, S.B., Sinha, S.N., Pollefeys, M.: Privacy preserving image-based localization. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

98.

Srinivasan, P.P., Tucker, R., Barron, J.T., Ramamoorthi, R., Ng, R., Snavely, N.: Pushing the boundaries of view extrapolation with multiplane images. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

99.

Sweeney, C.: Theia multiview geometry library (2016). http://theia-sfm.org

100.

Sweeney, C., Holynski, A., Curless, B., Seitz, S.M.: Structure from motion for panorama-style videos (2019). arXiv:1906.03539

101.

Szeliski, R.: Image alignment and stitching: a tutorial. Found. Trends Comput. Graph. Vis. 2(1), 1–104 (2006). https://doi.org/10.1561/0600000009MathSciNetCrossRefMATH

102.

Szeliski, R.: Computer Vision: Algorithms and Applications. Springer, London (2010). https://doi.org/10.1007/978-1-84882-935-0. http://szeliski.org/Book/CrossRefMATH

103.

Tarko, J., Tompkin, J., Richardt, C.: Real-time virtual object insertion for moving 360\(^\circ \) videos. In: Proceedings of the International Conference on Virtual-Reality Continuum and its Applications in Industry (VRCAI) (2019)

104.

Tatarchenko, M., Dosovitskiy, A., Brox, T.: Multi-view 3D models from single images with a convolutional network. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 322–337. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_20CrossRef

105.

Thies, J., Zollhöfer, M., Nießner, M.: Deferred neural rendering: image synthesis using neural textures. ACM Trans. Graph. (Proc. SIGGRAPH) 38(4), 66:1–66:12 (2019). https://doi.org/10.1145/3306346.3323035CrossRef

106.

Tulsiani, S., Tucker, R., Snavely, N.: Layer-structured 3D scene inference via view synthesis. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 311–327. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_19CrossRef

107.

Tung, H.Y.F., Cheng, R., Fragkiadaki, K.: Learning spatial common sense with geometry-aware recurrent networks. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2595–2603 (2019)

108.

Valve: Index headset. www.valvesoftware.com/en/index/headset. Accessed 30 Oct 2019

109.

Ventura, J.: Structure from motion on a sphere. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 53–68. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_4CrossRef

110.

Wei, S.E., et al.: VR facial animation via multiview image translation. ACM Trans. Graph. (Proc. SIGGRAPH) 38(4), 67:1–67:16 (2019). https://doi.org/10.1145/3306346.3323030CrossRef

111.

Weissig, C., Schreer, O., Eisert, P., Kauff, P.: The ultimate immersive experience: panoramic 3D video acquisition. In: Schoeffmann, K., Merialdo, B., Hauptmann, A.G., Ngo, C.-W., Andreopoulos, Y., Breiteneder, C. (eds.) MMM 2012. LNCS, vol. 7131, pp. 671–681. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-27355-1_72CrossRef

112.

Wetzstein, G., Lanman, D., Heidrich, W., Raskar, R.: Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays. ACM Trans. Graph. (Proc. SIGGRAPH) 30(4), 95:1–95:12 (2011). https://doi.org/10.1145/2010324.1964990CrossRef

113.

Wetzstein, G., Lanman, D., Hirsch, M., Raskar, R.: Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting. ACM Trans. Graph. (Proc. SIGGRAPH) 31(4), 80:1–80:11 (2012). https://doi.org/10.1145/2185520.2185576CrossRef

114.

Whelan, T., Salas-Moreno, R.F., Glocker, B., Davison, A.J., Leutenegger, S.: ElasticFusion: real-time dense SLAM and light source estimation. Int. J. Robot. Res. 35(14), 1697–1716 (2016). https://doi.org/10.1177/0278364916669237CrossRef

115.

Wood, D.N., et al.: Surface light fields for 3D photography. In: Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pp. 287–296 (2000). https://doi.org/10.1145/344779.344925

116.

Worrall, D.E., Garbin, S.J., Turmukhambetov, D., Brostow, G.J.: Interpretable transformations with encoder-decoder networks. In: Proceedings of the International Conference on Computer Vision (ICCV), pp. 5737–5746 (2017). https://doi.org/10.1109/ICCV.2017.611

117.

Wu, C.: VisualSFM: a visual structure from motion system (2011). http://ccwu.me/vsfm/

118.

Yang, J., Reed, S.E., Yang, M.H., Lee, H.: Weakly-supervised disentangling with recurrent transformations for 3D view synthesis. In: Advances in Neural Information Processing Systems (NIPS), pp. 1099–1107 (2015)

119.

Yifan, W., Serena, F., Wu, S., Öztireli, C., Sorkine-Hornung, O.: Differentiable surface splatting for point-based geometry processing. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 38(6) (2019). https://doi.org/10.1145/3355089.3356513

120.

Yücer, K., Sorkine-Hornung, A., Wang, O., Sorkine-Hornung, O.: Efficient 3D object segmentation from densely sampled light fields with applications to 3D reconstruction. ACM Trans. Graph. 35(3), 22:1–22:15 (2016). https://doi.org/10.1145/2876504CrossRef

121.

Zaragoza, J., Chin, T.J., Tran, Q.H., Brown, M.S., Suter, D.: As-projective-as-possible image stitching with moving DLT. IEEE Trans. Pattern Anal. Mach. Intell. 36(7), 1285–1298 (2014). https://doi.org/10.1109/TPAMI.2013.247CrossRef

122.

Zhang, F., Liu, F.: Parallax-tolerant image stitching. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3262–3269, June 2014. https://doi.org/10.1109/CVPR.2014.423

123.

Zheng, K.C., Kang, S.B., Cohen, M.F., Szeliski, R.: Layered depth panoramas. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR) (2007). https://doi.org/10.1109/CVPR.2007.383295

124.

Zhou, T., Tucker, R., Flynn, J., Fyffe, G., Snavely, N.: Stereo magnification: learning view synthesis using multiplane images. ACM Trans. Graph. (Proc. SIGGRAPH) 37(4), 65:1–65:12 (2018). https://doi.org/10.1145/3197517.3201323CrossRef

125.

Zhou, T., Tulsiani, S., Sun, W., Malik, J., Efros, A.A.: View synthesis by appearance flow. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 286–301. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_18CrossRef

126.

Zollhöfer, M., et al.: State of the art on monocular 3D face reconstruction, tracking, and applications. Comput. Graph. Forum 37(2), 523–550 (2018). https://doi.org/10.1111/cgf.13382CrossRef

Titel: Capture, Reconstruction, and Representation of the Visual Real World for Virtual Reality
verfasst von: Christian Richardt
James Tompkin
Gordon Wetzstein
Verlag: Springer International Publishing
Buch: Real VR – Immersive Digital Reality
Print ISBN: 978-3-030-41815-1

Electronic ISBN: 978-3-030-41816-8

Copyright-Jahr: 2020
DOI: https://doi.org/10.1007/978-3-030-41816-8_1

Neuer Inhalt

Bildnachweise

VDI-Icon, Profil Icon, inhalt2, Springer Professional Modul/© Springer Fachmedien Wiesbaden GmbH, Zukunftswerkstatt Sales Excellence_ieS/© Springer Fachmedien Wiesbaden GmbH, Search Icon, Banner Hanser, Strompreise/© vejaa / stock.adobe.com, Bunte Männchen, die Kunden darstelle, werden von einem riesigen Magneten angezogen. /© Oleksiy Mark, Dr. Daniel Schneider/© Fraunhofer IESE, Zeitschrift Wissensmanagement Cover, PatentFit-Logo/© Springer Fachmedien Wiesbaden GmbH, Zukunftswerkstatt Sales Excellence 2024/© AndreyPopov / Getty Images / iStock, 2023_Antrieb/© supervisuell, ATZ-Webinar: Prototypenfreie Entwicklung durch Offline- und Driver-in-the-Loop-HiL-Tests /© (c) VI-grade

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Neuer Inhalt

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.