Top

Published in:

2019 | OriginalPaper | Chapter

Computational Imaging in Projection Mapping

Author : Daisuke Iwai

Published in: Computational Color Imaging

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Projection mapping or spatial augmented reality (SAR) has been tremendously widespread over the world. The goal is to seamlessly merge physical and virtual worlds by superimposing computer generated graphics onto real surfaces. In projection mapping applications, target surfaces are generally not suitable for projection. They are textured and non-planar and conventional projectors are specifically designed to display high quality images onto uniformly white and flat surfaces only. Although researchers developed various algorithms to alleviate image quality degradations, the performances were limited by upper bounds resulting from the hardware. I briefly overview the recent advances in the projection mapping research field, particularly focuses on the computational imaging and display approach to overcome technical limitations of current projection hardware in terms of dynamic range, refresh rate, and depth-of-field. I also covers an emerging issue in the projection mapping research, which is dynamic projection mapping. This article is written by reorganizing a previously published state-of-the-art report paper by the same author [13] for an invited talk at the IAPR Computational Color Imaging Workshop (CCIW) 2019.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Improving Generalization Ability of Deep Neural Networks for Visual Recognition Tasks

next chapter On the Acquisition and Reproduction of Material Appearance

Aliaga, D.G., Yeung, Y.H., Law, A., Sajadi, B., Majumder, A.: Fast high-resolution appearance editing using superimposed projections. ACM Trans. Graph. 31(2), 13:1–13:13 (2012). https://doi.org/10.1145/2159516.2159518CrossRef

Asayama, H., Iwai, D., Sato, K.: Fabricating diminishable visual markers for geometric registration in projection mapping. IEEE Trans. Vis. Comput. Graph. 24(2), 1091–1102 (2018). https://doi.org/10.1109/TVCG.2017.2657634CrossRef

Asayama, H., Iwai, D., Sato, K.: Diminishable visual markers on fabricated projection object for dynamic spatial augmented reality. In: SIGGRAPH Asia 2015 Emerging Technologies, SA 2015, pp. 7:1–7:2. ACM, New York, NY, USA (2015). https://doi.org/10.1145/2818466.2818477

Bermano, A., Brüschweiler, P., Grundhöfer, A., Iwai, D., Bickel, B., Gross, M.: Augmenting physical avatars using projector-based illumination. ACM Trans. Graph. 32(6), 189:1–189:10 (2013). https://doi.org/10.1145/2508363.2508416CrossRef

Bermano, A.H., Billeter, M., Iwai, D., Grundhöfer, A.: Makeup lamps: live augmentation of human faces via projection. Comput. Graph. Forum 36(2), 311–323 (2017). https://doi.org/10.1111/cgf.13128CrossRef

Bimber, O., Emmerling, A.: Multifocal projection: a multiprojector technique for increasing focal depth. IEEE Trans. Vis. Comput. Graph. 12(4), 658–667 (2006). https://doi.org/10.1109/TVCG.2006.75CrossRef

Bimber, O., Iwai, D.: Superimposing dynamic range. ACM Trans. Graph. 27(5), 150:1–150:8 (2008). https://doi.org/10.1145/1409060.1409103CrossRef

Bimber, O., Iwai, D., Wetzstein, G., Grundhöfer, A.: The visual computing of projector-camera systems. Comput. Graph. Forum 27(8), 2219–2245 (2008). https://doi.org/10.1111/j.1467-8659.2008.01175.xCrossRef

Brown, M.S., Song, P., Cham, T.J.: Image pre-conditioning for out-of-focus projector blur. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006). vol. 2, pp. 1956–1963 (2006). https://doi.org/10.1109/CVPR.2006.145

10.

Damberg, G., Ballestad, A., Kozak, E., Minor, J., Kumaran, R., Gregson, J.: High brightness HDR projection using dynamic phase modulation. In: ACM SIGGRAPH 2015 Emerging Technologies, SIGGRAPH 2015, pp. 13:1–13:1. ACM, New York, NY, USA (2015). https://doi.org/10.1145/2782782.2792487

11.

Fujimoto, Y., et al.: Geometrically-correct projection-based texture mapping onto a deformable object. IEEE Trans. Vis. Comput. Graph. 20(4), 540–549 (2014). https://doi.org/10.1109/TVCG.2014.25CrossRef

12.

Grosse, M., Wetzstein, G., Grundhöfer, A., Bimber, O.: Coded aperture projection. ACM Trans. Graph. 29(3), 22:1–22:12 (2010). https://doi.org/10.1145/1805964.1805966CrossRef

13.

Grundhöfer, A., Iwai, D.: Recent advances in projection mapping algorithms, hardware and applications. Comput. Graph. Forum 37(2), 653–675. https://doi.org/10.1111/cgf.13387CrossRef

14.

Heide, F., Lanman, D., Reddy, D., Kautz, J., Pulli, K., Luebke, D.: Cascaded displays: spatiotemporal superresolution using offset pixel layers. ACM Trans. Graph. 33(4), 60:1–60:11 (2014). https://doi.org/10.1145/2601097.2601120CrossRefMATH

15.

Hiraki, T., Fukushima, S., Watase, H., Naemura, T.: Pixel-level visible light communication projector with interactive update of images and data. In: Proceedings of the International Display Workshops, pp. 1192–1195 (2018)

16.

Hirsch, M., Wetzstein, G., Raskar, R.: A compressive light field projection system. ACM Trans. Graph. 33(4), 58:1–58:12 (2014). https://doi.org/10.1145/2601097.2601144CrossRef

17.

Iwai, D., Mihara, S., Sato, K.: Extended depth-of-field projector by fast focal sweep projection. IEEE Trans. Vis. Comput. Graph. 21(4), 462–470 (2015). https://doi.org/10.1109/TVCG.2015.2391861CrossRef

18.

Iwai, D., Takeda, S., Hino, N., Sato, K.: Projection screen reflectance control for high contrast display using photochromic compounds and UV leds. Opt. Express 22(11), 13492–13506 (2014). https://doi.org/10.1364/OE.22.013492CrossRef

19.

Jones, B.R., Sodhi, R., Budhiraja, P., Karsch, K., Bailey, B., Forsyth, D.: Projectibles: optimizing surface color for projection. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, UIST 2015, pp. 137–146. ACM, New York, NY, USA (2015). https://doi.org/10.1145/2807442.2807486

20.

Kagami, S., Hashimoto, K.: Sticky projection mapping: 450-fps tracking projection onto a moving planar surface. In: SIGGRAPH Asia 2015 Emerging Technologies, SA 2015, pp. 23:1–23:3. ACM, New York, NY, USA (2015). https://doi.org/10.1145/2818466.2818485

21.

Kagami, S., Hashimoto, K.: A full-color single-chip-DLP projector with an embedded 2400-fps homography warping engine. In: ACM SIGGRAPH 2018 Emerging Technologies, SIGGRAPH 2018, pp. 1:1–1:2. ACM, New York, NY, USA (2018). https://doi.org/10.1145/3214907.3214927

22.

Kawabe, T., Fukiage, T., Sawayama, M., Nishida, S.: Deformation lamps: a projection technique to make static objects perceptually dynamic. ACM Trans. Appl. Percept. 13(2), 10:1–10:17 (2016). https://doi.org/10.1145/2874358CrossRef

23.

Kitajima, Y., Iwai, D., Sato, K.: Simultaneous projection and positioning of laser projector pixels. IEEE Trans. Vis. Comput. Graph. 23(11), 2419–2429 (2017). https://doi.org/10.1109/TVCG.2017.2734478CrossRef

24.

Lincoln, P., et al.: From motion to photons in 80 microseconds: towards minimal latency for virtual and augmented reality. IEEE Trans. Vis. Comput. Graph. 22(4), 1367–1376 (2016). https://doi.org/10.1109/TVCG.2016.2518038CrossRef

25.

Ma, C., Suo, J., Dai, Q., Raskar, R., Wetzstein, G.: High-rank coded aperture projection for extended depth of field. In: 2013 IEEE International Conference on Computational Photography (ICCP), pp. 1–9, April 2013. https://doi.org/10.1109/ICCPhot.2013.6528303

26.

Masia, B., Wetzstein, G., Didyk, P., Gutierrez, D.: A survey on computational displays: pushing the boundaries of optics, computation, and perception. Comput. Graph. 37(8), 1012–1038 (2013). https://doi.org/10.1016/j.cag.2013.10.003CrossRef

27.

Miyashita, L., Yamazaki, T., Uehara, K., Watanabe, Y., Ishikawa, M.: Portable lumipen: dynamic SAR in your hand. In: 2018 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6, July 2018. https://doi.org/10.1109/ICME.2018.8486514

28.

Miyashita, L., Watanabe, Y., Ishikawa, M.: Midas projection: Markerless and modelless dynamic projection mapping for material representation. In: SIGGRAPH Asia 2018 Technical Papers, SIGGRAPH Asia 2018, pp. 196:1–196:12. ACM, New York, NY, USA (2018). https://doi.org/10.1145/3272127.3275045CrossRef

29.

Nagase, M., Iwai, D., Sato, K.: Dynamic defocus and occlusion compensation of projected imagery by model-based optimal projector selection in multi-projection environment. Virtual Real. 15(2–3), 119–132 (2011). https://doi.org/10.1007/s10055-010-0168-4CrossRef

30.

Nakamura, T., Horisaki, R., Tanida, J.: Computational superposition projector for extended depth of field and field of view. Opt. Lett. 38(9), 1560–1562 (2013). https://doi.org/10.1364/OL.38.001560CrossRef

31.

Narita, G., Watanabe, Y., Ishikawa, M.: Dynamic projection mapping onto deforming non-rigid surface using deformable dot cluster marker. IEEE Trans. Vis. Comput. Graph. 23(3), 1235–1248 (2017). https://doi.org/10.1109/TVCG.2016.2592910CrossRef

32.

Narita, G., Watanabe, Y., Ishikawa, M.: Dynamic projection mapping onto a deformable object with occlusion based on high-speed tracking of dot marker array. In: Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology, VRST 2015, pp. 149–152. ACM, New York, NY, USA (2015). https://doi.org/10.1145/2821592.2821618

33.

Ng, A., Lepinski, J., Wigdor, D., Sanders, S., Dietz, P.: Designing for low-latency direct-touch input. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, UIST 2012, pp. 453–464. ACM, New York, NY, USA (2012). https://doi.org/10.1145/2380116.2380174

34.

Okumura, K., Oku, H., Ishikawa, M.: Lumipen: projection-based mixed reality for dynamic objects. In: 2012 IEEE International Conference on Multimedia and Expo (ICME), pp. 699–704, July 2012. https://doi.org/10.1109/ICME.2012.34

35.

Okutani, N., Takezawa, T., Iwai, D., Sato, K.: Stereoscopic capture in projection mapping. IEEE Access 6, 65894–65900 (2018). https://doi.org/10.1109/ACCESS.2018.2875905CrossRef

36.

Punpongsanon, P., Iwai, D., Sato, K.: Projection-based visualization of tangential deformation of nonrigid surface by deformation estimation using infrared texture. Virtual Real. 19(1), 45–56 (2015). https://doi.org/10.1007/s10055-014-0256-yCrossRef

37.

Regan, M., Miller, G.S.P.: The problem of persistence with rotating displays. IEEE Trans. Vis. Comput. Graph. 23(4), 1295–1301 (2017). https://doi.org/10.1109/TVCG.2017.2656979CrossRef

38.

Shimazu, S., Iwai, D., Sato, K.: 3D high dynamic range display system. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 235–236, October 2011. https://doi.org/10.1109/ISMAR.2011.6092393

39.

Siegl, C., Lange, V., Stamminger, M., Bauer, F., Thies, J.: FaceForge: markerless non-rigid face multi-projection mapping. IEEE Trans. Vis. Comput. Graph. 23(11), 2440–2446 (2017). https://doi.org/10.1109/TVCG.2017.2734428CrossRef

40.

Sueishi, T., Oku, H., Ishikawa, M.: Robust high-speed tracking against illumination changes for dynamic projection mapping. In: 2015 IEEE Virtual Reality (VR), pp. 97–104, March 2015. https://doi.org/10.1109/VR.2015.7223330

41.

Watanabe, Y., Kato, T., ishikawa, M.: Extended dot cluster marker for high-speed 3D tracking in dynamic projection mapping. In: 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 52–61, October 2017. https://doi.org/10.1109/ISMAR.2017.22

42.

Watanabe, Y., Narita, G., Tatsuno, S., Yuasa, T., Sumino, K., Ishikawa, M.: High-speed 8-bit image projector at 1,000 fps with 3 ms delay. In: Proceedings of the International Display Workshops, pp. 1064–1065 (2015)

43.

Wetzstein, G., Bimber, O.: Radiometric compensation through inverse light transport. In: 2007 15th Pacific Conference on Computer Graphics and Applications, PG 2007, pp. 391–399, October 2007. https://doi.org/10.1109/PG.2007.47

44.

Zhang, L., Nayar, S.: Projection defocus analysis for scene capture and image display. ACM Trans. Graph. 25(3), 907–915 (2006). https://doi.org/10.1145/1141911.1141974CrossRef

45.

Zheng, F., Whitted, T., Lastra, A., Lincoln, P., State, A., Maimone, A., Fuchs, H.: Minimizing latency for augmented reality displays: Frames considered harmful. In: 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 195–200, September 2014. https://doi.org/10.1109/ISMAR.2014.6948427

Title: Computational Imaging in Projection Mapping
Author: Daisuke Iwai
Publisher: Springer International Publishing
Book: Computational Color Imaging
Print ISBN: 978-3-030-13939-1

Electronic ISBN: 978-3-030-13940-7

Copyright Year: 2019
DOI: https://doi.org/10.1007/978-3-030-13940-7_2

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner