Jungjin Lee
Skip Abstract Section
Skip Supplemental Material SectionAbstract
This paper presents Rich360, a novel system for creating and viewing a 360° panoramic video obtained from multiple cameras placed on a structured rig. Rich360 provides an as-rich-as-possible 360° viewing experience by effectively resolving two issues that occur in the existing pipeline. First, a deformable spherical projection surface is utilized to minimize the parallax from multiple cameras. The surface is deformed spatio-temporally according to the depth constraints estimated from the overlapping video regions. This enables fast and efficient parallax-free stitching independent of the number of views. Next, a non-uniform spherical ray sampling is performed. The density of the sampling varies depending on the importance of the image region. Finally, for interactive viewing, the non-uniformly sampled video is mapped onto a uniform viewing sphere using a UV map. This approach can preserve the richness of the input videos when the resolution of the final 360° panoramic video is smaller than the overall resolution of the input videos, which is the case for most 360° panoramic videos. We show various results from Rich360 to demonstrate the richness of the output video and the advancement in the stitching results.
Supplemental Material
Available for Download
zip
a63-lee-supp.zip (483.3 MB)
Supplemental files.
- Birklbauer, C., and Bimber, O. 2014. Panorama light-field imaging. Computer Graphics Forum 33, 2, 43--52. Google Scholar
Digital Library
- Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In Computer Vision-ECCV 2004. 25--36.Google Scholar
- Carroll, R., Agrawala, M., and Agarwala, A. 2009. Optimizing content-preserving projections for wide-angle images. ACM Transactions on Graphics (TOG) 28, 3, 43. Google ScholarDigital Library
- Carroll, R., Agarwala, A., and Agrawala, M. 2010. Image warps for artistic perspective manipulation. ACM Transactions on Graphics (TOG) 29, 4, 127. Google ScholarDigital Library
- Chang, C.-H., Hu, M.-C., Cheng, W.-H., and Chuang, Y.-Y. 2013. Rectangling stereographic projection for wide-angle image visualization. In Proceedings of the IEEE International Conference on Computer Vision, 2824--2831. Google ScholarDigital Library
- Chang, C.-H., Sato, Y., and Chuang, Y.-Y. 2014. Shape-preserving half-projective warps for image stitching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3254--3261. Google ScholarDigital Library
- Chaurasia, G., Duchene, S., Sorkine-Hornung, O., and Drettakis, G. 2013. Depth synthesis and local warps for plausible image-based navigation. ACM Transactions on Graphics (TOG) 32, 3, 30. Google ScholarDigital Library
- Collet, A., Chuang, M., Sweeney, P., Gillett, D., Evseev, D., Calabrese, D., Hoppe, H., Kirk, A., and Sullivan, S. 2015. High-quality streamable free-viewpoint video. ACM Transactions on Graphics (TOG) 34, 4 (July), 69:1--69:13. Google ScholarDigital Library
- Hartley, R., and Zisserman, A. 2003. Multiple view geometry in computer vision. Cambridge university press. Google ScholarDigital Library
- He, K., Chang, H., and Sun, J. 2013. Rectangling panoramic images via warping. ACM Transactions on Graphics (TOG) 32, 4, 79. Google ScholarDigital Library
- Igarashi, T., Moscovich, T., and Hughes, J. F. 2005. As-rigid-as-possible shape manipulation. ACM transactions on Graphics (TOG) 24, 3, 1134--1141. Google ScholarDigital Library
- Jiang, W., and Gu, J. 2015. Video stitching with spatial-temporal content-preserving warping. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 42--48.Google Scholar
- Kopf, J., Lischinski, D., Deussen, O., Cohen-Or, D., and Cohen, M. 2009. Locally adapted projections to reduce panorama distortions. Computer Graphics Forum 28, 4, 1083--1089. Google ScholarDigital Library
- Krähenbühl, P., Lang, M., Hornung, A., and Gross, M. 2009. A system for retargeting of streaming video. ACM Transactions on Graphics (TOG) 28, 5, 126. Google ScholarDigital Library
- Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A., and Gross, M. 2010. Nonlinear disparity mapping for stereoscopic 3d. ACM Transactions on Graphics (TOG) 29, 4, 75. Google ScholarDigital Library
- Li, B., Heng, L., Koser, K., and Pollefeys, M. 2013. A multiple-camera system calibration toolbox using a feature descriptor-based calibration pattern. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 1301--1307.Google Scholar
- Li, S., Yuan, L., Sun, J., and Quan, L. 2015. Dual-feature warping-based motion model estimation. In Proceedings of the IEEE International Conference on Computer Vision, 4283--4291. Google ScholarDigital Library
- Lin, W.-Y., Liu, S., Matsushita, Y., Ng, T.-T., and Cheong, L.-F. 2011. Smoothly varying affine stitching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 345--352. Google ScholarDigital Library
- Lin, C.-C., Pankanti, S. U., Ramamurthy, K. N., and Ar-avkin, A. Y. 2015. Adaptive as-natural-as-possible image stitching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1155--1163.Google Scholar
- Liu, F., Gleicher, M., Jin, H., and Agarwala, A. 2009. Content-preserving warps for 3d video stabilization. ACM Transactions on Graphics (TOG) 28, 3, 44. Google ScholarDigital Library
- Liu, S., Yuan, L., Tan, P., and Sun, J. 2013. Bundled camera paths for video stabilization. ACM Transactions on Graphics (TOG) 32, 4, 78. Google ScholarDigital Library
- Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. International journal of computer vision 60, 2, 91--110. Google ScholarDigital Library
- Mei, C., and Rives, P. 2007. Single view point omnidirectional camera calibration from planar grids. In IEEE International Conference on Robotics and Automation, 3945--3950.Google Scholar
- Panozzo, D., Weber, O., and Sorkine, O. 2012. Robust image retargeting via axis-aligned deformation. Computer Graphics Forum 31, 2pt1, 229--236. Google ScholarDigital Library
- Perazzi, F., Sorkine-Hornung, A., Zimmer, H., Kaufmann, P., Wang, O., Watson, S., and Gross, M. 2015. Panoramic video from unstructured camera arrays. Computer Graphics Forum 34, 2, 57--68. Google ScholarDigital Library
- Richardt, C., Pritch, Y., Zimmer, H., and Sorkine-Hornung, A. 2013. Megastereo: Constructing high-resolution stereo panoramas. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1256--1263. Google ScholarDigital Library
- Shum, H.-Y., Chan, S.-C., and Kang, S. B. 2008. Image-based rendering. Springer Science & Business Media. Google ScholarDigital Library
- Szeliski, R. 2006. Image alignment and stitching: A tutorial. Foundations and Trends® in Computer Graphics and Vision 2, 1, 1--104. Google ScholarDigital Library
- Uyttendaele, M., Criminisi, A., Kang, S. B., Winder, S., Szeliski, R., and Hartley, R. 2004. Image-based interactive exploration of real-world environments. IEEE Computer Graphics and Applications 24, 3, 52--63. Google ScholarDigital Library
- Viola, P., and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, I--511.Google Scholar
- Vollmer, J., Mencl, R., and Mueller, H. 1999. Improved laplacian smoothing of noisy surface meshes. Computer Graphics Forum 18, 3, 131--138.Google ScholarCross Ref
- von Gioi, R. G., Jakubowicz, J., Morel, J.-M., and Randall, G. 2008. Lsd: A fast line segment detector with a false detection control. IEEE Transactions on Pattern Analysis & Machine Intelligence, 4, 722--732. Google ScholarDigital Library
- Wang, Y.-S., Fu, H., Sorkine, O., Lee, T.-Y., and Seidel, H.-P. 2009. Motion-aware temporal coherence for video resizing. ACM Transactions on Graphics (TOG) 28, 5, 127. Google ScholarDigital Library
- Wang, Y.-S., Lin, H.-C., Sorkine, O., and Lee, T.-Y. 2010. Motion-based video retargeting with optimized crop-and-warp. ACM Transactions on Graphics (TOG) 29, 4, 90. Google ScholarDigital Library
- Wang, Y.-S., Liu, F., Hsu, P.-S., and Lee, T.-Y. 2013. Spatially and temporally optimized video stabilization. IEEE Transactions on Visualization and Computer Graphics 19, 8, 1354--1361. Google ScholarDigital Library
- Xu, W., and Mulligan, J. 2013. Panoramic video stitching from commodity hdtv cameras. Multimedia systems 19, 5, 407--426. Google ScholarDigital Library
- Yildirim, G., and Susstrunk, S. 2015. Fasa: fast, accurate, and size-aware salient object detection. In Computer Vision-ACCV2014. 514--528.Google Scholar
- Zaragoza, J., Chin, T.-J., Tran, Q.-H., Brown, M. S., and Suter, D. 2014. As-projective-as-possible image stitching with moving dlt. IEEE Transactions on Pattern Analysis and Machine Intelligence 36,7, 1285--1298. Google ScholarDigital Library
- Zelnik-Manor, L., Peters, G., and Perona, P. 2005. Squaring the circle in panoramas. In Proceedings of the IEEE International Conference on Computer Vision, vol. 2, 1292--1299. Google ScholarDigital Library
- Zhang, F., and Liu, F. 2014. Parallax-tolerant image stitching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3262--3269. Google ScholarDigital Library
- Zhi, Q., and Cooperstock, J. R. 2012. Toward dynamic image mosaic generation with robustness to parallax. IEEE Transactions on Image Processing 21, 1, 366--378. Google ScholarDigital Library
Index Terms
- Rich360: optimized spherical representation from structured panoramic camera arrays
Recommendations
Effective omnistereo panorama video generation by deformable spheres
SIGGRAPH '17: ACM SIGGRAPH 2017 PostersWe presents an alternative and effective method for omnistereo panorama video generation by using deformable spheres. Deformable spheres use vertex based spherical mesh to represent the depth of a scene. Meanwhile, the spherical mesh was substituted for ...
Stabilization of panoramic videos from mobile multi-camera platforms
Wide field of view panoramic videos have recently become popular due to the availability of high resolution displays. These panoramic videos are generated by stitching video frames captured from a panoramic video acquisition system, typically comprising ...
A multicamera setup for generating stereo panoramic video
Traditional visual communication systems convey only two-dimensional (2-D) fixed field-of-view (FOV) video information. The viewer is presented with a series of flat, nonstereoscopic images, which fail to provide a realistic sense of depth. Furthermore, ...
Comments