ABSTRACT
We introduce a novel workflow for stereoscopic 2D to 3D conversion in which the user "paints" depth onto a 2D image via sparse scribbles, instantaneously receiving intuitive 3D feedback. This workflow is enabled by the introduction of a discontinuous warping technique that creates stereoscopic pairs from sparse, possibly erroneous user input. Our method assumes a piecewise continuous depth representation, preserving visual continuity in most areas, while creating sharp depth discontinuities at important object boundaries. As opposed to prior work that relies strictly on a per pixel depth map, our scribbles are processed as soft constraints in a global solve and operate entirely on image domain disparity, allowing for relaxed input requirements. This formulation also allows us to simultaneously compute a disparity-and-content-aware stretching of background areas to automatically fill disoccluded regions with valid stereo information. We tightly integrate all steps of stereo content conversion into a single optimization framework, which can then be solved on a GPU at interactive rates. The instant feedback received while painting depth allows even untrained users to quickly create compelling 3D scenes from single-view footage.
- {AS07} Avidan S., Shamir A.: Seam carving for content-aware image resizing. ACM Trans. Graph. 26, 3 (2007), 10. 3 Google ScholarDigital Library
- {CAA09} Carroll R., Agrawala M., Agarwala A.: Optimizing content-preserving projections for wide-angle images. ACM Trans. Graph. 28, 3 (2009). 2 Google ScholarDigital Library
- {CW93} Chen S. E., Williams L.: View interpolation for image synthesis. In SIGGRAPH (1993), pp. 279--288. 2 Google ScholarDigital Library
- {DRE* 10} Didyk P., Ritschel T., Eisemann E., Myszkowski K., Seidel H.-P.: Adaptive image-space stereo view synthesis. In VMV Workshop (Siegen, Germany, 2010), pp. 299--306. 2Google Scholar
- {GWCO09} Guttmann M., Wolf L., Cohen-Or D.: Semiautomatic stereo extraction from video footage. In Computer Vision, 2009 IEEE 12th International Conference on (Oct 2009), pp. 136--142. 2Google ScholarCross Ref
- {GZMT10} Goferman S., Zelnik-Manor L., Tal A.: Context-aware saliency detection. In CVPR (2010), pp. 2376--2383. 4Google ScholarCross Ref
- {HW09} Harris J. M., Wilcox L. M.: The role of monocularly visible regions in depth and surface perception. Vision Research 49, 22 (2009), 2666--2685. Vision Research Special Issue - Vision Research Reviews. 5Google ScholarCross Ref
- {KKS07} Knorr S., Kunter M., Sikora T.: Super-resolution stereo- and multi-view synthesis from monocular video sequences. In 3DIM (2007), pp. 55--64. 2 Google ScholarDigital Library
- {KLHG09} Krähenbühl P., Lang M., Hornung A., Gross M. H.: A system for retargeting of streaming video. ACM Trans. Graph. 28, 5 (2009). 6, 7 Google ScholarDigital Library
- {LHW* 10} Lang M., Hornung A., Wang O., Poulakos S., Smolic A., Gross M. H.: Nonlinear disparity mapping for stereoscopic 3d. ACM Trans. Graph. 29, 4 (2010). 2, 3, 4 Google ScholarDigital Library
- {LLW04} Levin A., Lischinski D., Weiss Y.: Colorization using optimization. ACM Trans. Graph. 23, 3 (2004), 689--694. 3 Google ScholarDigital Library
- {RGSS10} Rubinstein M., Gutierrez D., Sorkine O., Shamir A.: A comparative study of image retargeting. ACM Trans. Graph. 29, 5 (2010), to appear. 5, 7 Google ScholarDigital Library
- {SCSI08} Simakov D., Caspi Y., Shechtman E., Irani M.: Summarizing visual data using bidirectional similarity. In CVPR (2008). 3Google Scholar
- {SS09} Shamir A., Sorkine O.: Visual media retargeting. In SIGGRAPH ASIA Courses (2009). 2 Google ScholarDigital Library
- {SSJ* 10} Sýkora D., Sedlacek D., Jinchao S., Dingliana J., Collins S.: Adding depth to cartoons using sparse depth (in)equalities. Comput. Graph. Forum 29, 2 (2010), 615--623. 2Google ScholarCross Ref
- {vdHDT*07} van den Hengel A., Dick A. R., Thormählen T., Ward B., Torr P. H. S.: Videotrace: rapid interactive scene modelling from video. ACM Trans. Graph. 26, 3 (2007), 86. 2 Google ScholarDigital Library
- {WKB11} Ward B., Kang S. B., Bennett E. P.: Depth director: A system for adding depth to movies. IEEE Computer Graphics and Applications 31 (2011), 36--48. 2 Google ScholarDigital Library
- {WLKT09} Wei L.-Y., Lefebvre S., Kwatra V., Turk G.: State of the art in example-based texture synthesis. In Eurographics 2009, State of the Art Report, EG-STAR (2009), Eurographics Association. 3Google Scholar
- {ZDPSS01} Zhang L., Dugas-Phocion G., Samson J.-S., Seitz S. M.: Single view modeling of free-form scenes. In CVPR (1) (2001), pp. 990--997. 2Google Scholar
- {ZHQ*07} Zhang G., Hua W., Qin X., Wong T.-T., Bao H.: Stereoscopic video synthesis from a monocular video. IEEE Trans. Vis. Comput. Graph. 13, 4 (2007), 686--696. 2 Google ScholarDigital Library
- {ZKU*04} Zitnick C. L., Kang S. B., Uyttendaele M., Winder S. A. J., Szeliski R.: High-quality video view interpolation using a layered representation. ACM Trans. Graph. 23, 3 (2004), 600--608. 2 Google ScholarDigital Library
- {ZPvBG02} Zwicker M., Pfister H., van Baar J., Gross M. H.: Ewa splatting. IEEE Trans. Vis. Comput. Graph. 8, 3 (2002), 223--238. 6 Google ScholarDigital Library
Index Terms
- StereoBrush: interactive 2D to 3D conversion using discontinuous warps
Recommendations
Dense 3-D Reconstruction of an Outdoor Scene by Hundreds-Baseline Stereo Using a Hand-Held Video Camera
Three-dimensional (3-D) models of outdoor scenes are widely used for object recognition, navigation, mixed reality, and so on. Because such models are often made manually with high costs, automatic 3-D reconstruction has been widely investigated. In ...
Projection defocus analysis for scene capture and image display
In order to produce bright images, projectors have large apertures and hence narrow depths of field. In this paper, we present methods for robust scene capture and enhanced image display based on projection defocus analysis. We model a projector's ...
Catadioptric Stereo Using Planar Mirrors
By using mirror reflections of a scene, stereo images can be captured with a single camera (catadioptric stereo). In addition to simplifying data acquisition single camera stereo provides both geometric and radiometric advantages over traditional two ...
Comments