Anton van den Hengel
Abstract
VideoTrace is a system for interactively generating realistic 3D models of objects from video---models that might be inserted into a video game, a simulation environment, or another video sequence. The user interacts with VideoTrace by tracing the shape of the object to be modelled over one or more frames of the video. By interpreting the sketch drawn by the user in light of 3D information obtained from computer vision techniques, a small number of simple 2D interactions can be used to generate a realistic 3D model. Each of the sketching operations in VideoTrace provides an intuitive and powerful means of modelling shape from video, and executes quickly enough to be used interactively. Immediate feedback allows the user to model rapidly those parts of the scene which are of interest and to the level of detail required. The combination of automated and manual reconstruction allows VideoTrace to model parts of the scene not visible, and to succeed in cases where purely automated approaches would fail.
Supplemental Material
- Agarwala, A., Hertzmann, A., Salesin, D. H., and Seitz, S. M. 2004. Keyframe-based tracking for rotoscoping and animation. ACM Trans. Graphics 23, 3, 584--591. Google Scholar
Digital Library
- Boykov, Y., and Kolmogorov, V. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Analysis and Machine Intelligence 26, 9, 1124--1137. Google ScholarDigital Library
- Dick, A., Torr, P., and Cipolla, R. 2004. Modelling and interpretation of architecture from several images. International Journal of Computer Vision 60, 2 (November), 111--134. Google ScholarDigital Library
- Eos Systems, 2005. Photomodeler: A commercial photogrammetry product http://www.photomodeler.com.Google Scholar
- Gibson, S., Hubbold, R., Cook, J., and Howard, T. 2003. Interactive reconstruction of virtual environments from video sequences. Computers & Graphics 27, 2 (April), 293--301.Google Scholar
- Karpenko, O. A., and Hughes, J. F. 2006. Smoothsketch: 3D free-form shapes from complex sketches. In ACM SIGGraph, Computer Graphics. Google ScholarDigital Library
- Müller, P., Wonka, P., Haegler, S., Ulmer, A., and Gool, L. V. 2006. Procedural modeling of buildings. ACM Transactions on Graphics 25, 3, 614--623. Google ScholarDigital Library
- Niem, W., and Broszio, H. 1995. Mapping texture from multiple camera views onto 3d-object models for computer animation. In Proc. International Workshop on Stereoscopic and Three Dimensional Imaging, 99--105.Google Scholar
- Piegl, L., and Tiller, W. 1997. The NURBS book (2nd ed.). Springer-Verlag New York, Inc., New York, NY, USA. Google ScholarDigital Library
- Pollefeys, M., Gool, L. V., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., and Koch, R. 2004. Visual modeling with a hand-held camera. International Journal of Computer Vision 59, 3, 207--232. Google ScholarDigital Library
- Quan, L., Tan, P., Zeng, G., Yuan, L., Wang, J., and Kang, S. B. 2006. Image-based plant modeling. ACM Transactions on Graphics 25, 3, 599--604. Google ScholarDigital Library
- Ren, X., and Malik, J. 2003. Learning a classification model for segmentation. In Proc. 9th Int'l. Conf. Computer Vision, vol. 1, 10--17. Google ScholarDigital Library
- Shpitalni, M., and Lipson, H. 1997. Classification of sketch strokes and corner detection using conic sections and adaptive clustering. Trans. of the ASME, Journal of Mechanical Design 119, 1, 131--135.Google ScholarCross Ref
- Taylor, C., Debevec, P., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. ACM SIGGraph, Computer Graphics, 11--20. Google ScholarDigital Library
- Thormählen, T. 2006. Zuverlässige Schätzung der Kamerabewegung aus einer Bildfolge. PhD thesis, University of Hannover, related software 'Voodoo Camera Tracker' can be downloaded from http://www.digilab.uni-hannover.de.Google Scholar
- van den Hengel, A., Dick, A., Thormahlen, T., Torr, P., and Ward, B. 2006. Building models of regular scenes from structure and motion. In Proc. 17th British Machine Vision Conference, I:197--206.Google Scholar
- Wei, Y., Ofek, E., Quan, L., and Shum, H.-Y. 2005. Modeling hair from multiple views. In ACM SIGGraph, Computer Graphics, ACM Press, New York, NY, USA, 816--820. Google ScholarDigital Library
- Wilczkowiak, M., Sturm, P., and Boyer, E. 2005. Using geometric constraints through parallelepipeds for calibration and 3d modeling. IEEE Trans. Pattern Analysis and Machine Intelligence 27, 2 (February), 194--207. Google ScholarDigital Library
Index Terms
- VideoTrace: rapid interactive scene modelling from video
Recommendations
VideoTrace: rapid interactive scene modelling from video
SIGGRAPH '07: ACM SIGGRAPH 2007 papersVideoTrace is a system for interactively generating realistic 3D models of objects from video---models that might be inserted into a video game, a simulation environment, or another video sequence. The user interacts with VideoTrace by tracing the shape ...
A shape hierarchy for 3D modelling from video
GRAPHITE '07: Proceedings of the 5th international conference on Computer graphics and interactive techniques in Australia and Southeast AsiaThis paper describes an interactive method for generating a model of a scene from image data. The method uses the camera parameters and point cloud typically generated by structure-and-motion estimation as a starting point for developing a higher level ...
Multi-view dense 3D modelling of untextured objects from a moving projector-cameras system
Structured light methods achieve 3D modelling by observing with a camera system, a known pattern projected on the scene. The main drawback of single projection structured light methods is that moving the projector changes significatively the appearance ...
Reviews
Vishnu Vardhan Makkapati
The generation of realistic three-dimensional (3D) models of objects from video has been considered by the computer vision and graphics communities for several years. Several automatic methods have been proposed, but they suffer from ambiguities in image data, degeneracy of camera motion, and lack of appropriate feature points on the model surface. This paper proposes a system, called VideoTrace, to solve these problems by using user interaction and computer vision techniques. The input video sequence is preprocessed to obtain a 3D point cloud using structure and motion analysis, and is also segmented such that a large number of small clusters of adjacent pixels have the same color. The 3D point cloud is overlaid on top of a frame from the video sequence, and a user can trace an object on any frame of the video. The inaccuracies in tracing are overcome by automatic refinement, and a polygonal face of the object is modeled by joining the line segments drawn by the user. Nonuniform rational B-splines are used to model the surface, and extrusions are appropriately handled. The system uses mirror planes to capture global properties of the surface, such as regularity and symmetry. The scheme has been evaluated using a wide variety of video sequences, and the results are promising. VideoTrace combines the power of automatic and manual modeling systems. The system may be of help to users interested in accurate 3D reconstruction from video. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments