Vivek Kwatra
Irfan Essa
Abstract
We present a novel technique for texture synthesis using optimization. We define a Markov Random Field (MRF)-based similarity metric for measuring the quality of synthesized texture with respect to a given input sample. This allows us to formulate the synthesis problem as minimization of an energy function, which is optimized using an Expectation Maximization (EM)-like algorithm. In contrast to most example-based techniques that do region-growing, ours is a joint optimization approach that progressively refines the entire texture. Additionally, our approach is ideally suited to allow for controllable synthesis of textures. Specifically, we demonstrate controllability by animating image textures using flow fields. We allow for general two-dimensional flow fields that may dynamically change over time. Applications of this technique include dynamic texturing of fluid animations and texture-based flow visualization.
Supplemental Material
- Ashikhmin, M. 2001. Synthesizing natural textures. 2001 ACM Symposium on Interactive 3D Graphics (March), 217--226. Google Scholar
Digital Library
- Bhat, K. S., Seitz, S. M., Hodgins, J. K., and Khosla, P. K. 2004. Flow-based video synthesis and editing. ACM Transactions on Graphics (SIGGRAPH 2004) 23, 3 (August). Google ScholarDigital Library
- Bregler, C., Covell, M., and Slaney, M. 1997. Video rewrite: Driving visual speech with audio. Proceedings of SIGGRAPH 97 (August), 353--360. ISBN 0-89791-896-7. Held in Los Angeles, California. Google ScholarDigital Library
- Cohen, M. F., Shade, J., Hiller, S., and Deussen, O. 2003. Wang tiles for image and texture generation. ACM Transactions on Graphics, SIGGRAPH 2003 22, 3, 287--294. Google ScholarDigital Library
- Coleman, D., Holland, P., Kaden, N., Klema, V., and Peters, S. C. 1980. A system of subroutines for iteratively reweighted least squares computations. ACM Trans. Math. Softw. 6, 3, 327--336. Google ScholarDigital Library
- DeBonet, J. S. 1997. Multiresolution sampling procedure for analysis and synthesis of texture images. Proceedings of ACM SIGGRAPH 97 (August), 361--368. Google ScholarDigital Library
- Dellaert, F., Kwatra, V., and Oh, S. M. 2005. Mixture trees for modeling and fast conditional sampling with applications in vision and graphics. In IEEE Computer Vision and Pattern Recognition. Google ScholarDigital Library
- Doretto, G., and Soatto, S. 2003. Editable dynamic textures. In IEEE Computer Vision and Pattern Recognition, II: 137--142.Google Scholar
- Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. Proceedings of SIGGRAPH 2001, 341--346. Google ScholarDigital Library
- Efros, A., and Leung, T. 1999. Texture synthesis by non-parametric sampling. In International Conference on Computer Vision, 1033--1038. Google ScholarDigital Library
- Elkan, C. 2003. Using the triangle inequality to accelerate k-means. In International Conference on Machine Learning.Google Scholar
- Ezzat, T., Geiger, G., and Poggio, T. 2002. Trainable videorealistic speech animation. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques, ACM Press, 388--398. Google ScholarDigital Library
- Fitzgibbon, A., Wexler, Y., and Zisserman, A. 2003. Image-based rendering using image-based priors. In International Conference on Computer Vision. Google ScholarDigital Library
- Freeman, W. T., Jones, T. R., and Pasztor, E. C. 2002. Example-based super-resolution. IEEE Comput. Graph. Appl. 22, 2, 56--65. Google ScholarDigital Library
- Heeger, D. J., and Bergen, J. R. 1995. Pyramid-based texture analysis/synthesis. Proceedings of ACM SIGGRAPH 95 (August), 229--238. Google ScholarDigital Library
- Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. Proceedings of SIGGRAPH 2001 (August), 327--340. ISBN 1-58113-292-1. Google ScholarDigital Library
- Johnson, S. C. 1967. Hierarchical clustering schemes. Psychometrika 2, 241--254.Google ScholarCross Ref
- Jojic, N., Frey, B., and Kannan, A. 2003. Epitomic analysis of appearance and shape. In International Conference on Computer Vision. Google ScholarDigital Library
- Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics, SIGGRAPH 2003 22, 3 (July), 277--286. Google ScholarDigital Library
- Liang, L., Liu, C., Xu, Y.-Q., Guo, B., and Shum, H.-Y. 2001. Real-time texture synthesis by patch-based sampling. ACM Transactions on Graphics Vol. 20, No. 3 (July), 127--150. Google ScholarDigital Library
- McLachlan, G., and Krishnan, T. 1997. The EM algorithm and extensions. Wiley series in probability and statistics. John Wiley & Sons.Google Scholar
- Neyret, F. 2003. Advected textures. Symposium on Computer Animation'03 (July). Google ScholarDigital Library
- Paget, R., and Longstaff, I. D. 1998. Texture synthesis via a non-causal nonparametric multiscale markov random field. IEEE Transactions on Image Processing 7, 6 (June), 925--931. Google ScholarDigital Library
- Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Transactions on Graphics, SIGGRAPH 2003 22, 3, 313--318. Google ScholarDigital Library
- Portilla, J., and Simoncelli, E. P. 2000. A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal of Computer Vision 40, 1 (October), 49--70. Google ScholarDigital Library
- Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. Proceedings of ACM SIGGRAPH 2000 (July), 489--498. Google ScholarDigital Library
- Wei, L.-Y., and Levoy, M. 2000. Fast texture synthesis using tree-structured vector quantization. Proceedings of ACM SIGGRAPH 2000 (July), 479--488. ISBN 1-58113-208-5. Google ScholarDigital Library
- Wei, L.-Y., and Levoy, M. 2002. Order-independent texture synthesis. Tech. Rep. TR-2002-01, Stanford University CS Department.Google Scholar
- Wexler, Y., Shechtman, E., and Irani, M. 2004. Space-time video completion. In CVPR 2004, 120--127.Google Scholar
- Wu, Q., and Yu, Y. 2004. Feature matching and deformation for texture synthesis. ACM Transactions on Graphics (SIGGRAPH 2004) (August). Google ScholarDigital Library
- Zhang, J., Zhou, K., Velho, L., Guo, B., and Shum, H.-Y. 2003. Synthesis of progressively-variant textures on arbitrary surfaces. ACM Transactions on Graphics, SIGGRAPH 2003 22, 3, 295--302. Google ScholarDigital Library
- Zhang, E., Mischaikow, K., and Turk, G. 2004. Vector field design on surfaces. Tech. Rep. 04--16, Georgia Institute of Technology.Google Scholar
Index Terms
- Texture optimization for example-based synthesis
Recommendations
Image quilting for texture synthesis and transfer
SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniquesWe present a simple image-based method of generating novel visual appearance in which a new image is synthesized by stitching together small patches of existing images. We call this process image quilting. First, we use quilting as a fast and very ...
Texture optimization for example-based synthesis
SIGGRAPH '05: ACM SIGGRAPH 2005 PapersWe present a novel technique for texture synthesis using optimization. We define a Markov Random Field (MRF)-based similarity metric for measuring the quality of synthesized texture with respect to a given input sample. This allows us to formulate the ...
Fast example-based surface texture synthesis via discrete optimization
We synthesize and animate general texture patterns over arbitrary 3D mesh surfaces. The animation is controlled by flow fields over the target mesh, and the texture can be arbitrary user input as long it satisfies the Markov-Random-Field assumptions. We ...
Comments