Rosália G. Schneider
Abstract
We introduce an approach for sketch classification based on Fisher vectors that significantly outperforms existing techniques. For the TU-Berlin sketch benchmark [Eitz et al. 2012a], our recognition rate is close to human performance on the same task. Motivated by these results, we propose a different benchmark for the evaluation of sketch classification algorithms. Our key idea is that the relevant aspect when recognizing a sketch is not the intention of the person who made the drawing, but the information that was effectively expressed. We modify the original benchmark to capture this concept more precisely and, as such, to provide a more adequate tool for the evaluation of sketch classification techniques. Finally, we perform a classification-driven analysis which is able to recover semantic aspects of the individual sketches, such as the quality of the drawing and the importance of each part of the sketch for the recognition.
- Barla, P., Thollot, J., and Sillion, F. X. 2005. Geometric clustering for line drawing simplification. In ACM SIGGRAPH 2005 Sketches, ACM, New York, NY, USA, SIGGRAPH '05. Google Scholar
Digital Library
- Cao, X., Zhang, H., Liu, S., Guo, X., and Lin, L. 2013. Sym-fish: A symmetry-aware flip invariant sketch histogram shape descriptor. In IEEE International Conference on Computer Vision (ICCV). Google ScholarDigital Library
- Csurka, G., Dance, C. R., Fan, L., Willamowski, J., and Bray, C. 2004. Visual categorization with bags of keypoints. In In Workshop on Statistical Learning in Computer Vision, ECCV, 1--22.Google Scholar
- Davis, J., Agrawala, M., Chuang, E., Popović, Z., and Salesin, D. 2003. A sketching interface for articulated figure animation. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA '03, 320--328. Google ScholarDigital Library
- Donmez, N., and Singh, K. 2012. Concepture: A regular language based framework for recognizing gestures with varying and repetitive patterns. In Proceedings of the International Symposium on Sketch-Based Interfaces and Modeling, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SBIM '12, 29--37. Google ScholarDigital Library
- Eitz, M., Hildebrand, K., Boubekeur, T., and Alexa, M. 2011. Sketch-based image retrieval: Benchmark and bag-of-features descriptors. IEEE Transactions on Visualization and Computer Graphics 17, 11 (Nov.), 1624--1636. Google ScholarDigital Library
- Eitz, M., Hays, J., and Alexa, M. 2012. How do humans sketch objects? ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4, 44:1--44:10. Google ScholarDigital Library
- Eitz, M., Richter, R., Boubekeur, T., Hildebrand, K., and Alexa, M. 2012. Sketch-based shape retrieval. ACM Trans. Graph. 31, 4 (July), 31:1--31:10. Google ScholarDigital Library
- Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., and Zisserman, A. 2010. The pascal visual object classes (voc) challenge. International Journal of Computer Vision 88, 2 (June), 303--338. Google ScholarDigital Library
- Hammond, T., and Davis, R. 2007. Ladder, a sketching language for user interface developers. In ACM SIGGRAPH 2007 Courses, ACM, New York, NY, USA, SIGGRAPH '07. Google ScholarDigital Library
- Hearst, M. A., Dumais, S., Osman, E., Platt, J., and Scholkopf, B. 1998. Support vector machines. Intelligent Systems and their Applications, IEEE 13, 4, 18--28. Google ScholarDigital Library
- Hoiem, D., Chodpathumwan, Y., and Dai, Q. 2012. Diagnosing error in object detectors. In Proceedings of the 12th European Conference on Computer Vision - Volume Part III, Springer-Verlag, Berlin, Heidelberg, ECCV'12, 340--353. Google ScholarDigital Library
- Jaakkola, T., and Haussler, D. 1998. Exploiting generative models in discriminative classifiers. In In Advances in Neural Information Processing Systems 11, MIT Press, 487--493. Google ScholarDigital Library
- LaViola, Jr., J. J., and Zeleznik, R. C. 2004. Mathpad2: A system for the creation and exploration of mathematical sketches. ACM Trans. Graph. 23, 3 (Aug.), 432--440. Google ScholarDigital Library
- Lazebnik, S., Schmid, C., and Ponce, J. 2006. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, vol. 2, 2169--2178. Google ScholarDigital Library
- Li, Y., Song, Y.-Z., and Gong, S. 2013. Sketch recognition by ensemble matching of structured features. In In British Machine Vision Conference (BMVC).Google Scholar
- Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 91--110. Google ScholarDigital Library
- Olsen, L., Samavati, F. F., Sousa, M. C., and Jorge, J. A. 2009. Sketch-based modeling: A survey. Computers & Graphics 33, 1, 85--103. Google ScholarDigital Library
- Ouyang, T. Y., and Davis, R. 2011. Chemink: A natural real-time recognition system for chemical drawings. In Proceedings of the 16th International Conference on Intelligent User Interfaces, ACM, New York, NY, USA, IUI '11, 267--276. Google ScholarDigital Library
- Perronnin, F., Liu, Y., Sanchez, J., and Poirier, H. 2010. Large-scale image retrieval with compressed fisher vectors. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, 3384--3391.Google Scholar
- Rubine, D. 1991. Specifying gestures by example. In Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, NY, USA, SIGGRAPH '91, 329--337. Google ScholarDigital Library
- Sánchez, J., Perronnin, F., Mensink, T., and Verbeek, J. 2013. Image classification with the fisher vector: Theory and practice. International Journal of Computer Vision 105, 3, 222--245. Google ScholarDigital Library
- Schmidt, R., Wyvill, B., Sousa, M. C., and Jorge, J. A. 2006. Shapeshop: Sketch-based solid modeling with blobtrees. In ACM SIGGRAPH 2006 Courses, ACM, New York, NY, USA, SIGGRAPH '06. Google ScholarDigital Library
- Sezgin, T. M. 2001. Sketch based interfaces: Early processing for sketch understanding. In Proceedings of PUI-2001. NY, ACM Press. Google ScholarDigital Library
- Shesh, A., and Chen, B. 2008. Efficient and dynamic simplification of line drawings. Comput. Graph. Forum 27, 2, 537--545.Google ScholarCross Ref
- Sivic, J., and Zisserman, A. 2003. Video Google: A text retrieval approach to object matching in videos. In Proceedings of the International Conference on Computer Vision, vol. 2, 1470--1477. Google ScholarDigital Library
- Sutherland, I. E. 1964. Sketch pad a man-machine graphical communication system. In Proceedings of the SHARE Design Automation Workshop, ACM, New York, NY, USA, DAC '64, 6.329--6.346. Google ScholarDigital Library
- Torralba, A., and Efros, A. A. 2011. Unbiased look at dataset bias. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, Washington, DC, USA, CVPR '11, 1521--1528. Google ScholarDigital Library
- Vedaldi, A., and Fulkerson, B., 2008. VLFeat: An open and portable library of computer vision algorithms. http://www.vlfeat.org/.Google Scholar
Index Terms
- Sketch classification and classification-driven analysis using Fisher vectors
Recommendations
Context-based sketch classification
Expressive '18: Proceedings of the Joint Symposium on Computational Aesthetics and Sketch-Based Interfaces and Modeling and Non-Photorealistic Animation and RenderingWe present a novel context-based sketch classification framework using relations extracted from scene images. Most of existing methods perform sketch classification by considering individually sketched objects and often fail to identify their correct ...
Uncertainty in Communication with a Sketch
Sketches are one of the main tools for the communication of design ideas during the conceptual phase of the design process. In design communication, one of the major problems is the uncertainty associated with imprecisely defined sketches. There is a need ...
Automatic understanding of sketch maps using context-aware classification
We present the first comprehensive system for offline classifying sketch map objects.We created a database of labeled sketch maps for training and evaluation purposes.Context-awareness improves the classification of sketch map objects greatly. Sketching ...
Comments