nach oben

Machine Vision and Applications

Erschienen in:

01.09.2018 | Original Paper

A survey of sketch-based image retrieval

verfasst von: Yi Li, Wenzhao Li

Erschienen in: Machine Vision and Applications | Ausgabe 7/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Sketch-based image retrieval (SBIR) has been studied since the early 1990s and has drawn more and more interest recently. Yet, a comprehensive review of the SBIR field is still absent. This survey tries to fill in this gap by reviewing the representative papers studying the SBIR problem. More importantly, this survey tries to answer two important questions which are generally not well discussed: what are the objectives of SBIR, and what is the general methodology of SBIR? The reviewed papers are organized in a chronological way and analyzed by answering these two important questions. As a novel trend, fine-grained SBIR has become the main topic for the recent research. The discussion on it is also integrated. From this survey, we hope that different perspectives can be observed, common values can be discovered and new ideas can be inspired.

Vorheriger Artikel Detection of reactions to sound via gaze and global eye motion analysis using camera streaming

Nächster Artikel Key-frame selection for automatic summarization of surveillance videos: a method of multiple change-point detection

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Abdulbaqi, H.A., Sulong, G., Hashem, S.H.: A sketch based image retrieval: a rreview of literature. J. Theor. Appl. Inf. Technol. 63(1), 158–167 (2014)

Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24(4), 509–522 (2002)CrossRef

Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975)CrossRef

Birari, D.R., Shinde, J.: Survey on sketch based image retrieval. Int. J. Adv. Res. Comput. Commun. Eng. 4(12), 513–516 (2015)

Candemir, S., Borovikov, E., Santosh, K.C., Antani, S.K., Thoma, G.R.: Rsilc: rotation- and scale-invariant, line-based color-aware descriptor. Image Vis. Comput. 42, 1–12 (2015)CrossRef

Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)CrossRef

Cao, Y., Wang, C., Zhang, L., Zhang, L.: Edgel index for large-scale sketch-based image search. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 761–768 (2011)

Chalechale, A., Naghdy, G., Mertins, A.: Sketch-based image matching using angular partitioning. IEEE Trans. Syst. Man Cybern. 35(1), 28–41 (2005)CrossRef

Chans, Y., Lei, Z., Lopresti, D.P., Kung, S.Y.: A feature-based approach for image retrieval by sketch. In: SPIE International Symposium on Voice, Video and Data Communications, pp. 220–231 (1997)

10.

Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: Proceedings of the IEEE Computer Conference on Computer Vision and Pattern Recognition, pp. 539–546 (2005)

11.

Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 886–893 (2005)

12.

Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image retrieval: ideas, influences, and trends of the new age. ACM Comput. Surv. 40(2), 5:1–5:60 (2008)CrossRef

13.

Del Bimbo, A., Pala, P.: Visual image retrieval by elastic matching of user sketches. IEEE Trans. Pattern Anal. Mach. Intell. 19(2), 121–132 (1997)CrossRef

14.

Del Bimbo, A., Pala, P., Santini, S.: Visual image retrieval by elastic deformation of object sketches. In: IEEE Symposium on Visual Languages, pp. 216–223 (1994)

15.

Del Bimbo, A., Pala, P., Santini, S.: Image retrieval by elastic matching of shapes and image patterns. In: IEEE International Conference on Multimedia Computing and Systems, pp. 215–218 (1996)

16.

Eitz, M., Hays, J., Alexa, M.: How do humans sketch objects? ACM Trans. Graph. (Proceedings of SIGGRAPH) 31(4), 44:1–44:10 (2012)

17.

Eitz, M., Hildebrand, K., Boubekeur, T., Alexa, M.: A descriptor for large scale image retrieval based on sketched feature lines. In: Eurographics Symposium on Sketch-Based Interfaces and Modeling, pp. 29–36 (2009)

18.

Eitz, M., Hildebrand, K., Boubekeur, T., Alexa, M.: An evaluation of descriptors for large-scale image retrieval from sketched feature lines. Comput. Graph. 34(5), 482–498 (2010)CrossRef

19.

Eitz, M., Hildebrand, K., Boubekeur, T., Alexa, M.: Sketch-based image retrieval: Benchmark and bag-of-features descriptors. IEEE Trans. Vis. Comput. Graph. 17(11), 1624–1636 (2011)CrossRef

20.

Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2010 (VOC2010) Results. http://www.pascal-network.org/challenges/VOC/voc2010/workshop/index.html

21.

Faloutsos, C., Barber, R., Flickner, M., Hafner, J., Niblack, W., Petkovic, D., Equitz, W.: Efficient and effective querying by image content. J. Intell. Inf. Syst. 3(3–4), 231–262 (1994)CrossRef

22.

Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)CrossRef

23.

Fukunage, K., Narendra, P.M.: A branch and bound algorithm for computing k-nearest neighbors. IEEE Trans. Comput. 24(7), 750–753 (1975)CrossRef

24.

Gaidhani, P.A., Bagal, S.: Survey paper on sketch based and content based image retrieval. Int. J. Sci. Res. 4(12) (2015)

25.

Gao, Y., Leung, M.K.: Face recognition using line edge map. IEEE Trans. Pattern Anal. Mach. Intell. 24(6), 764–779 (2002)CrossRef

26.

Hirata, K., Kato, T.: Query by visual example—content based image retrieval. In: International Conference on Extending Database Technology: Advances in Database Technology, pp. 56–71 (1992)

27.

Hu, R., Barnard, M., Collomosse, J.: Gradient field descriptor for sketch based retrieval and localization. In: IEEE International Conference on Image Processing, pp. 1025–1028 (2010)

28.

Hu, R., Collomosse, J.: A performance evaluation of gradient field hog descriptor for sketch based image retrieval. Comput. Vis. Image Underst. 117, 790–806 (2013)CrossRef

29.

Hu, R., Wang, T., Collomosse, J.: A bag-of-regions approach to sketch based image retrieval. In: IEEE International Conference on Image Processing, pp. 3661–3664 (2011)

30.

Joachims, T.: Optimizing search engines using clickthrough data. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 133–142 (2002)

31.

Kato, T., Kurita, T., Otsu, N., Kyoji, H.: A sketch retrieval method for full color image database-query by visual example. In: IAPR International Conference on Computer Vision and Applications, pp. 530–533 (1992)

32.

Kendall, M.G.: A new measure of rank correlation. Biometrika 30(1/2), 81–93 (1938)CrossRef

33.

Knutsson, H.: Representing local structure using tensors. In: Scandinavian Conference on Image Analysis, pp. 244–251 (1989)

34.

Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

35.

Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2169–2178 (2006)

36.

Li, Y., Hospedales, T.M., Song, Y.Z., Gong, S.: Fine-grained sketch-based image retrieval by matching deformable part models. In: In British Machine Vision Conference (BMVC) (2014)

37.

Li, Y., Song, Y.Z., Hospedales, T., Gong, S.: Free-hand sketch synthesis with deformable stroke models. Int. J. Comput. Vis. 122(1), 169–190 (2017)MathSciNetCrossRef

38.

Lowe, D.G.: Object recognition from local scale-invariant features. In: IEEE International Conference on Computer Vision, 20–25 September, 1999, Kerkyra, Corfu, Greece, Proceedings, vol. 2, pp. 1150–1157 (1999)

39.

Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)MathSciNetCrossRef

40.

Manjunath, B.S., Salembier, P., Sikora, t.: Introduction to MPEG-7: Multimedia Content Description Interface. Wiley, New York (2002)

41.

Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: IEEE International Conference on Computer Vision Theory and Applications, pp. 331–340 (2009)

42.

Niblack, W., Barber, R., Equitz, W., Flickner, M., Glasman, E.H., Petkovic, D., Yanker, P., Faloutsos, C., Taubin, G.: The qbic project: querying images by content, using color, texture, and shape. In: Storage and Retrieval for Image and Video Databases (SPIE), pp. 173–187 (1993)

43.

Opelt, A., Pinz, A., Zisserman, A.: Learning an alphabet of shape and appearance for multi-class object detection. Int. J. Comput. Vis. 80(1), 16–44 (2008)CrossRef

44.

Parui, S., Mittal, A.: Similarity-invariant sketch-based image retrieval in large databases. In: European Conference on Computer Vision, pp. 398–414 (2014)

45.

Rajendran, R.K., Chang, S.F.: Image retrieval with sketches and compositions. In: IEEE International Conference on Multimedia and Expo, pp. 717–720 (2000)

46.

Rui, Y., Huang, T.S., Chang, S.F.: Image retrieval: current techniques, promising directions, and open issues. J. Vis. Commun. Image Represent. 10(1), 39–62 (1999)CrossRef

47.

Sangkloy, P., Burnell, N., Ham, C., Hays, J.: The sketchy database: Learning to retrieve badly drawn bunnies. In: SIGGRAPH (2016)

48.

Santosh, K., Lamiroy, B., Wendling, L.: DTW-radon-based shape descriptor for pattern recognition. Int. J. Pattern Recogn. Artif. Intell. 27(3) (2013)MathSciNetCrossRef

49.

Santosh, K.C., Lamiroy, B., Wendling, L.: Symbol recognition using spatial relations. Pattern Recogn. Lett. 33(3), 331–341 (2012)CrossRef

50.

Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

51.

Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in videos. In: IEEE International Conference on Computer Vision, pp. 1470–1477 (2003)

52.

Sobel, I., Feldman, G.: An Isotropic 3x3 Image Gradient Operator for Image Processing. In: Pattern Classification and Scene Analysis, pp. 271–272 (1973)

53.

Song, J., Song, Y., Xiang, T., Hospedales, T., Ruan, X.: Deep multi-task attribute-based ranking for fine-grained sketch-based image retrieval. In: British Machine Vision Conference (2016)

54.

Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

55.

Tang, X., Wang, X.: Face sketch recognition. IEEE Trans. Circ. Syst. Video Technol. 14(1), 50–57 (2004)CrossRef

56.

Thayananthan, A., Stenger, B., Torr, P.H.S., Cipolla, R.: Shape context and chamfer matching in cluttered scenes. In: IEEE Conference on Computer Vision and Pattern Recognition (2003)

57.

Wang, J., Song, Y., Leung, T., Rosenberg, C., Wang, J., Philbin, J., Chen, B., Wu, Y.: Learning fine-grained image similarity with deep ranking. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1386–1393 (2014)

58.

Yu, Q., Liu, F., Song, Y.Z., Xiang, T., Hospedales, T.M., Loy, C.C.: Sketch me that shoe. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)

59.

Yu, Q., Yang, Y., Song, Y., Xiang, T., Hospedales, T.: Sketch-a-net that beats humans. In: British Machine Vision Conference, pp. 7.1–7.12 (2015)

60.

Zobel, J., Moffat, A.: Inverted files for text search engines. ACM Comput. Surv. 38(2), 6.1–6.56 (2006)CrossRef

Titel: A survey of sketch-based image retrieval
verfasst von: Yi Li
Wenzhao Li
Publikationsdatum: 01.09.2018
Verlag: Springer Berlin Heidelberg
Erschienen in: Machine Vision and Applications / Ausgabe 7/2018
Print ISSN: 0932-8092
Elektronische ISSN: 1432-1769
DOI: https://doi.org/10.1007/s00138-018-0953-8

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Weitere Artikel der Ausgabe 7/2018

Localization of 3D objects using model-constrained SLAM

Mixture of counting CNNs

Detection of reactions to sound via gaze and global eye motion analysis using camera streaming

Visual tracking of resident space objects via an RFS-based multi-Bernoulli track-before-detect method

End-to-end temporal attention extraction and human action recognition

Key-frame selection for automatic summarization of surveillance videos: a method of multiple change-point detection

Premium Partner