nach oben

Erschienen in:

01.12.2023 | SCIENTIFIC SCHOOLS OF THE FEDERAL RESEARCH CENTER “COMPUTER SCIENCE AND CONTROL” OF THE RUSSIAN ACADEMY OF SCIENCES, MOSCOW, THE RUSSIAN FEDERATION

Advances of the Scientific School of V.L. Arlazarov in Dataset Creation and Training Sample Synthesis for Solving Modern Computer Vision Problems

verfasst von: Y. S. Chernyshova, A. V. Sheshkus, K. B. Bulatov, V. V. Arlazarov

Erschienen in: Pattern Recognition and Image Analysis | Ausgabe 4/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

This paper considers a scientific school of synthesis of samples and creation of datasets, which is a part of the family of scientific schools associated with image processing and analysis, originating from the work of a team led by Prof. V.L. Arlazarov in the 1970s. As part of the work of the school, the researchers have obtained important fundamental and applied results as well as set new research tasks. Over the years of the school’s existence the scientific team has developed several algorithms and systems for the synthesis and augmentation of image samples. Moreover, they have created and published more than ten open annotated image datasets, including the unique MIDV dataset family that contains synthesized images of identity documents and is the first in the world to allow a full open comparison of recognition systems for such documents.

Vorheriger Artikel Development of the Neural Network Based Recognition Methods at V.L. Arlazarov’s Scientific School

Nächster Artikel High-Performance Digital Image Processing

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Google Dataset Search Online. https://datasetsearch.research.google.com/. Cited October 6, 2022.

Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 On the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95/46/EC (General Data Protection Regulation).

Federal Law of the Russian Federation On the Personal Data on July 27, 2006, no. 152-FZ.

S. An, M. Lee, S. Park, H. Yang, and J. So, “An ensemble of simple convolutional neural network models for MNIST digit recognition” (2008). https://doi.org/10.48550/arXiv.2008.10400

V. V. Arlazarov, K. Bulatov, T. Chernov, and V. L. Arlazarov, “MIDV-500: A dataset for identity document analysis and recognition on mobile devices in video stream,” Comput. Opt. 43, 818–824 (2019). https://doi.org/10.18287/2412-6179-2019-43-5-818-824CrossRef

V. V. Arlazarov, N. V. Reshetnyak, and O. A. Slavin, “Formation of the set of graphic images of symbols in problems of symbol classifier learning,” Tr. Inst. Sist. Anal. Ross. Akad. Nauk 64 (4), 73–79 (2014).

P. V. Bezmaternykh, D. A. Ilin, and D. P. Nikolaev, “U-Net-bin: Hacking the document image binarization contest,” Comput. Opt. 43, 825–832 (2019). https://doi.org/10.18287/2412-6179-2019-43-5-825-832CrossRef

K. B. Bulatov, E. V. Emelianova, D. V. Tropin, N. S. Skoryukina, Y. S. Chernyshova, A. V. Sheshkus, S. A. Usilin, Z. Ming, J.-C. Burie, M. Luqman, and V. V. Arlazarov, “MIDV-2020: A comprehensive benchmark dataset for identity document analysis,” Comput. Opt. 46, 252–270 (2022). https://doi.org/10.18287/2412-6179-co-1006CrossRef

K. Bulatov, D. Matalov, and V. V. Arlazarov, “MIDV-2019: Challenges of the modern mobile-based document OCR,” Proc. SPIE 11433, 114332N (2020). https://doi.org/10.1117/12.2558438

10.

M. O. Chekanov, O. S. Shipitko, M. Griguletskii, and A. Buzmakov, “X-ray and visible spectra circular motion images dataset,” Proc. SPIE 12084, 120841J (2022). https://doi.org/10.1117/12.2623485

11.

Yu. S. Chernyshova, E. V. Emelianova, A. V. Sheshkus, and V. V. Arlazarov, “MIDV-LAIT: A challenging dataset for recognition of IDs with Perso-Arabic, Thai, and Indian Scripts,” in Document Analysis and Recognition–ICDAR 2021, Ed. by J. Lladós, D. Lopresti, and S. Uchida, Lecture Notes in Computer Science, Vol. 12822 (Springer, Cham, 2021), pp. 258–272. https://doi.org/10.1007/978-3-030-86331-9_17CrossRef

12.

Yu. S. Chernyshova, A. V. Gayer, and A. V. Sheshkus, “Generation method of synthetic training data for mobile OCR system,” Proc. SPIE 10696, 106962G (2018). https://doi.org/10.1117/12.2310119

13.

Yu. S. Chernyshova, A. V. Sheshkus, and V. V. Arlazarov, “Two-step CNN framework for text line recognition in camera-captured images,” IEEE Access 8, 32587–32600 (2020). https://doi.org/10.1109/access.2020.2974051CrossRef

14.

Z. Dai, H. Liu, Q. Le, and V. Tan, “CoAtNet: Marrying convolution and attention for all data sizes,” Adv. Neural Inf. Process. Syst. 34, 3965–3977 (2021).

15.

L.-P. de las Heras, O. R. Terrades, J. Llados, D. Fernandez-Mota, and C. Canero, “Use case visual bag-of-words techniques for camera based identity document classification,” in 2015 13th Int. Conf. on Document Analysis and Recognition (ICDAR) (IEEE, 2015), pp. 721–725. https://doi.org/10.1109/icdar.2015.7333856

16.

A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale” (2020). https://doi.org/10.48550/arXiv.2010.11929

17.

S. O. Emel’yanov, A. A. Ivanova, E. A. Shvets, and D. P. Nikolaev, “Augmentation methods for training samples in problems of image classification,” Sensornye Sist. 32, 236–245 (2018). https://doi.org/10.1134/S0235009218030058CrossRef

18.

E. I. Ershov, A. V. Belokopytov, and A. V. Savchik, “Problems of creating the dataset for solving the problem of illumination estimation,” in Proc. ITNT-2020 (Novaya Tekhnika, Samara), Vol. 4, pp. 1090–1097.

19.

E. Ershov, A. Savchik, I. Semenkov, N. Banić, A. Belokopytov, D. Senshina, K. Koščević, M. Subašić, and S. Lončarić, “The Cube++ illumination estimation dataset,” IEEE Access 8, 227511–227527 (2020). https://doi.org/10.1109/access.2020.3045066CrossRef

20.

E. Ershov, A. Savchik, I. Semenkov, N. Banić, K. Koščević, M. Subašić, A. Belokopytov, A. Terekhin, D. Senshina, A. Nikonorov, Z. Li, Ya. Qian, M. Buzzelli, R. Riva, S. Bianco, R. Schettini, J. Barron, S. Lončarić, and D. Nikolaev, “Illumination estimation challenge: The experience of the first 2 years,” Color Res. Appl. 46, 705–718 (2021). https://doi.org/10.1002/col.22675CrossRef

21.

A. V. Gaier, Yu. S. Chernyshova, and A. V. Sheshkus, “Generation of artificial training sample for the problem of symbol recognition of the fields in the Russian passport,” Sensornye Sist. 32, 230–235 (2018). https://doi.org/10.1134/S023500921803006XCrossRef

22.

A. V. Gayer, Yu. S. Chernyshova, and A. V. Sheshkus, “Effective real-time augmentation of training dataset for the neural networks learning,” Proc. SPIE 11041, 11041I1 (2019). https://doi.org/10.1117/12.2522969

23.

A. V. Gaier, A. V. Sheshkus, and Yu. S. Chernyshova, “Augmentation on the fly for the neural networks learning,” Tr. Inst. Sistemnogo Anal. Ross. Akad. Nauk 68 (1), 150–157 (2018). https://doi.org/10.14357/20790279180517CrossRef

24.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2016), pp. 770–778. https://doi.org/10.1109/cvpr.2016.90

25.

G. Huang, Z. Liu, L. Van Der Maaten, and K. Weinberger, “Densely connected convolutional networks,” in 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2017), pp. 4700–4708. https://doi.org/10.1109/cvpr.2017.243

26.

D. Ilin and V. Krivtsov, “Creating Training Datasets For OCR In Mobile Device Video Stream,” in Eur. Conf. of Modelling and Simulation 2015 Proc., Ed. by V. M. Mladenov, P. Georgieva, G. Spasov, and G. Petrova (Eur. Council for Modelling and Simulation, 2015), pp. 516–520. https://doi.org/10.7148/2015-0516

27.

A. S. Ingacheva, “Spectral model of signal of single-channel X-ray measuring tools using polychromatic probing radiation,” Sensornye Sist. 33, 212–221 (2019). https://doi.org/10.1134/S0235009219030028CrossRef

28.

A. A. Ivanova, F. A. Fedorenko, and I. A. Konovalenko, “Preparation of the training set for creating the neural network projective-invariant descriptors of singular points,” in Proc. 40th Int. School-Conf. Information Technologies and Systems 2016 of the Institute for Information Transmission Problems of the Russian Academy of Sciences (Inst. Probl. Peredachi Informatsii Ross. Akad. Nauk, Moscow, 2016), pp. 303–308.

29.

T. Khanipov, I. Koptelov, A. Grigoryev, E. Kuznetsova, and D. Nikolaev, “Vision-based industrial automatic vehicle classifier,” Proc. SPIE 9445, 944511 (2015). https://doi.org/10.1117/12.2181557

30.

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE 86, 2278–2324 (1998). https://doi.org/10.1109/5.726791CrossRef

31.

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, Ch.-Ya. Fu, and A. Berg, “SSD: Single shot MultiBox detector,” in Computer Vision–ECCV 2016, Ed. by B. Leibe, J. Matas, N. Sebe, and M. Welling, Lecture Notes in Computer Science, Vol. 9905 (Springer, Cham, 2016), pp. 21–37. https://doi.org/10.1007/978-3-319-46448-0_2CrossRef

32.

D. P. Matalov, S. A. Usilin, and V. V. Arlazarov, “Single-sample augmentation framework for training Viola-Jones classifiers,” Proc. SPIE 11433, 114330I (2020). https://doi.org/10.1117/12.2559435

33.

D. P. Matalov, S. A. Usilin, and V. V. Arlazarov, “About Viola–Jones image classifier structure in the problem of stamp detection in document images,” Proc. SPIE 11605, 116050V (2021). https://doi.org/10.1117/12.2586842

34.

A. N. Milovzorov, P. V. Bezmaternykh, and V. V. Arlazarov, “Generative approach for 1D barcode dataset population for mobile-based recognition,” Proc. SPIE 11605, 116051P (2021). https://doi.org/10.1117/12.2587628

35.

S. R. Narang, M. K. Jindal, and M. Kumar, “Ancient text recognition: A review,” Artif. Intell. Rev. 53, 5517–5558 (2020). https://doi.org/10.1007/s10462-020-09827-4CrossRef

36.

D. P. Nikolaev, D. V. Polevoi, and N. A. Tarasova, “Synthesis of the training sample in the problem of text recognition in the three-dimensional space,” Inf. Tekhnol. Vychisl. Sist., No. 3, 82–88 (2014).

37.

D. P. Nikolaev, E. A. Shvets, and D. A. Shepelev, “Constructing the passability map on the basis of distance sensor readings by the stochastic gradient method,” Tr. Inst. Sist. Anal. Ross. Akad. Nauk 66 (1), 64–69 (2016).

38.

E. Panfilova, A. Grigoryev, and V. Burmistrov, “Elongated boundaries detector parameters optimisation based on generation of synthetic data from aerial imagery,” in Eur. Conf. of Modelling and Simulation 2022, Ed. by I. A. Hameed, A. Hasan, S. A.-A. Alaliyat, and M. Iacono (Eur. Council for Modelling and Simulation, 2022), Vol. 36, pp. 167–173. https://doi.org/10.7148/2022-0167

39.

D. V. Polevoy, I. V. Sigareva, D. M. Ershova, V. V. Arlazarov, D. P. Nikolaev, Z. Ming, M. L. Luqman, and J.-Ch. Burie, “Document Liveness Challenge Dataset (DLC-2021),” J. Imaging 8, 181 (2022). https://doi.org/10.3390/jimaging8070181CrossRefPubMedPubMedCentral

40.

M. A. Povolotskii, E. G. Kuznetsova, N. V. Utkin, and D. P. Nikolaev, “Segmentation of license plate of cars with application of the algorithm of dynamic transformation of the temporal axis,” Sensornye Sist. 32 (1), 50–59 (2018). https://doi.org/10.7868/S0235009218010080CrossRef

41.

M. A. Povolotskiy, E. G. Kuznetsova, and T. M. Khanipov, “Russian license plate segmentation based on dynamic time warping,” in Eur. Conf. of Modelling and Simulation 2017 Proc., Ed. by Z. Z. Paprika, P. Horák, K. Váradi, P. T. Zwierczyk, A. Vidovics-Dancs, and J. P. Rádics (Eur. Council for Modelling and Simulation, 2017), pp. 285–291. https://doi.org/10.7148/2017-0285

42.

M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-Ch. Chen, “MobileNetV2: Inverted residuals and linear bottlenecks,” in 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (IEEE, 2018), pp. 4510–4520. https://doi.org/10.1109/cvpr.2018.00474

43.

D. A. Shepelev, V. P. Bozhkova, E. I. Ershov, and D. P. Nikolaev, “On a problem of modeling the underwater images based on above-water ones,” in Proc. ITNT-2020 (Novaya Tekhnika, Samara,), Vol. 4, pp. 1081–1089.

44.

D. A. Shepelev, V. P. Bozhkova, E. I. Ershov, and D. P. Nikolaev, “Simulating shot noise of color underwater images,” Comput. Opt. 44, 671–679 (2020). https://doi.org/10.18287/2412-6179-co-754CrossRef

45.

D. A. Shepelev, V. P. Bozhkova, E. I. Ershov, and D. P. Nikolaev, “Simulation of underwater color images using banded spectral model,” in Eur. Conf. of Modelling and Simulation 2020 Proc., Ed. by M. Steglich, Ch. Muller, G. Neumann, and M. Walther (Eur. Council for Modelling and Simulation, 2020), Vol. 34. https://doi.org/10.7148/2020-0011

46.

A. V. Sheshkus, Yu. S. Chernyshova, A. V. Gaier, A. E. Lynchenko, and D. P. Nikolaev, Automatic system of generating data and training artificial neural networks Smart NNCreator, Rospatent.

47.

A. Sheshkus, A. Chirvonaya, and V. L. Arlazarov, “Tiny CNN for feature point description for document analysis: Approach and dataset,” Comput. Opt. 46, 429–435 (2022). https://doi.org/10.18287/2412-6179-co-1016CrossRef

48.

A. V. Shoshin and E. A. Shvets, “Veiling glare removal: Synthetic dataset generation, metrics and neural network architecture,” Comput. Opt. 45, 615–626 (2021). https://doi.org/10.18287/2412-6179-co-883CrossRef

49.

M. S. Shutov, M. I. Gil’manov, and A. S. Ignacheva, “Methods of modelling ring artifacts in computed tomography,” in Proc. 64th All-Russian Sci. Conf. of the Moscow Institute of Physics and Technology (Mosk. Fiz.-Tekh. Inst., Dolgoprudnyi, Moscow oblast,) .

50.

E. A. Shvets and D. A. Shepelev, Imitation modeling system for constructing the passability map on the basis of sonar records, Rospatent.

51.

N. Skoryukina, D. P. Nikolaev, A. Sheshkus, and D. Polevoy, “Real time rectangular document detection on mobile devices,” Proc. SPIE 9445, 94452A (2015). https://doi.org/10.1117/12.2181377

52.

D. G. Slugin, Program for augmenting tomographic data for modeling deviations from the basic model of reconstruction, Rospatent.

53.

A. Smagina, E. Ershov, and A. Grigoryev, “Multiple light source dataset for colour research,” Proc. SPIE 11433, 114332C (2020). https://doi.org/10.1117/12.2559491

54.

V. Soboleva and O. Shipitko, “Raindrops on windshield: Dataset and lightweight gradient-based detection algorithm,” in 2021 IEEE Symp. Series on Computational Intelligence (SSCI), Orlando, Fla., 2022 (IEEE, 2022), pp. 1–7. https://doi.org/10.1109/SSCI50451.2021.9659915

55.

S. A. Usilin, V. V. Arlazarov, N. S. Rokhlin, S. A. Rudyka, S. A. Matveev, and A. A. Zatsarinny, “Training Viola–Jones detectors for 3D objects based on fully synthetic data for use in rescue missions with UAV,” Vestn. Yuzhno-Ural. Gos. Univ. Mat. Model. Programirovanie 13 (4), 94–106 (2020). https://doi.org/10.14529/mmp200408CrossRef

56.

S. Usilin, D. Nikolaev, V. Postnikov, and G. Schaefer, “Visual appearance based document image classification,” in 2010 IEEE Int. Conf. on Image Processing, Hong Kong, 2010 (IEEE), pp. 2133–2136. https://doi.org/10.1109/ICIP.2010.5652024

57.

S. A. Usilin, O. A. Slavin, and V. V. Arlazarov, “Memory consumption and computation efficiency improvements of Viola–Jones object detection method for UAVs,” in Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021, Ed. by A. Del Bimbo, Lecture Notes in Computer Science, Vol. 12665 (Springer, Cham, 2021), pp. 243–252. https://doi.org/10.1007/978-3-030-68821-9_23CrossRef

58.

A. V. Uskov, A. D. Astakhov, A. R. Bitman, N. E. Buzikashvili, A. N. Vannik, A. Ya. Podrabinovich, V. V. Postnikov, O. A. Slavin, D. V. Solov’ev, and P. S. Khlebutin, “Creation of the database of optical images of text fonts and hand-written and printed symbols,” in Otchet RFFI 97-07-90209.

59.

V. V. Volkov and E. A. Shvets, “Dataset and method for estimating the algorithms of comparison of optical and radar images based on stable points,” Inf. Tekhnol. Vychisl. Sist., No. 2, 44–57 (2021). https://doi.org/10.14357/20718632210205

60.

M. Yim, Y. Kim, H. Cho, and S. Park, “SynthTIGER: Synthetic text image generator towards better text recognition models,” in Document Analysis and Recognition-ICDAR 2021, Ed. by J. Lladós, D. Lopresti, and S. Uchida, Lecture Notes in Computer Science, Vol. 12824 (Springer,), pp. 109–124. https://doi.org/10.1007/978-3-030-86337-1_8CrossRef

61.

A. Zharkov and I. Zagaynov, “Universal barcode detector via semantic segmentation,” in 2019 Int. Conf. on Document Analysis and Recognition (ICDAR), Sydney, 2019 (IEEE, 2019), pp. 837–843. https://doi.org/10.1109/ICDAR.2019.00139

62.

X. Zhong, E. Shafiei, A. Bavani, and Yepes, “Image-based table recognition: Data, model, and evaluation,” in Computer Vision-ECCV 2020, Ed. by A. Vedaldi, H. Bischof, T. Brox, and J. M. Frahm, Lecture Notes in Computer Science, Vol. 12366 (Springer, Cham.), pp. 564–580. https://doi.org/10.1007/978-3-030-58589-1_34CrossRef

63.

A. Zhukovskii, S. Usilin, N. Tarasova, and D. Nikolaev, “Synthesis of training sample on the basis of real data in image recognition problems,” in Proc. 35th Interdisciplinary School-Conf. Information Technologies and Systems 2012 of the Institute for Information Transmission Problems of the Russian Academy of Sciences (Inst. Probl. Peredachi Informatsii, Moscow, 2012), pp. 377–382.

64.

P. K. Zlobin, Y. S. Chernyshova, A. V. Sheshkus, and V. V. Arlazarov, “Character sequence prediction method for training data creation in the task of text recognition,” Proc. SPIE 12084, 120840R (2022). https://doi.org/10.1117/12.2623773

Titel: Advances of the Scientific School of V.L. Arlazarov in Dataset Creation and Training Sample Synthesis for Solving Modern Computer Vision Problems
verfasst von: Y. S. Chernyshova
A. V. Sheshkus
K. B. Bulatov
V. V. Arlazarov
Publikationsdatum: 01.12.2023
Verlag: Pleiades Publishing
Erschienen in: Pattern Recognition and Image Analysis / Ausgabe 4/2023
Print ISSN: 1054-6618
Elektronische ISSN: 1555-6212
DOI: https://doi.org/10.1134/S1054661823040107

SCIENTIFIC SCHOOLS OF THE FEDERAL RESEARCH CENTER “COMPUTER SCIENCE AND CONTROL” OF THE RUSSIAN ACADEMY OF SCIENCES, MOSCOW, THE RUSSIAN FEDERATION

Solving Fundamental and Applied Problems of Digital Image Processing at the Institute of Automation and Electrometry and Other Scientific Schools of the Siberian Branch of the Russian Academy of Sciences

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Weitere Artikel der Ausgabe 4/2023

What Is a Scientific School?

The Physical Principles of the Construction of Systems for Safe Monitoring of the State of a Human Operator

Application of Cascade Methods as a Universal Object Detection Tool

Ural School of Pattern Recognition: Majoritarian Approach to Ensemble Learning

Spectral Methods in Data Analysis and Pattern Recognition Problems: Works of the Pushchino School

Solving Fundamental and Applied Problems of Digital Image Processing at the Institute of Automation and Electrometry and Other Scientific Schools of the Siberian Branch of the Russian Academy of Sciences

Premium Partner