Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Arabian Journal for Science and Engineering
Published in: Arabian Journal for Science and Engineering 8/2022

29-11-2021 | Research Article-Computer Engineering and Computer Science

Arabic and Latin Scene Text Recognition by Combining Handcrafted and Deep-Learned Features

Authors: Maroua Tounsi, Ikram Moalla, Umapada Pal, Adel M. Alimi

Published in: Arabian Journal for Science and Engineering | Issue 8/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Recognizing text within camera-captured images has been a very significant research topic for the last decades. In this paper, we intend to recognize Latin/Arabic text in natural scenes. For this reason, we present a comparative study between handcrafted and hybrid features. To acquire handcrafted features, we employ a standard bag of features (BoF) model based on a variant of the dense scale-invariant feature transform (SIFT) features. However, hybrid features are obtained by combining handcrafted features with deep-learned features using deep sparse auto-encoder (SAE). Indeed, an SAE-based method is applied in the local feature learning step to enhance discriminative as well as representative abilities of character image features. In the recognition step, we use hidden Markov model (HMM) with the aim of constructing a hybrid BoF-SAE-HMM architecture. We extensively evaluate our system upon various cropped word datasets of Arabic as well as Latin script. Using handcrafted features, the mean recognition accuracy obtained is 70.5 % for Arabic and 82.3 % for Latin script. Using hybrid features, we reached a mean recognition accuracy of 79.2 % and 91.7 % for Arabic and Latin scripts, respectively. Hence, combination of deep-learned features with handcrafted features leads to a considerable improvement of the recognition accuracy.
previous article A Heuristic Local-sensitive Program-Wide Diffing Method for IoT Binary Files
next article Evaluating the Performance of Data Level Methods Using KEEL Tool to Address Class Imbalance Problem

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Footnotes
Literature
1.
Almazan, J.; Gordo, A.; Fornes, A.; Valveny, E.: Word spotting and recognition with embedded attributes. IEEE Trans. Pattern Anal. Mach. Intell. 12, 2552–2566 (2014)CrossRef
2.
3.
Bai, Xiang; Yao, Cong; Liu, Wenyu: Strokelets: a learned multi-scale mid-level representation for scene text recognition. In: IEEE Trans. Image Process. 25(6), 2789–2802 (2016)MathSciNetMATH
4.
Bissacco, A., Cummins, M., Netzer Y., Neven, H.: PhotoOCR: reading text in uncontrolled conditions. 2013 IEEE International Conference on Computer Vision, pp. 785–792 (2013)
5.
Campos, Teófilo Emídio de.; Babu, Bodla Rakesh.; Varma, Manik.: Character Recognition in Natural Images. In: VISAPP 2009 - Proceedings of the Fourth International Conference on Computer Vision Theory and Applications, Lisboa, Portugal, February 5-8, 2009 - Volume 2, pp. 273–280 (2009).
6.
Chen, Chongmu.; Wang, Da-Han.; Wang, Hanzi.: Scene character recognition using PCANet. In: Proceedings of the 7th International Conference on Internet Multimedia Computing and Service, ICIMCS 2015, Zhangjiajie, Hunan, China, August 19-21, 2015, 65:1–65:4 (2015).
7.
Fraz, M.; Sarfraz, M.; Saquib, E.; Eran, A.: Exploiting colour information for better scene text detection and recognition. In Int. J. Doc. Anal. Recognit. (IJDAR) 18(2), 153–167 (2015)CrossRef
8.
Gao, S.; Wang, C.; Xiao, B.; Shi, C.; Zhou, W.; Zhang, Z.: Learning co-occurrence strokes for scene character recognition based on spatiality embedded dictionary. Presented at the (2014)
9.
Gomez, R.; Shi, B.; Gomez, L.; Numann, L.; Veit, A.; Matas, J.; Belongie, S.; Karatzas, D.: ICDAR2017 Robust Reading Challenge on COCO-Text. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR). Vol. 01, pp. 1435–1443 (2017).
10.
He, Pan.; Huang, Weilin.; Qiao, Yu.; Loy, Chen Change.; Tang, Xiaoou.: Reading Scene Text in Deep Convolutional Sequences. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA. Ed. by Dale Schuurmans and Michael P. Wellman. AAAI Press, pp. 3501–3508 (2016).
11.
Karatzas, Dimosthenis; et al.: In: ICDAR 2015 competition on Robust Reading, vol. 23–26, pp. 1156–1160. , Nancy, France, August (2015)
12.
13.
Lee, Chen-Yu.; Bhardwaj, Anurag.; Di, Wei.; Jagadeesh, Vignesh.; Piramuthu, Robinson.: Region-Based Discriminative Feature Pooling for Scene Text Recognition. In: pp. 4050–4057 (2014).
14.
Lee, Chen-Yu.; Osindero, Simon.: Recursive Recurrent Nets with Attention Modeling for OCR in the Wild. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp. 2231–2239 (2016).
15.
Lee, Honglak; Battle, Alexis; Raina, Rajat; Ng, Andrew Y.: In: Efficient sparse coding algorithms, vol. 4–7, pp. 801–808. , Vancouver, British Columbia, Canada December (2006)
16.
Liu, Xiyan; Meng, Gaofeng; Pan, Chunhong: Scene text detection and recognition with advances in deep learning: a survey. In: Int. J. Doc. Anal. Recognit. 22(2), 143–162 (2019)
17.
Long, Shangbang; He, Xin; Yao, Cong: Scene Text Detection and Recognition: The Deep Learning Era. International Journal of Computer Vision 1–24 (2020)
18.
Lowe, David G.: Distinctive image features from scale-invariant key-points. In: Int. J. Comput. Vision Nov. 60(2), 91–110 (2004)
19.
Lucas, S. M.; Panaretos, A.; Sosa, L.; Tang, A.; Wong, S.; Young, R.: ICDAR 2003 robust reading competitions. In: Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings. Pp. 682–687 (2003).
20.
Luo, C.; Jin, L.; Sun, Z.: MORAN: a multi-object rectified attention network for scene text recognition. Pattern Recognit. 90, 109–118 (2019)CrossRef
21.
Moubtahij, H.; El, H.; Akram, S.K.: Using features of local densities, statistics and HMM toolkit (HTK) for offline Arabic handwriting text recognition. In J. Electr. Syst. Inform. Technol. 4(3), 387–396 (2017)CrossRef
22.
Neumann, Lukas; Matas, Jiri: In: On Combining Multiple Segmentations in Scene Text Recognition, pp. 523–527. IEEE Computer Society (2013)
23.
Phan, T.Q.; Shivakumara, P.; Tian, S.; Tan, C.: L.: Recognizing Text with Perspective Distortion in Natural Scenes. Presented at the (2013)
24.
Rabiner, L. R.: A tutorial on hidden Markov models and selected applications in speech recognition. In: Proceedings of the IEEE 77.2, pp. 257–286 (1989).
25.
Ray, Anupama; Rajeswar, Sai; Chaudhury, Santanu: Scene Text Analysis Using Deep Belief Networks. Presented at the (2014)
26.
Roy, Sangheeta; Roy, Partha Pratim; Shivakumara, Palaiahnakote; Louloudis, Georgios; Tan, Chew Lim; Pal, Umapada (eds.): : In: HMM-Based Multi Oriented Text Recognition in Natural Scene Image, vol. 5–8, pp. 288–292. , Naha, Japan November (2013)
27.
Shi, Baoguang; Wang, Xinggang; Lyu, Pengyuan; Yao, Cong; Bai, Xiang: In: Robust Scene Text Recognition with Automatic Rectification, vol. 27–30, pp. 4168–4176. , Las Vegas, NV, USA, June (2016)
28.
Shi, C.; Wang, Y.; Jia, F.; He, K.; Wang, C.; Xiao, B.: Fisher vector for scene character recognition A comprehensive evaluation. Pattern Recognit 72, 1–14 (2017)CrossRef
29.
Su, B.; Lu, S.: Accurate recognition of words in scenes without character segmentation using recurrent neural network. Pattern Recognit. 63, 397–405 (2017)CrossRef
30.
Tan, Zhi Rong; Tian, Shangxuan; Tan, Chew Lim: In: Using pyramid of histogram of oriented gradients on natural scene text recognition, vol. 27–30, pp. 2629–2633. , Paris, France October (2014)
31.
Tian, Shangxuan; Bhattacharya, Ujjwal; Lu, Shijian; Su, Bolan; Wang, Qingqing; Wei, Xiaohua; Lu, Yue; Tan, Chew Lim.: Multilingual scene character recognition with co-occurrence of histogram of oriented gradients. In: Pattern Recognition 51, pp. 125–134 (2016).
32.
Tian, Shangxuan; Lu, Shijian,; Su, Bolan; Tan, Chew Lim.: Scene Text Recognition Using Co-occurrence of Histogram of Oriented Gradients. In: 2013 12th International Conference on Document Analysis and Recognition, Washington, DC, USA, August 25-28, 2013, pp. 912–916 (2013).
33.
Tounsi, Maroua; Moalla, Ikram; Alimi, Adel M.: Supervised dictionary learning in BoF framework for Scene Character recognition. In: 23rd International Conference on Pattern Recognition, ICPR 2016, Cancún, Mexico, December 4-8, 2016, pp. 3987–3992 (2016).
34.
Tounsi, Maroua; Moalla, Ikram; Alimi, Adel M.: ARASTI: A database for Arabic scene text recognition. In: 1st International Workshop on Arabic Script Analysis and Recognition, ASAR 2017, Nancy, France, April 3-5, 2017, pp. 140–144 (2017).
35.
Tounsi, Maroua; Moalla, Ikram; Alimi, Adel M.; Lebourgeois, Frank: In: Arabic characters recognition in natural scenes using sparse coding for feature representations, vol. 23–26, pp. 1036–1040. , Nancy, France, August (2015)
36.
Vincent, Pascal; Larochelle, Hugo; Bengio, Yoshua; Manzagol, Pierre-Antoine.: In: Extracting and Composing Robust Features with Denoising Autoencoders, pp. 1096–1103. , Helsinki, Finland (2008)
37.
Vincent, P.; Larochelle, H.; Lajoie, I.; Bengio, Y.; Manzagol, P.A.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)MathSciNetMATH
38.
Wang, Kai; Babenko, Boris; Belongie, Serge J.: End-to-end scene text recognition. In: IEEE International Conference on Computer Vision, ICCV 2011, Barcelona, Spain, November 6-13, 2011, pp. 1457–1464 (2011).
39.
Wang, Z.; Teng, S.; Liu, G.; Zhao, Z.; Wu, H.: Hierarchical sparse representation with deep dictionary for multi-modal classification. Neurocomputing 253, 65–69 (2017)CrossRef
40.
Xie, Hongtao; Fang, Shancheng; Zha, Zheng-Jun; Yang, Yating; Li, Yan; Zhang, Yongdong.: Convolutional Attention Networks for Scene Text Recognition. In: 15.1s. issn: 1551–6857 (2019).
41.
Yang, L.; Wang, P.; Li, H.; Li, Z.; Zhang, Y.: A holistic representation guided attention network for scene text recognition. Neurocomputing 414, 67–75 (2020)CrossRef
42.
Ye, Qixiang; Doermann, David S.: Text detection and recognition in imagery: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 37(7), 1480–1500 (2015)CrossRef
43.
Yi, C.; Yang, X.; Tian, Y.: Feature representations for scene text character recognition: a comparative study. ICDAR 13, 907–911 (2013)
44.
Young, Steve J.; Kershaw, D.; Odell, J.; Ollason, D.; Valtchev, V.; Woodland, P.: The HTK Book Version 3.4. Cambridge University Press, Cambridge (2006)
45.
Zhang, Dong; Wang, Da-Han.; Wang, Hanzi: In: Scene text recognition using sparse coding based features, vol. 27–30, pp. 1066–1070. , Paris, France October (2014)
46.
Zhu, Yingying; Yao, Cong; Bai, Xiang: Scene text detection and recognition: recent advances and future trends. In: Front. Comput. Sci. 10(1), 19–36 (2016)
Metadata
Title
Arabic and Latin Scene Text Recognition by Combining Handcrafted and Deep-Learned Features
Authors
Maroua Tounsi
Ikram Moalla
Umapada Pal
Adel M. Alimi
Publication date
29-11-2021
Publisher
Springer Berlin Heidelberg
Published in
Arabian Journal for Science and Engineering / Issue 8/2022
Print ISSN: 2193-567X
Electronic ISSN: 2191-4281
DOI
https://doi.org/10.1007/s13369-021-06311-1

Other articles of this Issue 8/2022

Arabian Journal for Science and Engineering 8/2022 Go to the issue

Research Article-Computer Engineering and Computer Science

Optimization of Product Switching Processes in Assembly Lines

Research Article-Computer Engineering and Computer Science

A Feature Selection Method Using Dynamic Dependency and Redundancy Analysis

Research Article-Computer Engineering and Computer Science

Automatic Prediction of Glycemic Index Category from Food Images Using Machine Learning Approaches

Research Article-Computer Engineering and Computer Science

Transformer-Based Word Embedding With CNN Model to Detect Sarcasm and Irony

Research Article-Computer Engineering and Computer Science

A Hybrid Feature Selection Approach for Parkinson’s Detection Based on Mutual Information Gain and Recursive Feature Elimination

Research Article-Computer Engineering and Computer Science

Spectrum Allocation in Cognitive Radio Networks Using Gene Therapy-Based Evolutionary Algorithms

Premium Partners

    Image Credits