Top

Annals of Data Science

23-05-2024

A Deep Convolutional Neural Network-Based Approach for Visual Search & Recommendation of Grocery Products

Authors: Nawreen Anan Khandaker, Amrin Rahman, Amrin Akter Pinky, Tasmiah Tamzid Anannya

Published in: Annals of Data Science

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Search and recommendation are two essential features of any e-commerce website for finding and purchasing a specific product. Visual Search is a promising and quick method in comparison to a textual-based search method. Hence, the objective of this research is to propose a conceptual framework for developing a visual search and recommendation system for grocery products using Ensemble Learning with CNN models. Traditional Deep learning and Ensemble Learning techniques were implemented with a publicly available and a self-made data set containing 3174 and 3162 images respectively. Various combinations of the suitable models found from research findings were used to find the best-fitted model for both the search and recommendation functionalities. All the models were evaluated using suitable performance metrics and the Ensemble Learning approach performed better. The best-performed results for visual searching are obtained by incorporating VGG16 and MobileNet with an accuracy of 99.8% for classification and in the case of product recommendation, the combination of MobileNET and ResNET50 performs better than other techniques.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Imran (2020) On-site search statistics for ecommerce and others. https://www.addsearch.com/blog/site-search-statistics. Accessed 26 June 2023

pixyle.ai (2020) Is visual search making text-based search obsolete? https://www.pixyle.ai/product-discovery/is-visual-search-making-text-based-search-obsolete. Accessed 28 July 2023

Shi Y (2022) Advances in big data analytics: theory, algorithm and practice. Springer, SingaporeCrossRef

Olson DL, Shi Y (2007) Introduction to business data mining. McGraw-Hill/Irwin, New York

Shi Y, Tian YJ, Kou G, Peng Y, Li JP (2011) Optimization based data mining: theory and applications. Springer, BerlinCrossRef

Tien JM (2017) Internet of things, real-time decision making, and artificial intelligence. Ann Data Sci 4(2):149–178CrossRef

Felfernig A, Jeran M, Ninaus G, Reinfrank F and Reiterer S, Stettinger M (2014) Basic approaches in recommendation systems

Umer S, Mohanta PP, Rout RK, Pandey HM (2021) Machine learning method for cosmetic product recognition: a visual searching approach. Multimed Tools Appl 80:34997–35023. https://doi.org/10.1007/s11042-020-09079-yCrossRef

Bishop CM, Nasrabadi NM (2006) Pattern recognition and machine learning. Springer, New York

10.

Chang CC, Lin CJ (2011) A library for Support Vector Machines. ACM Trans Intell Syst Technol 2(3):2157–6904. https://doi.org/10.1145/1961189.1961199CrossRef

11.

Mullick SS, Datta S, Das S (2018) Adaptive learning-based k-nearest neighbor classifiers with resilience to class imbalance. IEEE Trans Neural Netw Learn Syst 29(11):5713–5725. https://doi.org/10.1109/TNNLS.2018.2812279CrossRef

12.

Raschka S (2015) Python machine learning. Packt Publishing Ltd, Birmingham

13.

Safavian SR, Landgrebe D (1991) A survey of decision tree classifier methodology. IEEE Trans Syst Man Cybern 21(3):660–674. https://doi.org/10.1109/21.97458CrossRef

14.

Rinaldi AM, Russo C, Tommasino C (2021) Visual query posing in multimedia web document retrieval. In: Proceedings of the 15th international conference on semantic computing (ICSC). IEEE, Laguna Hills. https://doi.org/10.1109/ICSC50631.2021.00086

15.

Smelyakov K, Sandrkin D, Ruban I, Martovytskyi V, Romanenkov Y (2018) Search by image. New search engine service model. In: Proceedings of the international scientific-practical conference problems of infocommunications. Science and Technology (PIC S &T). IEEE, Kharkiv. https://doi.org/10.1109/INFOCOMMST.2018.8632117

16.

Hu C, Li Q, Zhang Z, Chang Kh, Zhang R (2020) A multimodal fusion framework for brand recognition from product image and context. In: Proceedings of the international conference on multimedia & expo workshops (ICMEW), pp 1–4. https://doi.org/10.1109/ICMEW46912.2020.9105947

17.

Wei Z, Song X and Lin Y (2020) Image feature recognition algorithm for rural revitalization product design based on visual attention model. In: Proceedings of the 4th international conference on inventive systems and control (ICISC). IEEE, Coimbatore. https://doi.org/10.1109/ICISC47916.2020.9171207

18.

Sogi N, Souza LS, Gatto BB, Fukui K (2020) Metric learning with a-based scalar product for image-set recognition. In: Proceedings of the conference on computer vision and pattern recognition workshops (CVPRW). IEEE, Seattle. https://doi.org/10.1109/CVPRW50498.2020.00433

19.

Boriya A, Malla SS, Manjunath R, Velicheti V, Eirinaki, M (2019) ViSeR: A visual search engine for e-retail. In: Proceedings of the first international conference on transdisciplinary AI (TransAI). IEEE, Laguna Hills. https://doi.org/10.1109/TransAI46475.2019.00021

20.

Dakhare BS, Khatu A, Singh HR, Gupta A (2020) Visual E-commerce application using deep learning. Int Res J Eng Technol (IRJET) 7(3):1–9

21.

Kim Y (2021) A study on the efficient product search service for the damaged image information. arXiv preprint arXiv:2111.07346

22.

Zhang Ya, Pan P, Zheng Y, Zhao K, Zhang Y, Ren X and Jin R (2018) Visual search at Alibaba. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 993–1001. https://doi.org/10.1145/3219819.3219820

23.

Szegedy C, Wei L, Yangqing J, Sermanet P, Reed S, Anguelov D, Erhan, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1–9. https://doi.org/10.1109/CVPR.2015.7298594

24.

Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg A, Fei-Fei L (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115:211–252. https://doi.org/10.1007/s11263-015-0816-yCrossRef

25.

Li J, Liu H, Gui C, Chen J, Ni Z, Wang N, Chen Y (2018) The Design and implementation of a real time visual search system on JD E-commerce platform. In: Proceedings of the 19th international middleware conference industry. https://doi.org/10.1145/3284028.3284030

26.

Hu H, Wang Y, Yang L, Komlev P, Huang L, Chen X, Huang J, Wu Y, Merchant M, Sacheti A (2018) Web-scale responsive visual search at bing. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 359–367. https://doi.org/10.1145/3219819.3219843

27.

Smelyakov K, Chupryna A, Sandrkin D, Kolisnyk M (2020) Search by image engine for big data warehouse. In: Proceedings of the 2020 IEEE open conference of electrical, electronic and information sciences (eStream), pp 1–4

28.

Mu C, Zhao J, Yang G, Zhang J, Yan Z (2018) Towards practical visual search engine within elasticsearch. arXiv preprint arXiv:1806.08896

29.

Karmakar C, Datcu M (2020) A fast search system for remote sensing imagery based on bag of visual words and latent Dirichlet allocation. In: Proceeding of the international geoscience and remote sensing symposium (IGARSS). IEEE, Waikoloa. https://doi.org/10.1109/IGARSS39084.2020.9324359

30.

Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60:84–90. https://doi.org/10.1145/3065386CrossRef

31.

He K, Zhang X, Ren A, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37:1904–1916. https://doi.org/10.1109/TPAMI.2015.2389824CrossRef

32.

Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv. https://doi.org/10.48550/arXiv.1502.03167

33.

He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Las Vegas. https://doi.org/10.1109/CVPR.2016.90

34.

Marichal X, De VC, Macq B (1997) Towards visual search engine based on fuzzy logic

35.

Sumaiya, Armanuzzaman M (2020) Enhancement of resulting image search engine (ERISE) by content-based image retrieval system. In: Proceedings of the IEEE region 10 symposium (TENSYMP). IEEE, Dhaka. https://doi.org/10.1109/TENSYMP50017.2020.9230653

36.

Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. arXiv. https://doi.org/10.48550/arXiv.1409.1556

37.

Hur C, Hyun C, Park H (2020) Automatic image recommendation for economic topics using visual and semantic information. In: Proceedings of the 14th international conference on semantic computing (ICSC). IEEE, San Diego. https://doi.org/10.1109/ICSC.2020.00037

38.

Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Honolulu. https://doi.org/10.1109/CVPR.2017.690

39.

Tautkute I, Trzciński T, Skorup AP, Brocki L, Marasek K (2019) DeepStyle: multimodal search engine for fashion and interior design. IEEE Access 7:84613–84628. https://doi.org/10.1109/ACCESS.2019.2923552CrossRef

40.

Eswaran A, Varshini E (2022) Reverse image search engine for garment industry. In: Proceedings of the 2022 8th international conference on advanced computing and communication systems (ICACCS). IEEE, Coimbatore, pp. 414–418

Title: A Deep Convolutional Neural Network-Based Approach for Visual Search & Recommendation of Grocery Products
Authors: Nawreen Anan Khandaker
Amrin Rahman
Amrin Akter Pinky
Tasmiah Tamzid Anannya
Publication date: 23-05-2024
Publisher: Springer Berlin Heidelberg
Published in: Annals of Data Science
Print ISSN: 2198-5804
Electronic ISSN: 2198-5812
DOI: https://doi.org/10.1007/s40745-024-00540-5

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"