Top

Arabian Journal for Science and Engineering

Published in:

23-04-2021 | Research Article-Computer Engineering and Computer Science

EnPSO: An AutoML Technique for Generating Ensemble Recommender System

Authors: Garima Gupta, Rahul Katarya

Published in: Arabian Journal for Science and Engineering | Issue 9/2021

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

With the explosive increase in data on the web, recommending items to users is becoming more complex. In recent times, the best recommender systems have come from ensemble learning, which combines many models and techniques to generate recommendations that can draw the best characteristics of the constituent models. These ensemble models can improve accuracy, and they are also able to reduce the biases that come with each model. However, the massive number of permutations in which recommendation models are combined to create an ensemble model adds another layer of complexity to an already complex problem. Thus, in line with the recent trend in Automated Machine Learning (AutoML) aimed at reducing the complexity associated with model selection, there is a need for a machine learning framework that can learn the best ensemble model for a given problem given the base models. We proposed a system Ensemble with Particle Swarm Optimization, which intelligently optimizes the recommendations by identifying the best ensemble architecture for the data at hand. Our proposed AutoML system can improve recommendations for the MovieLens dataset by combining the results from base techniques without any user effort. The major challenge in creating such a system is to develop a framework for generating ensemble models and finding an efficient way to reach the best performing model since the search space can be vast. To overcome this challenge, we have used hierarchical models to generate ensembles and Particle Swarm Optimization to find the optimal ensemble models.

previous article D-SAG: A Distributed Sort-Based Algorithm for Graph Clustering

next article Online Reviews Analysis for Customer Segmentation through Dimensionality Reduction and Deep Learning Techniques

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Liang, H.; Tao, X.; Xu, Y.; Nayak, R.; Li, Y.: Connecting users and items with weighted tags for personalized item recommendations. 51 (2010). https://doi.org/10.1145/1810617.1810628

Jamali, M.; Ester, M.: TrustWalker: a random walk model for combining trust-based and item-based recommendation. In: KDD ’09 Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 397–406 (2009).

Yuan, J.; Shalaby, W.; Korayem, M.; Lin, D.; AlJadda, K.; Luo, J.: Solving cold-start problem in large-scale recommendation engines: {A} deep learning approach. CoRR. abs/1611.0, 1901–1910 (2016)

Nilashi, M.; Ibrahim, O.; Bagherifard, K.: A recommender system based on collaborative filtering using ontology and dimensionality reduction techniques. Expert Syst. Appl. 92, 507–520 (2018). https://doi.org/10.1016/j.eswa.2017.09.058CrossRef

Zoph, B.; Le, Q.V.: Neural architecture search with reinforcement learning. Cvpr 2019, 1–11 (2019)

Yao, Q.; Wang, M.; Chen, Y.; Dai, W.; Li, Y.F.; Tu, W.W.; Yang, Q.; Yu, Y.: Taking human out of learning applications: a survey on automated machine learning (2018). arXiv:1810.13306

Zöller, M.A.; Huber, M.F.: Benchmark and survey of automated machine learning frameworks. J. Artif. Intell. Res. 70, 409–472 (2021)MathSciNetCrossRef

He, X.; Zhao, K.; Chu, X.: AutoML: a survey of the state-of-the-art. Knowl.-Based Syst. 212, 106622 (2021)CrossRef

Lili, C.: Recommender algorithms based on boosting ensemble learning. Int. J. Smart Sens. Intell. Syst. 8, 368–386 (2015). https://doi.org/10.21307/ijssis-2017-763CrossRef

10.

Yu, K.; Schwaighofer, A.; Tresp, V.; Ma, W.; Zhang, H.: Collaborative ensemble learning: combining collaborative and content-based information filtering via hierarchical bayes. In: UAI’03 Proceedings of the Ninth Conference of Uncertainty in Artificial Intelligence, pp. 353–360 (2003).

11.

Schclar, A.; Tsikinovsky, A.; Rokach, L.; Meisels, A.; Antwarg, L.: Ensemble methods for improving the performance of neighborhood-based collaborative filtering. In: RecSys’09—Proceedings of the 3rd ACM Conference on Recommender Systems, pp. 261–264 (2009).

12.

Fortes, A.; Manzato, M.: Ensemble learning in recommender systems: combining multiple user interactions for ranking personalization. In: WebMedia 2014—Proceedings of the 20th Brazilian Symposium on Multimedia and the Web, pp. 47–54 (2014). https://doi.org/10.1145/2664551.2664556

13.

Hao, Y.; Zhang, P.; Zhang, F.: Multiview ensemble method for detecting shilling attacks in collaborative recommender systems. Commun. Netw. Secur. (2018) https://doi.org/10.1155/2018/8174603CrossRef

14.

Srikanth, T.; Shashi, M.: A scalable ensemble architecture for collaborative filtering in recommender systems. Int. J. Appl. Eng. Res. 11, 5103–5109 (2016)

15.

Ayaki, T.; Yanagimoto, H.; Yoshioka, M.: Recommendation from access logs with ensemble learning. Artif. Life Robot. 22, 163–167 (2017). https://doi.org/10.1007/s10015-016-0346-xCrossRef

16.

Da Costa, A.F.; Manzato, M.G.: Exploiting multimodal interactions in recommender systems with ensemble algorithms. Inf. Syst. 56, 120–132 (2016). https://doi.org/10.1016/j.is.2015.09.007CrossRef

17.

Tiemann, M.; Pauws, S.; Vignoli, F.: Ensemble learning for hybrid music recommendation. In: Proceedings of the 8th International Conference on Music Information Retrieval, ISMIR 2007, pp. 179–180 (2007)

18.

Wu, M.: Collaborative filtering via ensembles of matrix factorizations. In: KDD Cup and Workshop 2007, pp. 43–47 (2007)

19.

TahsinÖztürk, H.; Dede, T.; Türker, E.: Optimum design of reinforced concrete counterfort retaining walls using TLBO. Jaya Algor. Struct. 25, 285–296 (2020). https://doi.org/10.1016/j.istruc.2020.03.020CrossRef

20.

Atmaca, B.; Dede, T.; Grzywinski, M.: Optimization of cables size and prestressing force for a single pylon cable-stayed bridge with Jaya algorithm. Steel Compos. Struct. 34, 853–862 (2020). https://doi.org/10.12989/scs.2020.34.6.853CrossRef

21.

Dede, T.; Grzywinski, M.; Selejdak, J.: Continuous size optimization of large-scale dome structures with dynamic constraints. Struct. Eng. Mech. 73, 397–405 (2020). https://doi.org/10.12989/sem.2020.73.4.397CrossRef

22.

Kalemci, E.N.; İkizler, S.B.; Dede, T.; Angın, Z.: Design of reinforced concrete cantilever retaining wall using Grey wolf optimization algorithm. Structures 23, 245–253 (2020). https://doi.org/10.1016/j.istruc.2019.09.013CrossRef

23.

Sharafati, A.; Asadollah, S.B.H.S.; Hosseinzadeh, M.: The potential of new ensemble machine learning models for effluent quality parameters prediction and related uncertainty. Process. Saf. Environ. Prot. 140, 68–78 (2020). https://doi.org/10.1016/j.psep.2020.04.045CrossRef

24.

Tahir, M.F.; Haoyong, C.; Mehmood, K.; Larik, N.A.; Khan, A.; Javed, M.S.: Short term load forecasting using bootstrap aggregating based ensemble artificial neural network. Recent Adv. Electr. Electron. Eng. (Formerly Recent Patents Electr Electron. Eng. 13, 980–992 (2019). https://doi.org/10.2174/2213111607666191111095329CrossRef

25.

Shamshirband, S.; JafariNodoushan, E.; Adolf, J.E.; Abdul Manaf, A.; Mosavi, A.; Chau, K.: Wing: ensemble models with uncertainty analysis for multi-day ahead forecasting of chlorophyll a concentration in coastal waters. Eng. Appl. Comput. Fluid Mech. 13, 91–101 (2019). https://doi.org/10.1080/19942060.2018.1553742CrossRef

26.

Alizadeh, M.J.; JafariNodoushan, E.; Kalarestaghi, N.; Chau, K.W.: Toward multi-day-ahead forecasting of suspended sediment concentration using ensemble models. Environ. Sci. Pollut. Res. 24, 28017–28025 (2017). https://doi.org/10.1007/s11356-017-0405-4CrossRef

27.

Homsi, R.; Shiru, M.S.; Shahid, S.; Ismail, T.; Harun, S.B.; Al-Ansari, N.; Chau, K.W.; Yaseen, Z.M.: Precipitation projection using a CMIP5 GCM ensemble model: a regional investigation of Syria. Eng. Appl. Comput. Fluid Mech. 14, 90–106 (2020). https://doi.org/10.1080/19942060.2019.1683076CrossRef

28.

Wu, C.L.; Chau, K.W.: Prediction of rainfall time series using modular soft computingmethods. Eng. Appl. Artif. Intell. 26, 997–1007 (2013). https://doi.org/10.1016/j.engappai.2012.05.023CrossRef

29.

Lee, Y.; Kim, K.-J.: Product recommender systems using multi-model ensemble techniques. J. Intell. Inf. Syst. 19, 39–54 (2013). https://doi.org/10.13088/jiis.2013.19.2.039CrossRef

30.

Lommatzsch, A.: Real-time news recommendation using context-aware ensembles. Lecture Notes Computer Science (Including Subseries Lecture Notes Artificial Intelligence. Lecture Notes Bioinformatics). 8416 LNCS, pp. 51–62 (2014). https://doi.org/https://doi.org/10.1007/978-3-319-06028-6_5

31.

Nilashi, M.; Bagherifard, K.; Rahmani, M.; Rafe, V.: A recommender system for tourism industry using cluster ensemble and prediction machine learning techniques. Comput. Ind. Eng. 109, 357–368 (2017). https://doi.org/10.1016/j.cie.2017.05.016CrossRef

32.

Tsai, C.F.; Hung, C.: Cluster ensembles in collaborative filtering recommendation. Appl. Soft Comput. J. 12, 1417–1425 (2012). https://doi.org/10.1016/j.asoc.2011.11.016CrossRef

33.

Zhang, W.; Zou, H.; Luo, L.; Liu, Q.; Wu, W.; Xiao, W.: Predicting potential side effects of drugs by recommender methods and ensemble learning. Neurocomputing 173, 979–987 (2016). https://doi.org/10.1016/j.neucom.2015.08.054CrossRef

34.

Logesh, R.; Subramaniyaswamy, V.; Malathi, D.; Sivaramakrishnan, N.; Vijayakumar, V.: Enhancing recommendation stability of collaborative filtering recommender system through bio-inspired clustering ensemble method. Neural Comput. Appl. (2018). https://doi.org/10.1007/s00521-018-3891-5CrossRef

35.

Manzato, M.G.; Domingues, M.A.; Fortes, A.C.; Sundermann, C.V.; D’Addio, R.M.; Conrado, M.S.; Rezende, S.O.; Pimentel, M.G.C.: Mining unstructured content for recommender systems: an ensemble approach. Inf. Retr. J. 19, 378–415 (2016). https://doi.org/10.1007/s10791-016-9280-8CrossRef

36.

Vinagre, J.; Jorge, A.M.; Gama, J.: Online bagging for recommender systems. Expert Syst. 35, 1–13 (2018). https://doi.org/10.1111/exsy.12303CrossRef

37.

Wu, H.; Yue, K.; Pei, Y.; Li, B.; Zhao, Y.; Dong, F.: Collaborative topic regression with social trust ensemble for recommendation in social media systems. Knowl. Based Syst. 97, 111–122 (2016). https://doi.org/10.1016/j.knosys.2016.01.011CrossRef

38.

Zheng, L.; Li, L.; Hong, W.; Li, T.: PENETRATE: personalized news recommendation using ensemble hierarchical clustering. Expert Syst. Appl. 40, 2127–2136 (2013). https://doi.org/10.1016/j.eswa.2012.10.029CrossRef

39.

Gomes, H.M.; Barddal, J.P.; Enembreck, F.; Bifet, A.: A survey on ensemble learning for data stream classification. ACM Comput. Surv. 50, 1–36 (2017). https://doi.org/10.1145/3054925CrossRef

40.

Bauman, K.; Liu, B.; Tuzhilin, A.: Aspect based recommendations. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discover Data Mining—KDD ’17, pp. 717–725 (2017). https://doi.org/10.1145/3097983.3098170

41.

Kant, S.; Mahara, T.: Merging user and item based collaborative filtering to alleviate data sparsity. Int. J. Syst. Assur. Eng. Manag. 9, 173–179 (2018). https://doi.org/10.1007/s13198-016-0500-9CrossRef

42.

Paper, C.: Cosine similarity metric learning for face for face verication. Accv (2015). https://doi.org/10.1007/978-3-642-19309-5CrossRef

43.

Shmueli, E.; Tassa, T.: Secure multi-party protocols for item-based collaborative filtering. In: Proceedings of the Eleventh ACM Conference on Recommender Systems—RecSys ’17, pp. 89–97 (2017). https://doi.org/10.1145/3109859.3109881

44.

Wang, J.; De Vries, A.P.; Reinders, M.J.T.: Unifying user-based and item-based collaborative filtering approaches by similarity fusion. In: Proceedings of the Twenty-Ninth Annual International ACM SIGIR Conference on Research and development in information 2006, pp. 501–508 (2006). https://doi.org/10.1145/1148170.1148257

45.

Vozalis, M.G.; Margaritis, K.G.: Applying SVD on generalized item-based filtering. Int. J. Comput. Sci. Appl. Technomath. Res. Found. 3, 27–51 (2006)

46.

Marlin, B.M.: Modeling user rating profiles for collaborative filtering. In: Advances in Neural Information Processing Systems, vol. 16, pp. 627–634 (2003)

47.

Praveen Kumar, D.; Amgoth, T.; Annavarapu, C.S.R.: Machine learning algorithms for wireless sensor networks: a survey. Inf. Fusion. 49, 1–25 (2019). https://doi.org/10.1016/j.inffus.2018.09.013CrossRef

48.

Lopes, M.E.: Estimating the algorithmic variance of randomized ensembles via the bootstrap. Ann. Stat. 47, 1088–1112 (2019). https://doi.org/10.1214/18-AOS1707MathSciNetCrossRefMATH

49.

Bühlmann, P.: Bagging, boosting and ensemble methods. In: Handbook of Computational Statistics, pp. 985–1022. Springer, Berlin, Heidelberg (2012)CrossRef

50.

Al-Stouhi, S.; Reddy, C.K.: Adaptive boosting for transfer learning using dynamic updates. Lecture Notes Computer Science (including Subseries Lecture Notes Artificial Intelligence Lecture Notes Bioinformatics). 6911 LNAI, pp. 60–75 (2011). https://doi.org/10.1007/978-3-642-23780-5_14

51.

Beygelzimer, A.; Hazan, E.; Kale, S.; Luo, H.: Online gradient boosting. Adv. Neural Inf. Process. Syst. 2015, 2458–2466 (2015)

52.

Dembczyñski, K.; Cheng, W.; Hüllermeier, E.: Bayes optimal multilabel classification via probabilistic classifier chains. In: ICML 2010—Proceedings, 27th International Conference Machine Learning, pp. 279–286 (2010)

53.

Zhang, Y.; Wang, S.; Ji, G.: A comprehensive survey on particle swarm optimization algorithm and its applications. Math. Probl. Eng. (2015). https://doi.org/10.1155/2015/931256MathSciNetCrossRefMATH

54.

Vikhar, P.A.: Evolutionary algorithms : a critical review and its future prospects, pp. 261–265 (2016). https://doi.org/https://doi.org/10.1109/ICGTSPICC.2016.7955308

55.

Maulik, U.; Bandyopadhyay, S.: Genetic algorithm-based clustering technique. Pattern Recognit. 33, 1455–1465 (2000). https://doi.org/10.1016/S0031-3203(99)00137-5CrossRef

56.

Dorigo, M.; Stützle, T.: The Ant colony optimization metaheuristic: algorithms, applications, and advances. Handb. Metaheur. (2006). https://doi.org/10.1007/0-306-48056-5_9CrossRefMATH

57.

Karaboga, D.; Akay, B.: A comparative study of artificial bee colony algorithm. Appl. Math. Comput. 214, 108–132 (2009). https://doi.org/10.1016/j.amc.2009.03.090MathSciNetCrossRefMATH

58.

Shehab, M.; Khader, A.T.; Al-betar, M.A.: A survey on applications and variants of the cuckoo search algorithm. Appl. Soft Comput. J. (2017). https://doi.org/10.1016/j.asoc.2017.02.034CrossRef

59.

Wang, H.; Wang, W.; Zhou, X.; Sun, H.; Zhao, J.; Yu, X.; Cui, Z.: PT US CR. Inf. Sci. (Ny). (2016). https://doi.org/10.1016/j.ins.2016.12.024CrossRef

60.

Oftadeh, R.; Mahjoob, M.J.; Shariatpanahi, M.: A novel meta-heuristic optimization algorithm inspired by group hunting of animals: hunting search. Comput. Math. Appl. 60, 2087–2098 (2010). https://doi.org/10.1016/j.camwa.2010.07.049CrossRefMATH

61.

Jin, Y.; Zhang, Q.; Zhou, A.; Sendhoff, B.; Tsang, E.: Prediction-based population Re-initialization for evolutionary dynamic multi-objective optimization. Evol. Multi Criterion Optim. (2007). https://doi.org/10.1007/978-3-540-70928-2_62CrossRef

62.

Büche, D.; Schraudolph, N.N.; Koumoutsakos, P.: Accelerating evolutionary algorithms with Gaussian process fitness function models. IEEE Trans Syst. Man Cybern. Part C Appl. Rev. 35, 183–194 (2005). https://doi.org/10.1109/TSMCC.2004.841917CrossRef

63.

Tsutsui, S.; Yamamura, M.; Higuchi, T.: Multi-parent recombination with simplex crossover in real coded genetic algorithms. Proc. Genet. Evol. Comput. Conf. 1, 657–664 (1999)

64.

Corus, D.; Oliveto, P.S.: Standard steady state genetic algorithms can Hillclimb faster than mutation-only evolutionary algorithms. IEEE Trans. Evol. Comput. 22, 720–732 (2018). https://doi.org/10.1109/TEVC.2017.2745715CrossRef

65.

Wang, D.; Tan, D.; Liu, L.: Particle swarm optimization algorithm: an overview. Soft Comput. 22, 387–408 (2018). https://doi.org/10.1007/s00500-016-2474-6CrossRef

66.

Marini, F.; Walczak, B.: Particle swarm optimization (PSO). A tutorial. Chemom. Intell. Lab. Syst. 149, 153–165 (2015). https://doi.org/10.1016/j.chemolab.2015.08.020CrossRef

67.

Cazzaniga, P.; Nobile, M.S.; Besozzi, D.: The impact of particles initialization in PSO: parameter estimation as a case in point. In: 2015 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology CIBCB (2015). https://doi.org/10.1109/CIBCB.2015.7300288

68.

Hahsler, M.: recommenderlab: a framework for developing and testing recommendation algorithms (2015). http://elib.ict.nsc.ru/jspui/bitstream/ICT/1861/1/recommenderlab.pdf

69.

Iqbal, N.; Zerguine, A.; Al-Dhahir, N.: Decision feedback equalization using particle swarm optimization. Signal Process. 108, 1–12 (2015). https://doi.org/10.1016/j.sigpro.2014.07.030CrossRef

70.

Gupta, G.; Katarya, R.: Research on understanding the effect of deep learning on user preferences. Arab. J. Sci. Eng. (2020). https://doi.org/10.1007/s13369-020-05112-2CrossRef

Title: EnPSO: An AutoML Technique for Generating Ensemble Recommender System
Authors: Garima Gupta
Rahul Katarya
Publication date: 23-04-2021
Publisher: Springer Berlin Heidelberg
Published in: Arabian Journal for Science and Engineering / Issue 9/2021
Print ISSN: 2193-567X
Electronic ISSN: 2191-4281
DOI: https://doi.org/10.1007/s13369-021-05670-z

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Other articles of this Issue 9/2021

A Bit Addressable Register with Variable Write/Read Data widths

Involving Spaces of Unicode Standard Within Irreversible Arabic Text Steganography for Practical Implementations

The Do’s and Don’ts for Increasing the Accuracy of Face Recognition on VGGFace2 Dataset

An Efficient MAC with Spectrum Handoff and Frame Fragmentation Strategies for Cognitive Radio Networks

Multi-Level Cross-Architecture Binary Code Similarity Metric

P-STORE: Extension of STORE Methodology to Elicit Privacy Requirements

Premium Partners