Abstract
As an information filtering tool, recommendation system can present interesting contents to specific users through utilizing community users’ information. Due to the increasingly strict collection of user privacy information and improvement of related policies, data are scattered in different organizations as data islands, making it difficult to train a reliable recommendation system model. As for federated learning, an emerging machine learning approach, it enables clients to co-train the model by uploading gradients, which avoids the server to collect sensitive data from clients. To address the problem of not independent and identically distribution in federated learning, we propose a federated recommendation system based on the clustering of historical parameters. The clients perform a weighted average of the historical learning parameters with the global parameters sent by the server through using the time decay factor. The server performs parameter aggregation and clustering on the received parameters. The system performs iterative training based on the users’ historical learning parameters. In addition, when it comes to the problem that the server lacks raw data and cannot provide personalized recommendations for users in the federated recommendation system, we propose a recommendation system model based on user embedding features. The server can use user embedding features for personalized recommendations and it cannot get users’ data through user embedding features. The clients use original data for the local personalized recommendations. We conduct experiments on the real dataset MovieLens-1M. The experimental results show that the proposed federated learning approach is better than the traditional federated learning approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The dataset used in this experiment is MovieLens-1M, which is a open source dataset for deep learning network testing, and can be downloaded from related websites. The dataset used in this experiment is downloaded from GroupLens platform: https://grouplens.org/datasets/movielens/1m/.
References
Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Transact Knowl Data Eng 17(6):734–749. https://doi.org/10.1109/TKDE.2005.99
Ammad-ud-din M, Ivannikova E, Khan SA, Oyomno W, Fu Q, Tan KE, Flanagan A (2019) Federated collaborative filtering for privacy-preserving personalized recommendation system. CoRR abs/1901.09888, arXiv:1901.09888
Anelli VW, Deldjoo Y, Noia TD, Ferrara A, Narducci F (2021) Federank: user controlled feedback with federated recommender systems. In: Hiemstra D, Moens M, Mothe J, Perego R, Potthast M, Sebastiani F (eds) Advances in information retrieval - 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28 - April 1, 2021, Proceedings, Part I, Springer, Lecture Notes in Computer Science, vol 12656, pp 32–47, https://doi.org/10.1007/978-3-030-72113-8_3
Balabanovic M, Shoham Y (1997) Fab: content-based, collaborative recommendation. Commun ACM 40:66–72
Basu C, Hirsh H, Cohen W (1998) Recommendation as classification: using social and content-based information in recommendation. In: In Proceedings of the Fifteenth National Conference on Artificial Intelligence, AAAI Press, pp 714–720
Billsus D, Pazzani MJ (1998) Learning collaborative information filters. In: In Proc. 15th International Conf. on Machine Learning, Morgan Kaufmann, pp 46–54
Bonawitz KA, Eichner H, Grieskamp W, Huba D, Ingerman A, Ivanov V, Kiddon C, Konečný J, Mazzocchi S, McMahan B, Overveldt TV, Petrou D, Ramage D, Roselander J (2019) Towards federated learning at scale: system design. In: Talwalkar A, Smith V, Zaharia M (eds) Proceedings of machine learning and systems 2019, MLSys 2019, Stanford, CA, USA, March 31 - April 2, 2019, mlsys.org
Brendan McMahan H, Moore E, Ramage D, Hampson S, Agüera y Arcas B (2016) Communication-efficient learning of deep networks from decentralized data. arXiv e-prints arXiv:1602.05629
Chai D, Wang L, Chen K, Yang Q (2021) Secure federated matrix factorization. IEEE Intell Syst 36(5):11–20. https://doi.org/10.1109/MIS.2020.3014880
Chiu TC, Shih YY, Pang AC, Wang CS, Weng W, Chou CT (2020) Semisupervised distributed learning with non-iid data for aiot service platform. IEEE Internet Things J 7(10):9266–9277. https://doi.org/10.1109/JIOT.2020.2995162
Dinh DT, Fujinami T, Huynh VN (2019) Estimating the optimal number of clusters in categorical data clustering by silhouette coefficient. In: Chen J, Huynh VN, Nguyen GN, Tang X (eds) Knowledge and systems sciences. Springer Singapore, Singapore, pp 1–17
Dinh DT, Huynh VN, Sriboonchitta S (2021) Clustering mixed numerical and categorical data with missing values. Inform Sci 571:418–442. https://doi.org/10.1016/j.ins.2021.04.076
Goldberg D, Nichols D, Oki BM, Terry D (1992) Using collaborative filtering to weave an information tapestry. Commun ACM 35(12):61–70. https://doi.org/10.1145/138859.138867
Holzinger A, Malle B, Saranti A, Pfeifer B (2021) Towards multi-modal causability with graph neural networks enabling information fusion for explainable ai. Inform Fusion 71:28–37. https://doi.org/10.1016/j.inffus.2021.01.008
Huang L, Jiang B, Lv S, Liu Y, Li D (2018) Review of recommendation system based on deep learning. J Comput Sci 041(007):1619–1647
Huang Y, Chu L, Zhou Z, Wang L, Liu J, Pei J, Zhang Y (2021) Personalized cross-silo federated learning on non-iid data. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, AAAI Press, pp 7865–7873, https://ojs.aaai.org/index.php/AAAI/article/view/16960
Karlgren J (1990) An algebra for recommendations: using reader data as a basis for measuring document proximity. SYSLAB Tech Rep 179:1-11
Konečný J, McMahan HB, Yu FX, Richtárik P, Suresh AT, Bacon D (2016) Federated learning: strategies for improving communication efficiency. CoRR abs/1610.05492, arXiv:1610.05492
Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8):30–37. https://doi.org/10.1109/MC.2009.263
Li T, Sahu AK, Talwalkar A, Smith V (2020a) Federated learning: challenges, methods, and future directions. IEEE Signal Proces Mag 37(3):50–60. https://doi.org/10.1109/MSP.2020.2975749
Li T, Sahu AK, Zaheer M, Sanjabi M, Talwalkar A, Smith V (2020b) Federated optimization in heterogeneous networks. In: Dhillon I, Papailiopoulos D, Sze V (eds) Proceedings of Machine Learning and Systems, pp 429–450
McMahan B, Moore E, Ramage D, Hampson S, y Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In: Singh A, Zhu XJ (eds) Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA, PMLR, Proceedings of Machine Learning Research, vol 54, pp 1273–1282
Naumov M, Mudigere D, Shi HJM, Huang J, Sundaraman N, Park J, Wang X, Gupta U, Wu CJ, Azzolini AG, Dzhulgakov D, Mallevich A, Cherniavskii I, Lu Y, Krishnamoorthi R, Yu A, Kondratenko V, Pereira S, Chen X, Chen W, Rao V, Jia B, Xiong L, Smelyanskiy M (2019) Deep learning recommendation model for personalization and recommendation systems. arXiv e-prints arXiv:1906.00091
Pazzani MJ, Billsus D (2007) Content-based recommendation systems. The adaptive web: methods and strategies of web personalization. Volume 4321 of lecture notes in computer science. Springer-Verlag, Berlin, pp 325–341
Qi T, Wu F, Wu C, Huang Y, Xie X (2020) Privacy-preserving news recommendation model training via federated learning. CoRR abs/2003.09592, arXiv:2003.09592
Resnick P, Iacovou N, Suchak M, Bergstrom P, Riedl J (1994) Grouplens: An open architecture for collaborative filtering of netnews. In: Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work, Association for Computing Machinery, New York, NY, USA, CSCW ’94, p 175-186, https://doi.org/10.1145/192844.192905
Sarwar B, Karypis G, Konstan J, Riedl J (2001) Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web, Association for Computing Machinery, New York, NY, USA, WWW ’01, p 285-295, https://doi.org/10.1145/371920.372071
Su X, Khoshgoftaar TM (2009) A survey of collaborative filtering techniques. Adv in Artif Intell. https://doi.org/10.1155/2009/421425
Xu H, Wu X, Li X, Yan B (2009) A comparative study of internet recommendation systems. J Softw 20(002):350–362
Yang Q, Liu Y, Chen T, Tong Y (2019) Federated machine learning: concept and applications. ACM Trans Intell Syst Technol. https://doi.org/10.1145/3298981
Yi J, Wu F, Wu C, Liu R, Sun G, Xie X (2021) Efficient-fedrec: Efficient federated learning framework for privacy-preserving news recommendation. In: Moens M, Huang X, Specia L, Yih SW (eds) Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, Association for Computational Linguistics, pp 2814–2824
Zhang S, Yao L, Sun A, Tay Y (2019) Deep learning based recommender system: a survey and new perspectives. ACM Comput Surv. https://doi.org/10.1145/3285029
Zhao Y, Li M, Lai L, Suda N, Civin D, Chandra V (2018) Federated learning with non-iid data. CoRR abs/1806.00582, arxiv:1806.00582
Acknowledgements
This work was supported in part by the National Key Research and Development Program of China (No. 2020YFB1005804), in part by the National Natural Science Foundation of China under Grant 61632009 , and in part by the Guangdong Provincial Natural Science Foundation under Grant 2017A030308006.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jie, Z., Chen, S., Lai, J. et al. Personalized federated recommendation system with historical parameter clustering. J Ambient Intell Human Comput 14, 10555–10565 (2023). https://doi.org/10.1007/s12652-022-03709-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-022-03709-z