Skip to main content
Top
Published in: World Wide Web 2/2024

01-03-2024

High-level preferences as positive examples in contrastive learning for multi-interest sequential recommendation

Authors: Zizhong Zhu, Shuang Li, Yaokun Liu, Xiaowang Zhang, Zhiyong Feng, Yuexian Hou

Published in: World Wide Web | Issue 2/2024

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The sequential recommendation task based on the multi-interest framework aims to model multiple interests of users from different aspects to predict their future interactions. However, researchers rarely consider the differences in features between the interests generated by the model. In extreme cases, all interest capsules have the same meaning, leading to the failure of modeling users with multiple interests. To address this issue, we propose the High-level Preferences as positive examples in Contrastive Learning for multi-interest Sequence Recommendation framework (HPCL4SR), which uses contrastive learning to distinguish differences in interests based on user item interaction information. In order to find high-quality comparative examples, this paper introduces the category information to construct a global graph, learning the association between categories for high-level preference interest of users. Then, a multi-layer perceptron is used to adaptively fuse the low-level preference interest features of the user’s items and the high-level preference interest features of the categories. Finally, user multi-interest contrastive samples are obtained through item sequence information and corresponding categories, which are fed into contrastive learning to optimize model parameters and generate multi-interest representations that are more in line with the user sequence. In addition, when modeling the user’s item sequence information, in order to increase the differentiation between item representations, the category of the item is used to supervise the learning process. Extensive experiments on three real datasets demonstrate that our method outperforms existing multi-interest recommendation models.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Wang, S., Hu, L., Wang, Y., Cao, L., Sheng, Q.Z., Orgun, M.A.: Sequential recommender systems: Challenges, progress and prospects. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence: IJCAI, pp. 6332–6338 (2019). https://doi.org/10.24963/ijcai.2019/883 Wang, S., Hu, L., Wang, Y., Cao, L., Sheng, Q.Z., Orgun, M.A.: Sequential recommender systems: Challenges, progress and prospects. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence: IJCAI, pp. 6332–6338 (2019). https://​doi.​org/​10.​24963/​ijcai.​2019/​883
3.
go back to reference Pi, Q., Bian, W., Zhou, G., Zhu, X., Gai, K.: Practice on long sequential user behavior modeling for click-through rate prediction. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining: KDD, pp. 2671–2679 (2019). https://doi.org/10.1145/3292500.3330666 Pi, Q., Bian, W., Zhou, G., Zhu, X., Gai, K.: Practice on long sequential user behavior modeling for click-through rate prediction. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining: KDD, pp. 2671–2679 (2019). https://​doi.​org/​10.​1145/​3292500.​3330666
4.
5.
6.
go back to reference Wu, C., Wu, F., Qi, T., Huang, Y.: User modeling with click preference and reading satisfaction for news recommendation. In: Proceedings of the twenty-ninth international joint conference on artificial intelligence: IJCAI, pp. 3023–3029 (2020). https://doi.org/10.5555/3491440.3491858 Wu, C., Wu, F., Qi, T., Huang, Y.: User modeling with click preference and reading satisfaction for news recommendation. In: Proceedings of the twenty-ninth international joint conference on artificial intelligence: IJCAI, pp. 3023–3029 (2020). https://​doi.​org/​10.​5555/​3491440.​3491858
10.
11.
go back to reference Yuan, F., Karatzoglou, A., Arapakis, I., Jose, J.M., He, X.: A simple convolutional generative network for next item recommendation. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining: WSDM, pp. 582–590 (2019). https://doi.org/10.1145/3289600.3290975 Yuan, F., Karatzoglou, A., Arapakis, I., Jose, J.M., He, X.: A simple convolutional generative network for next item recommendation. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining: WSDM, pp. 582–590 (2019). https://​doi.​org/​10.​1145/​3289600.​3290975
14.
15.
16.
go back to reference Li, C., Liu, Z., Wu, M., Xu, Y., Zhao, H., Huang, P., Kang, G., Chen, Q., Li, W., Lee, D.L.: Multi-interest network with dynamic routing for recommendation at tmall. In: Proceedings of the 28th ACM international conference on information and knowledge management: CIKM, pp. 2615–2623 (2019). https://doi.org/10.1145/3357384.3357814 Li, C., Liu, Z., Wu, M., Xu, Y., Zhao, H., Huang, P., Kang, G., Chen, Q., Li, W., Lee, D.L.: Multi-interest network with dynamic routing for recommendation at tmall. In: Proceedings of the 28th ACM international conference on information and knowledge management: CIKM, pp. 2615–2623 (2019). https://​doi.​org/​10.​1145/​3357384.​3357814
17.
go back to reference Xiao, Z., Yang, L., Jiang, W., Wei, Y., Hu, Y., Wang, H.: Deep multi-interest network for click-through rate prediction. In: CIKM ’20: The 29th ACM international conference on information and knowledge management, pp. 2265–2268 (2020). https://doi.org/10.1145/3340531.3412092 Xiao, Z., Yang, L., Jiang, W., Wei, Y., Hu, Y., Wang, H.: Deep multi-interest network for click-through rate prediction. In: CIKM ’20: The 29th ACM international conference on information and knowledge management, pp. 2265–2268 (2020). https://​doi.​org/​10.​1145/​3340531.​3412092
18.
19.
go back to reference Tang, Z., Wang, L., Zou, L., Zhang, X., Zhou, J., Li, C.: Towards multi-interest pre-training with sparse capsule network. In: Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval: SIGIR, pp. 311–320 (2023). https://doi.org/10.1145/3539618.3591778 Tang, Z., Wang, L., Zou, L., Zhang, X., Zhou, J., Li, C.: Towards multi-interest pre-training with sparse capsule network. In: Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval: SIGIR, pp. 311–320 (2023). https://​doi.​org/​10.​1145/​3539618.​3591778
20.
go back to reference Xie, Y., Gao, J., Zhou, P., Ye, Q., Hua, Y., Kim, J.B., Wu, F., Kim, S.: Rethinking multi-interest learning for candidate matching in recommender systems. In: Proceedings of the 17th ACM conference on recommender systems: RecSys, pp. 283–293 (2023). https://doi.org/10.1145/3604915.3608766 Xie, Y., Gao, J., Zhou, P., Ye, Q., Hua, Y., Kim, J.B., Wu, F., Kim, S.: Rethinking multi-interest learning for candidate matching in recommender systems. In: Proceedings of the 17th ACM conference on recommender systems: RecSys, pp. 283–293 (2023). https://​doi.​org/​10.​1145/​3604915.​3608766
21.
go back to reference Lv, F., Jin, T., Yu, C., Sun, F., Lin, Q., Yang, K., Ng, W.: SDM: sequential deep matching model for online large-scale recommender system. In: Proceedings of the 28th ACM international conference on information and knowledge management: CIKM, pp. 2635–2643 (2019). https://doi.org/10.1145/3357384.3357818 Lv, F., Jin, T., Yu, C., Sun, F., Lin, Q., Yang, K., Ng, W.: SDM: sequential deep matching model for online large-scale recommender system. In: Proceedings of the 28th ACM international conference on information and knowledge management: CIKM, pp. 2635–2643 (2019). https://​doi.​org/​10.​1145/​3357384.​3357818
22.
go back to reference Tan, Q., Zhang, J., Yao, J., Liu, N., Zhou, J., Yang, H., Hu, X.: Sparse-interest network for sequential recommendation. In: WSDM ’21, The Fourteenth ACM international conference on web search and data mining: WSDM, pp. 598–606 (2021). https://doi.org/10.1145/3437963.3441811 Tan, Q., Zhang, J., Yao, J., Liu, N., Zhou, J., Yang, H., Hu, X.: Sparse-interest network for sequential recommendation. In: WSDM ’21, The Fourteenth ACM international conference on web search and data mining: WSDM, pp. 598–606 (2021). https://​doi.​org/​10.​1145/​3437963.​3441811
23.
go back to reference Chen, G., Zhang, X., Zhao, Y., Xue, C., Xiang, J.: Exploring periodicity and interactivity in multi-interest framework for sequential recommendation. In: Proceedings of the thirtieth international joint conference on artificial intelligence: IJCAI, pp. 1426–1433 (2021). https://doi.org/10.24963/ijcai.2021/197 Chen, G., Zhang, X., Zhao, Y., Xue, C., Xiang, J.: Exploring periodicity and interactivity in multi-interest framework for sequential recommendation. In: Proceedings of the thirtieth international joint conference on artificial intelligence: IJCAI, pp. 1426–1433 (2021). https://​doi.​org/​10.​24963/​ijcai.​2021/​197
24.
go back to reference Qiu, R., Huang, Z., Yin, H., Wang, Z.: Contrastive learning for representation degeneration problem in sequential recommendation. In: WSDM ’22: The fifteenth acm international conference on web search and data mining: WSDM, pp. 813–823 (2022). https://doi.org/10.1145/3488560.3498433 Qiu, R., Huang, Z., Yin, H., Wang, Z.: Contrastive learning for representation degeneration problem in sequential recommendation. In: WSDM ’22: The fifteenth acm international conference on web search and data mining: WSDM, pp. 813–823 (2022). https://​doi.​org/​10.​1145/​3488560.​3498433
25.
go back to reference Chai, Z., Chen, Z., Li, C., Xiao, R., Li, H., Wu, J., Chen, J., Tang, H.: User-aware multi-interest learning for candidate matching in recommenders. In: SIGIR ’22: The 45th international acm sigir conference on research and development in information retrieval: SIGIR, pp. 1326–1335 (2022). https://doi.org/10.1145/3477495.3532073 Chai, Z., Chen, Z., Li, C., Xiao, R., Li, H., Wu, J., Chen, J., Tang, H.: User-aware multi-interest learning for candidate matching in recommenders. In: SIGIR ’22: The 45th international acm sigir conference on research and development in information retrieval: SIGIR, pp. 1326–1335 (2022). https://​doi.​org/​10.​1145/​3477495.​3532073
26.
go back to reference Li, L., Zhang, Y., Liu, D., Chen, L.: Large language models for generative recommendation: A survey and visionary discussions. CoRR abs/2309.01157 (2023) https://doi.org/10.48550/ARXIV.2309.01157 https://arxiv.org/abs/2309.011572309.01157 Li, L., Zhang, Y., Liu, D., Chen, L.: Large language models for generative recommendation: A survey and visionary discussions. CoRR abs/2309.01157 (2023) https://​doi.​org/​10.​48550/​ARXIV.​2309.​01157 https://​arxiv.​org/​abs/​2309.​011572309.​01157
27.
go back to reference Hou, Y., Zhang, J., Lin, Z., Lu, H., Xie, R., McAuley, J.J., Zhao, W.X.: Large language models are zero-shot rankers for recommender systems. CoRR abs/2305.08845 (2023) https://doi.org/10.48550/ARXIV.2305.08845 https://arxiv.org/abs/2305.088452305.08845 Hou, Y., Zhang, J., Lin, Z., Lu, H., Xie, R., McAuley, J.J., Zhao, W.X.: Large language models are zero-shot rankers for recommender systems. CoRR abs/2305.08845 (2023) https://​doi.​org/​10.​48550/​ARXIV.​2305.​08845 https://​arxiv.​org/​abs/​2305.​088452305.​08845
29.
go back to reference Bao, K., Zhang, J., Zhang, Y., Wang, W., Feng, F., He, X.: Tallrec: An effective and efficient tuning framework to align large language model with recommendation. In: Zhang, J., Chen, L., Berkovsky, S., Zhang, M., Noia, T.D., Basilico, J., Pizzato, L., Song, Y. (eds.) Proceedings of the 17th ACM conference on recommender systems, RecSys 2023, Singapore, Singapore, September 18-22, 2023, pp. 1007–1014. ACM, ??? (2023). https://doi.org/10.1145/3604915.3608857 Bao, K., Zhang, J., Zhang, Y., Wang, W., Feng, F., He, X.: Tallrec: An effective and efficient tuning framework to align large language model with recommendation. In: Zhang, J., Chen, L., Berkovsky, S., Zhang, M., Noia, T.D., Basilico, J., Pizzato, L., Song, Y. (eds.) Proceedings of the 17th ACM conference on recommender systems, RecSys 2023, Singapore, Singapore, September 18-22, 2023, pp. 1007–1014. ACM, ??? (2023). https://​doi.​org/​10.​1145/​3604915.​3608857
32.
go back to reference Hjelm, R.D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., Bengio, Y.: Learning deep representations by mutual information estimation and maximization. In: 7th International conference on learning representations: ICLR (2019). https://openreview.net/forum?id=Bklr3j0cKX Hjelm, R.D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., Bengio, Y.: Learning deep representations by mutual information estimation and maximization. In: 7th International conference on learning representations: ICLR (2019). https://​openreview.​net/​forum?​id=​Bklr3j0cKX
36.
go back to reference Yan, Y., Li, R., Wang, S., Zhang, F., Wu, W., Xu, W.: Consert: A contrastive framework for self-supervised sentence representation transfer. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing: ACL/IJCNLP, pp. 5065–5075 (2021). https://doi.org/10.18653/v1/2021.acl-long.393 Yan, Y., Li, R., Wang, S., Zhang, F., Wu, W., Xu, W.: Consert: A contrastive framework for self-supervised sentence representation transfer. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing: ACL/IJCNLP, pp. 5065–5075 (2021). https://​doi.​org/​10.​18653/​v1/​2021.​acl-long.​393
38.
go back to reference Zhang, Y., Zhu, H., Wang, Y., Xu, N., Li, X., Zhao, B.: A contrastive framework for learning sentence representations from pairwise and triple-wise perspective in angular space. In: Proceedings of the 60th annual meeting of the association for computational linguistics: ACL, pp. 4892–4903 (2022). https://doi.org/10.18653/v1/2022.acl-long.336 Zhang, Y., Zhu, H., Wang, Y., Xu, N., Li, X., Zhao, B.: A contrastive framework for learning sentence representations from pairwise and triple-wise perspective in angular space. In: Proceedings of the 60th annual meeting of the association for computational linguistics: ACL, pp. 4892–4903 (2022). https://​doi.​org/​10.​18653/​v1/​2022.​acl-long.​336
40.
go back to reference Du, H., Shi, H., Zhao, P., Wang, D., Sheng, V.S., Liu, Y., Liu, G., Zhao, L.: Contrastive learning with bidirectional transformers for sequential recommendation. In: Proceedings of the 31st ACM international conference on information & knowledge management: CIKM, pp. 396–405 (2022). https://doi.org/10.1145/3511808.3557266 Du, H., Shi, H., Zhao, P., Wang, D., Sheng, V.S., Liu, Y., Liu, G., Zhao, L.: Contrastive learning with bidirectional transformers for sequential recommendation. In: Proceedings of the 31st ACM international conference on information & knowledge management: CIKM, pp. 396–405 (2022). https://​doi.​org/​10.​1145/​3511808.​3557266
42.
go back to reference Yuan, X., Duan, D., Tong, L., Shi, L., Zhang, C.: ICAI-SR: item categorical attribute integrated sequential recommendation. In: SIGIR ’21: The 44th international ACM SIGIR conference on research and development in information retrieval: SIGIR, pp. 1687–1691 (2021). https://doi.org/10.1145/3404835.3463060 Yuan, X., Duan, D., Tong, L., Shi, L., Zhang, C.: ICAI-SR: item categorical attribute integrated sequential recommendation. In: SIGIR ’21: The 44th international ACM SIGIR conference on research and development in information retrieval: SIGIR, pp. 1687–1691 (2021). https://​doi.​org/​10.​1145/​3404835.​3463060
43.
go back to reference Xie, Y., Zhou, P., Kim, S.: Decoupled side information fusion for sequential recommendation. In: SIGIR ’22: The 45th international ACM SIGIR conference on research and development in information retrieval :SIGIR, pp. 1611–1621 (2022). https://doi.org/10.1145/3477495.3531963 Xie, Y., Zhou, P., Kim, S.: Decoupled side information fusion for sequential recommendation. In: SIGIR ’22: The 45th international ACM SIGIR conference on research and development in information retrieval :SIGIR, pp. 1611–1621 (2022). https://​doi.​org/​10.​1145/​3477495.​3531963
47.
go back to reference Jean, S., Cho, K., Memisevic, R., Bengio, Y.: On using very large target vocabulary for neural machine translation. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing of the asian federation of natural language processing: ACL, pp. 1–10 (2015). https://doi.org/10.3115/v1/P15-1001 Jean, S., Cho, K., Memisevic, R., Bengio, Y.: On using very large target vocabulary for neural machine translation. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing of the asian federation of natural language processing: ACL, pp. 1–10 (2015). https://​doi.​org/​10.​3115/​v1/​P15-1001
Metadata
Title
High-level preferences as positive examples in contrastive learning for multi-interest sequential recommendation
Authors
Zizhong Zhu
Shuang Li
Yaokun Liu
Xiaowang Zhang
Zhiyong Feng
Yuexian Hou
Publication date
01-03-2024
Publisher
Springer US
Published in
World Wide Web / Issue 2/2024
Print ISSN: 1386-145X
Electronic ISSN: 1573-1413
DOI
https://doi.org/10.1007/s11280-024-01263-6

Other articles of this Issue 2/2024

World Wide Web 2/2024 Go to the issue

Premium Partner