Skip to main content
SpringerLink
Log in

JointCTR: a joint CTR prediction framework combining feature interaction and sequential behavior learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Click-through rate (CTR) is a positive feedback of user preferences or product purchases, and its small increase can bring huge benefits. Therefore, CTR prediction plays a key role in computing advertising and recommendation systems. Research shows that the accuracy of CTR prediction models is closely related to the input features. Existing related models usually focus on certain aspects of features, such as second-order interactions or temporal changes, and ignore the diversity of features. In this paper, a modular click-through rate prediction framework JointCTR is proposed. The framework integrates four types of prediction models, each of which learns different types of features, including original features, embedded features, interactive features, and sequential features. According to actual application scenarios, these models can be assembled or removed flexibly, and models of the same type can be replaced by each other. In order to avoid the impact of data diversity, sparsity, and high-dimensional features, an embedding layer is added to the framework to achieve unified embedding processing for different data types. To learn sequential behaviors, we propose a model SeqCTR based on the attention mechanism to capture the dynamics of user interests in the framework. To better learn high-order features, we apply HorderCTR proposed in our previous work to the framework to automatically identify high-value feature combinations. Extensive experiments on four public datasets show the effectiveness of the proposed framework, which yields competitive performance compared to state-of-the-art models (the AUC increases by + 0.24% on MovieLens-1 M and 0.85% on Criteo). Almost all different types of popular models can be assembled into it, that shows the flexibility and scalability of the framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. https://grouplens.org/datasets/movielens/

  2. https://www.kaggle.com/c/criteo-display-ad-challenge

  3. https://www.kaggle.com/c/avazu-ctr-prediction

References

  1. Guo W, Han J, Tang R, et al. (2019) Order-aware embedding neural network for CTR prediction. In: Proc. of the 42th International ACM SIGIR Conference on Research and Development in Information Retrieval

  2. Ktena S I, Myana P K, Tejani A, et al. (2019) Addressing delayed feedback for continuous training with neural networks in CTR prediction. In: Proc. of the 13th ACM Conference on Recommender Systems (RecSys)

  3. Bai X, Abasi R, Edizel B, Mantrach A (2019) Position-aware deep character-level CTR prediction for sponsored search. IEEE Trans Knowl Data Eng 33:1722–1736. https://doi.org/10.1109/TKDE.2019.2941881

    Article  Google Scholar 

  4. Pan F, Li S, Ao X, et al. (2019) Warm up cold-start advertisements: Improving CTR predictions via learning to learn ID embeddings. In: Proc. of the 42th International ACM SIGIR Conference on Research and Development in Information Retrieval (RecSys)

  5. Yang X, Deng T, Tan W, et al. (2019) Learning compositional, visual and relational representations for CTR prediction in sponsored search. In: Proc. of the 28th Conference on Information and Knowledge Management (CIKM)

  6. Andrew G, Gao J (2007) Scalable training of L1-regularized log-linear models. In: Proc. of the 24th International Conference on Machine Learning (ICML)

  7. He X, Pan J, Jin O, et al. (2014) Practical lessons from predicting clicks on ads at facebook. In: Proceedings of the 8th International Workshop on Data Mining for Online Advertising, (ADKDD) - in Conjunction with SIGKDD

  8. Gai K, Zhu X, Li H, et al. (2017) Learning piece-wise linear models from large scale data for ad click prediction. arXiv:1704.05194

  9. Rendle S (2011) Factorization machines with libFM. ACM Trans Intell Syst Technol 3(3):57

    Google Scholar 

  10. Juan Y, Zhuang Y, Chin W, et al. (2016) Field-aware factorization machines for CTR prediction. In: Proc. of the 10th ACM Conference on Recommender Systems (RecSys)

  11. Yan C, Zhang Q, Zhao X, Huang Y (2017) An intelligent field-aware factorization machine model. In: Proc. of the 22nd International Conference on Database Systems for Advanced Applications (DASFAA)

  12. He X, Liao L, Zhang H, et al. (2017) Neural collaborative filtering. In: Proc. of the 26th International World Wide Web Conference (WWW)

  13. Qu Y, Cai H, Ren K, et al. (2016) Product-based neural networks for user response prediction. In: Proc. of the 16th IEEE International Conference on Data Mining (ICDM)

  14. Cheng H, Koc L, Harmsen J, et al. (2016) Wide & deep learning for recommender systems. In: Proc. of the 1st Workshop on Deep Learning for Recommender Systems (DLRS) - in Conjunction with RecSys

  15. Wang R, Fu G, Fu B, et al. (2017) Deep & cross network for ad click predictions. In: Proc. of the 2017 AdKDD and TargetAd - In Conjunction with ACM SIGKDD

  16. Zhang W, Du T, Wang J (2016) Deep learning over multi-field categorical data - A case study on user response prediction. In: Proc. of the 38th European Conference on Information Retrieval (ECIR)

  17. He X, Chua TS (2017) Neural factorization machines for sparse predictive analytics. In: Proc. of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)

  18. Guo H, Tang R, Ye Y, et al. (2017) DeepFM: A factorization-machine based neural network for CTR prediction. In: Proc. of the 26th International Joint Conference on Artificial Intelligence (IJCAI)

  19. Li Z, Cui Z, Wu S, et al. (2019) Fi-GNN: Modeling feature interactions via graph neural networks for CTR prediction. In: Proc. of the 8th Conference on Information and Knowledge Management (CIKM)

  20. Huang T, Zhang Z, Zhang J, et al. (2019) FiBiNET: Combining feature importance and bilinear feature interaction for click-through rate prediction. In: Proc. of the 13th ACM Conference on Recommender Systems (RecSys)

  21. Yang Y, Shen S, Shen F, Zhao J et al (2020) Operation-aware neural networks for user response prediction. Neural Netw 121(1):161–168

    Article  Google Scholar 

  22. Liu B, Yu J, Tang R, Guo H, Chen Y, Zhang Y, et al. (2019) Feature generation by convolutional neural network for click-through rate prediction. In: Proc. of 2019 World Wide Web Conference (WWW)

  23. Wang R, Shivanna R, Cheng D Z, et al. (2020) DCN V2: Improved Deep & Cross Network for Feature Cross Learning in Web-scale Learning to Rank Systems. arXiv preprint arXiv:2008.13535

  24. Vaswani A, Shazeer N, Parmar N, et al. (2017) Attention is all you need. In: Proc. of the Conference on Advances in Neural Information Processing Systems (NIPS)

  25. Xiao J, Ye H, He X, et al. (2017) Attentional factorization machines: Learning the weight of feature interactions via attention networks. In: Proc. of the 26th International Joint Conference on Artificial Intelligence (IJCAI)

  26. Zhou G, Fan Y, Yan Y, et al. (2018) Deep interest network for click-through rate prediction. In: Proc. of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

  27. Song W, Duan Z, Xu Y, et al. (2019) AutoInt: Automatic feature interaction learning via self-attentive neural networks. In: Proc. of the 28th ACM International Conference on Information and Knowledge Management (CIKM)

  28. Li H, Duan H, Zheng Y, Wang Q, Wang Y (2020) A CTR prediction model based on user interest via attention mechanism. Appl Intell 50(4):1192–1203

    Article  Google Scholar 

  29. Wang Q, Liu F, Huang P et al (2020) A hierarchical attention model for CTR prediction based on user interest. IEEE Syst J 14(3):4015–4024

    Article  Google Scholar 

  30. Polikar R (2006) Essemble based systems in decision making[J]. IEEE Circ Syst Mag 6(3):21–45

    Article  Google Scholar 

  31. Hidasi B, Karatzoglou A, Baltrunas L, et al. (2016) Session-based recommendations with recurrent neural networks. In: Proc. of the 4th International Conference on Learning Representations (ICLR)

  32. Song Y, Elkahky A M, He X (2016) Multi-rate deep learning for temporal recommendation. In: Proc. of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval

  33. Yu F, Liu Q, Wu S, et al. A dynamic recurrent model for next basket recommendation. In: Proc. of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, 2016

  34. Zhou C, Bai J, Song J, et al. (2018) Atrank: An attention-based user behavior modeling framework for recommendation. In: Proc. of the 32nd AAAI Conference on Artificial Intelligence

  35. Xiong C, Merity S, Socher R (2016) Dynamic memory networks for visual and textual question answering. In: Proc. of the 33rd International Conference on International Conference on Machine Learning (ICML)

  36. Zhou G, Mou N, Fan Y, et al. (2019) Deep interest evolution network for click-through rate prediction. In: Proc. of the 33rd AAAI conference on Artificial Intelligence

  37. Yan C, Chen Y, Wan Y, et al. (2020) Modeling low- and high-order feature interactions with FM and self-attention network. Appl Intell https://doi.org/10.1007/s10489-020-01951-6

  38. Lian J, Chen Z, Zhou X, et al. (2018) xDeepFM: combining explicit and implicit feature interactions for recommender systems. In: Proc. of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Download references

Funding

This work was supported in part by Natural Science Foundation of Shanghai under Grant 19ZR1401900, and Shanghai Science and Technology Innovation Action Plan Project under Grant 19511101802.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Donghua University, Shanghai, 201620, China

    Cairong Yan, Xiaoke Li, Yizhou Chen & Yanting Zhang

Authors
  1. Cairong Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoke Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yizhou Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanting Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cairong Yan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, C., Li, X., Chen, Y. et al. JointCTR: a joint CTR prediction framework combining feature interaction and sequential behavior learning . Appl Intell 52, 4701–4714 (2022). https://doi.org/10.1007/s10489-021-02678-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02678-8

Keywords

Search

Navigation