Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Multimedia Systems
Published in: Multimedia Systems 2/2023

19-01-2023 | Special Issue Paper

SMPC: boosting social media popularity prediction with caption

Authors: An-An Liu, Xiaowen Wang, Ning Xu, Jing Liu, Yuting Su, Quan Zhang, Shenyuan Zhang, Yejun Tang, Junbo Guo, Guoqing Jin, Xuanya Li

Published in: Multimedia Systems | Issue 2/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Social media popularity prediction refers to using multi-modal content to predict the popularity of a post offered by an internet user. It is an effective way to explore advanced forecasting trends and make more popularity-sensitive strategic decisions for the future. Existing methods attempt to explore various multi-model features to solve this task, which only focus on local information, lacking global understanding for the post’s content. In this paper, we propose social media popularity prediction with caption (SMPC), a novel architecture that integrates the caption as the global representation into the existing multi-model-feature-based popularity prediction method. To make good use of the generated captions, we process them in word-level, sentence-level and length-level ways, obtaining three kinds of caption features. To incorporate caption features, we exploit seven variants of the architecture by concatenating features in all the possible manners, for the feature fusion and training different combinations for the CatBoost regression. Extensive experiments are conducted on Social Media Prediction Dataset (SMPD) and show that the proposed approaches can achieve competing results against state-of-the-art models.
previous article Improving text-image cross-modal retrieval with contrastive loss
next article Multi-window Transformer parallel fusion feature pyramid network for pedestrian orientation detection

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
Szabo, G., Huberman, B.A.: Predicting the popularity of online content. Commun. ACM 53(8), 80–88 (2010)CrossRef
2.
Kang, P., Lin, Z., Teng, S., Zhang, G., Guo, L., Zhang, W.: Catboost-based framework with additional user information for social media popularity prediction. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2677–2681 (2019)
3.
He, Z., He, Z., Wu, J., Yang, Z.: Feature construction for posts and users combined with lightgbm for social media popularity prediction. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2672–2676 (2019)
4.
Chen, J., Liang, D., Zhu, Z., Zhou, X., Ye, Z., Mo, X.: Social media popularity prediction based on visual-textual features with xgboost. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2692–2696 (2019)
5.
Wu, B., Mei, T., Cheng, W.-H., Zhang, Y.: Unfolding temporal dynamics: predicting social media popularity using multi-scale temporal decomposition. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)
6.
Li, L., Situ, R., Gao, J., Yang, Z., Liu, W.: A hybrid model combining convolutional neural network with xgboost for predicting social media popularity. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1912–1917 (2017)
7.
Bielski, A., Trzcinski, T.: Pay attention to virality: understanding popularity of social media videos with the attention mechanism. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2335–2337 (2018)
8.
Jing, P., Su, Y., Nie, L., Bai, X., Liu, J., Wang, M.: Low-rank multi-view embedding learning for micro-video popularity prediction. IEEE Trans. Knowl. Data Eng. 30(8), 1519–1532 (2017)CrossRef
9.
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
10.
Yang, J., Leskovec, J.: Patterns of temporal variation in online media. In: Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, pp. 177–186 (2011)
11.
Matsubara, Y., Sakurai, Y., Prakash, B.A., Li, L., Faloutsos, C.: Rise and fall patterns of information diffusion: model and implications. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 6–14 (2012)
12.
Figueiredo, F., Almeida, J.M., Gonçalves, M.A., Benevenuto, F.: Trendlearner: early prediction of popularity trends of user generated content. Inf. Sci. 349, 172–187 (2016)CrossRef
13.
Hu, Y., Hu, C., Fu, S., Shi, P., Ning, B.: Predicting the popularity of viral topics based on time series forecasting. Neurocomputing 210, 55–65 (2016)CrossRef
14.
Shen, H., Wang, D., Song, C., Barabási, A.-L.: Modeling and predicting popularity dynamics via reinforced Poisson processes. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 28 (2014)
15.
Gao, S., Ma, J., Chen, Z.: Modeling and predicting retweeting dynamics on microblogging platforms. In: Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, pp. 107–116 (2015)
16.
Bao, P., Shen, H.-W., Jin, X., Cheng, X.-Q.: Modeling and predicting popularity dynamics of microblogs using self-excited Hawkes processes. In: Proceedings of the 24th International Conference on World Wide Web, pp. 9–10 (2015)
17.
18.
Maki, D.P., Maki, D.P., Mali, D., Thompson, M., Thompson, M.: Mathematical Models and Applications: with Emphasis on the Social, Life, and Management Sciences. Prentice Hall (1973)
19.
Xiong, F., Liu, Y., Zhang, Z.-J., Zhu, J., Zhang, Y.: An information diffusion model based on retweeting mechanism for online social media. Phys. Lett. A 376(30–31), 2103–2108 (2012)CrossRef
20.
Wang, H., Li, Y., Feng, Z., Feng, L.: Retweeting analysis and prediction in microblogs: an epidemic inspired approach. China Commun. 10(3), 13–24 (2013)CrossRef
21.
Yang, J., Leskovec, J.: Modeling information diffusion in implicit networks. In: 2010 IEEE International Conference on Data Mining, pp. 599–608. IEEE (2010)
22.
Vilares, D., Alonso, M.A., Gómez-Rodríguez, C.: On the usefulness of lexical and syntactic processing in polarity classification of twitter messages. J. Am. Soc. Inf. Sci. 66(9), 1799–1816 (2015)
23.
Wu, B., Cheng, W.-H., Zhang, Y., Mei, T.: Time matters: Multi-scale temporalization of social media popularity. In: Proceedings of the 24th ACM International Conference on Multimedia, pp. 1336–1344 (2016)
24.
Wu, B., Cheng, W.-H., Zhang, Y., Huang, Q., Li, J., Mei, T.: Sequential prediction of social media popularity with deep temporal context networks. arXiv preprint arXiv:​1712.​04443 (2017)
25.
Zhang, W., Wang, W., Wang, J., Zha, H.: User-guided hierarchical attention network for multi-modal social image popularity prediction. In: Proceedings of the 2018 World Wide Web Conference, pp. 1277–1286 (2018)
26.
Zhang, D., Yao, L., Chen, K., Wang, S., Chang, X., Liu, Y.: Making sense of spatio-temporal preserving representations for EEG-based human intention recognition. IEEE Trans. Cybern. 50(7), 3033–3044 (2019)CrossRef
27.
Luo, M., Chang, X., Nie, L., Yang, Y., Hauptmann, A.G., Zheng, Q.: An adaptive semisupervised feature analysis for video semantic recognition. IEEE Trans. Cybern. 48(2), 648–660 (2017)CrossRef
28.
Chen, K., Yao, L., Zhang, D., Wang, X., Chang, X., Nie, F.: A semisupervised recurrent convolutional attention model for human activity recognition. IEEE Trans. Neural Netw. Learn. Syst. 31(5), 1747–1756 (2019)CrossRef
29.
Liu, Z., Wu, S., Jin, S., Liu, Q., Ji, S., Lu, S., Cheng, L.: Investigating pose representations and motion contexts modeling for 3D motion prediction. IEEE Trans. Pattern Anal. Mach. Intell. 45(1), 681–97 (2022)CrossRef
30.
Liu, A.-A., Zhou, H., Nie, W., Liu, Z., Liu, W., Xie, H., Mao, Z., Li, X., Song, D.: Hierarchical multi-view context modelling for 3D object classification and retrieval. Inf. Sci. 547, 984–995 (2021)CrossRef
31.
Rennie, S.J., Marcheret, E., Mroueh, Y., Ross, J., Goel, V.: Self-critical sequence training for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7008–7024 (2017)
32.
Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: a neural image caption generator. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156–3164 (2015)
33.
Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3128–3137 (2015)
34.
Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. Adv. Neural Inf. Process. Syst. 31 (2018)
35.
36.
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Process. Syst. 26 (2013)
37.
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:​1301.​3781 (2013)
38.
Wu, B., Cheng, W.-H., Liu, P., Liu, B., Zeng, Z., Luo, J.: Smp challenge: an overview of social media prediction challenge 2019. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2667–2671 (2019)
39.
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: common objects in context. In: European Conference on Computer Vision, pp. 740–755. Springer (2014)
40.
41.
Ding, K., Wang, R., Wang, S.: Social media popularity prediction: a multiple feature fusion approach with deep neural networks. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2682–2686 (2019)
42.
Xu, K., Lin, Z., Zhao, J., Shi, P., Deng, W., Wang, H.: Multimodal deep learning for social media popularity prediction with attention mechanism. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 4580–4584 (2020)
43.
Wang, K., Wang, P., Chen, X., Huang, Q., Mao, Z., Zhang, Y.: A feature generalization framework for social media popularity prediction. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 4570–4574 (2020)
Metadata
Title
SMPC: boosting social media popularity prediction with caption
Authors
An-An Liu
Xiaowen Wang
Ning Xu
Jing Liu
Yuting Su
Quan Zhang
Shenyuan Zhang
Yejun Tang
Junbo Guo
Guoqing Jin
Xuanya Li
Publication date
19-01-2023
Publisher
Springer Berlin Heidelberg
Published in
Multimedia Systems / Issue 2/2023
Print ISSN: 0942-4962
Electronic ISSN: 1432-1882
DOI
https://doi.org/10.1007/s00530-022-01030-5

Other articles of this Issue 2/2023

Multimedia Systems 2/2023 Go to the issue

Special Issue Paper

Attribute-aware style adaptation for person re-identification

Regular Paper

Guide and interact: scene-graph based generation and control of video captions

Special Issue Paper

Perturbation consistency and mutual information regularization for semi-supervised semantic segmentation

Special Issue Paper

Context-aware and ethics-first crowd mobility portraits over massive smart card data

Regular Paper

Multimodal metadata assignment for cultural heritage artifacts

Editorial

Special issue on human-centric intelligent multimedia understanding