Skip to main content
Erschienen in: Multimedia Systems 2/2023

19.01.2023 | Special Issue Paper

SMPC: boosting social media popularity prediction with caption

verfasst von: An-An Liu, Xiaowen Wang, Ning Xu, Jing Liu, Yuting Su, Quan Zhang, Shenyuan Zhang, Yejun Tang, Junbo Guo, Guoqing Jin, Xuanya Li

Erschienen in: Multimedia Systems | Ausgabe 2/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Social media popularity prediction refers to using multi-modal content to predict the popularity of a post offered by an internet user. It is an effective way to explore advanced forecasting trends and make more popularity-sensitive strategic decisions for the future. Existing methods attempt to explore various multi-model features to solve this task, which only focus on local information, lacking global understanding for the post’s content. In this paper, we propose social media popularity prediction with caption (SMPC), a novel architecture that integrates the caption as the global representation into the existing multi-model-feature-based popularity prediction method. To make good use of the generated captions, we process them in word-level, sentence-level and length-level ways, obtaining three kinds of caption features. To incorporate caption features, we exploit seven variants of the architecture by concatenating features in all the possible manners, for the feature fusion and training different combinations for the CatBoost regression. Extensive experiments are conducted on Social Media Prediction Dataset (SMPD) and show that the proposed approaches can achieve competing results against state-of-the-art models.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Szabo, G., Huberman, B.A.: Predicting the popularity of online content. Commun. ACM 53(8), 80–88 (2010)CrossRef Szabo, G., Huberman, B.A.: Predicting the popularity of online content. Commun. ACM 53(8), 80–88 (2010)CrossRef
2.
Zurück zum Zitat Kang, P., Lin, Z., Teng, S., Zhang, G., Guo, L., Zhang, W.: Catboost-based framework with additional user information for social media popularity prediction. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2677–2681 (2019) Kang, P., Lin, Z., Teng, S., Zhang, G., Guo, L., Zhang, W.: Catboost-based framework with additional user information for social media popularity prediction. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2677–2681 (2019)
3.
Zurück zum Zitat He, Z., He, Z., Wu, J., Yang, Z.: Feature construction for posts and users combined with lightgbm for social media popularity prediction. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2672–2676 (2019) He, Z., He, Z., Wu, J., Yang, Z.: Feature construction for posts and users combined with lightgbm for social media popularity prediction. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2672–2676 (2019)
4.
Zurück zum Zitat Chen, J., Liang, D., Zhu, Z., Zhou, X., Ye, Z., Mo, X.: Social media popularity prediction based on visual-textual features with xgboost. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2692–2696 (2019) Chen, J., Liang, D., Zhu, Z., Zhou, X., Ye, Z., Mo, X.: Social media popularity prediction based on visual-textual features with xgboost. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2692–2696 (2019)
5.
Zurück zum Zitat Wu, B., Mei, T., Cheng, W.-H., Zhang, Y.: Unfolding temporal dynamics: predicting social media popularity using multi-scale temporal decomposition. In: Thirtieth AAAI Conference on Artificial Intelligence (2016) Wu, B., Mei, T., Cheng, W.-H., Zhang, Y.: Unfolding temporal dynamics: predicting social media popularity using multi-scale temporal decomposition. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)
6.
Zurück zum Zitat Li, L., Situ, R., Gao, J., Yang, Z., Liu, W.: A hybrid model combining convolutional neural network with xgboost for predicting social media popularity. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1912–1917 (2017) Li, L., Situ, R., Gao, J., Yang, Z., Liu, W.: A hybrid model combining convolutional neural network with xgboost for predicting social media popularity. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1912–1917 (2017)
7.
Zurück zum Zitat Bielski, A., Trzcinski, T.: Pay attention to virality: understanding popularity of social media videos with the attention mechanism. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2335–2337 (2018) Bielski, A., Trzcinski, T.: Pay attention to virality: understanding popularity of social media videos with the attention mechanism. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2335–2337 (2018)
8.
Zurück zum Zitat Jing, P., Su, Y., Nie, L., Bai, X., Liu, J., Wang, M.: Low-rank multi-view embedding learning for micro-video popularity prediction. IEEE Trans. Knowl. Data Eng. 30(8), 1519–1532 (2017)CrossRef Jing, P., Su, Y., Nie, L., Bai, X., Liu, J., Wang, M.: Low-rank multi-view embedding learning for micro-video popularity prediction. IEEE Trans. Knowl. Data Eng. 30(8), 1519–1532 (2017)CrossRef
9.
Zurück zum Zitat He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016) He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
10.
Zurück zum Zitat Yang, J., Leskovec, J.: Patterns of temporal variation in online media. In: Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, pp. 177–186 (2011) Yang, J., Leskovec, J.: Patterns of temporal variation in online media. In: Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, pp. 177–186 (2011)
11.
Zurück zum Zitat Matsubara, Y., Sakurai, Y., Prakash, B.A., Li, L., Faloutsos, C.: Rise and fall patterns of information diffusion: model and implications. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 6–14 (2012) Matsubara, Y., Sakurai, Y., Prakash, B.A., Li, L., Faloutsos, C.: Rise and fall patterns of information diffusion: model and implications. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 6–14 (2012)
12.
Zurück zum Zitat Figueiredo, F., Almeida, J.M., Gonçalves, M.A., Benevenuto, F.: Trendlearner: early prediction of popularity trends of user generated content. Inf. Sci. 349, 172–187 (2016)CrossRef Figueiredo, F., Almeida, J.M., Gonçalves, M.A., Benevenuto, F.: Trendlearner: early prediction of popularity trends of user generated content. Inf. Sci. 349, 172–187 (2016)CrossRef
13.
Zurück zum Zitat Hu, Y., Hu, C., Fu, S., Shi, P., Ning, B.: Predicting the popularity of viral topics based on time series forecasting. Neurocomputing 210, 55–65 (2016)CrossRef Hu, Y., Hu, C., Fu, S., Shi, P., Ning, B.: Predicting the popularity of viral topics based on time series forecasting. Neurocomputing 210, 55–65 (2016)CrossRef
14.
Zurück zum Zitat Shen, H., Wang, D., Song, C., Barabási, A.-L.: Modeling and predicting popularity dynamics via reinforced Poisson processes. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 28 (2014) Shen, H., Wang, D., Song, C., Barabási, A.-L.: Modeling and predicting popularity dynamics via reinforced Poisson processes. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 28 (2014)
15.
Zurück zum Zitat Gao, S., Ma, J., Chen, Z.: Modeling and predicting retweeting dynamics on microblogging platforms. In: Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, pp. 107–116 (2015) Gao, S., Ma, J., Chen, Z.: Modeling and predicting retweeting dynamics on microblogging platforms. In: Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, pp. 107–116 (2015)
16.
Zurück zum Zitat Bao, P., Shen, H.-W., Jin, X., Cheng, X.-Q.: Modeling and predicting popularity dynamics of microblogs using self-excited Hawkes processes. In: Proceedings of the 24th International Conference on World Wide Web, pp. 9–10 (2015) Bao, P., Shen, H.-W., Jin, X., Cheng, X.-Q.: Modeling and predicting popularity dynamics of microblogs using self-excited Hawkes processes. In: Proceedings of the 24th International Conference on World Wide Web, pp. 9–10 (2015)
18.
Zurück zum Zitat Maki, D.P., Maki, D.P., Mali, D., Thompson, M., Thompson, M.: Mathematical Models and Applications: with Emphasis on the Social, Life, and Management Sciences. Prentice Hall (1973) Maki, D.P., Maki, D.P., Mali, D., Thompson, M., Thompson, M.: Mathematical Models and Applications: with Emphasis on the Social, Life, and Management Sciences. Prentice Hall (1973)
19.
Zurück zum Zitat Xiong, F., Liu, Y., Zhang, Z.-J., Zhu, J., Zhang, Y.: An information diffusion model based on retweeting mechanism for online social media. Phys. Lett. A 376(30–31), 2103–2108 (2012)CrossRef Xiong, F., Liu, Y., Zhang, Z.-J., Zhu, J., Zhang, Y.: An information diffusion model based on retweeting mechanism for online social media. Phys. Lett. A 376(30–31), 2103–2108 (2012)CrossRef
20.
Zurück zum Zitat Wang, H., Li, Y., Feng, Z., Feng, L.: Retweeting analysis and prediction in microblogs: an epidemic inspired approach. China Commun. 10(3), 13–24 (2013)CrossRef Wang, H., Li, Y., Feng, Z., Feng, L.: Retweeting analysis and prediction in microblogs: an epidemic inspired approach. China Commun. 10(3), 13–24 (2013)CrossRef
21.
Zurück zum Zitat Yang, J., Leskovec, J.: Modeling information diffusion in implicit networks. In: 2010 IEEE International Conference on Data Mining, pp. 599–608. IEEE (2010) Yang, J., Leskovec, J.: Modeling information diffusion in implicit networks. In: 2010 IEEE International Conference on Data Mining, pp. 599–608. IEEE (2010)
22.
Zurück zum Zitat Vilares, D., Alonso, M.A., Gómez-Rodríguez, C.: On the usefulness of lexical and syntactic processing in polarity classification of twitter messages. J. Am. Soc. Inf. Sci. 66(9), 1799–1816 (2015) Vilares, D., Alonso, M.A., Gómez-Rodríguez, C.: On the usefulness of lexical and syntactic processing in polarity classification of twitter messages. J. Am. Soc. Inf. Sci. 66(9), 1799–1816 (2015)
23.
Zurück zum Zitat Wu, B., Cheng, W.-H., Zhang, Y., Mei, T.: Time matters: Multi-scale temporalization of social media popularity. In: Proceedings of the 24th ACM International Conference on Multimedia, pp. 1336–1344 (2016) Wu, B., Cheng, W.-H., Zhang, Y., Mei, T.: Time matters: Multi-scale temporalization of social media popularity. In: Proceedings of the 24th ACM International Conference on Multimedia, pp. 1336–1344 (2016)
24.
Zurück zum Zitat Wu, B., Cheng, W.-H., Zhang, Y., Huang, Q., Li, J., Mei, T.: Sequential prediction of social media popularity with deep temporal context networks. arXiv preprint arXiv:1712.04443 (2017) Wu, B., Cheng, W.-H., Zhang, Y., Huang, Q., Li, J., Mei, T.: Sequential prediction of social media popularity with deep temporal context networks. arXiv preprint arXiv:​1712.​04443 (2017)
25.
Zurück zum Zitat Zhang, W., Wang, W., Wang, J., Zha, H.: User-guided hierarchical attention network for multi-modal social image popularity prediction. In: Proceedings of the 2018 World Wide Web Conference, pp. 1277–1286 (2018) Zhang, W., Wang, W., Wang, J., Zha, H.: User-guided hierarchical attention network for multi-modal social image popularity prediction. In: Proceedings of the 2018 World Wide Web Conference, pp. 1277–1286 (2018)
26.
Zurück zum Zitat Zhang, D., Yao, L., Chen, K., Wang, S., Chang, X., Liu, Y.: Making sense of spatio-temporal preserving representations for EEG-based human intention recognition. IEEE Trans. Cybern. 50(7), 3033–3044 (2019)CrossRef Zhang, D., Yao, L., Chen, K., Wang, S., Chang, X., Liu, Y.: Making sense of spatio-temporal preserving representations for EEG-based human intention recognition. IEEE Trans. Cybern. 50(7), 3033–3044 (2019)CrossRef
27.
Zurück zum Zitat Luo, M., Chang, X., Nie, L., Yang, Y., Hauptmann, A.G., Zheng, Q.: An adaptive semisupervised feature analysis for video semantic recognition. IEEE Trans. Cybern. 48(2), 648–660 (2017)CrossRef Luo, M., Chang, X., Nie, L., Yang, Y., Hauptmann, A.G., Zheng, Q.: An adaptive semisupervised feature analysis for video semantic recognition. IEEE Trans. Cybern. 48(2), 648–660 (2017)CrossRef
28.
Zurück zum Zitat Chen, K., Yao, L., Zhang, D., Wang, X., Chang, X., Nie, F.: A semisupervised recurrent convolutional attention model for human activity recognition. IEEE Trans. Neural Netw. Learn. Syst. 31(5), 1747–1756 (2019)CrossRef Chen, K., Yao, L., Zhang, D., Wang, X., Chang, X., Nie, F.: A semisupervised recurrent convolutional attention model for human activity recognition. IEEE Trans. Neural Netw. Learn. Syst. 31(5), 1747–1756 (2019)CrossRef
29.
Zurück zum Zitat Liu, Z., Wu, S., Jin, S., Liu, Q., Ji, S., Lu, S., Cheng, L.: Investigating pose representations and motion contexts modeling for 3D motion prediction. IEEE Trans. Pattern Anal. Mach. Intell. 45(1), 681–97 (2022)CrossRef Liu, Z., Wu, S., Jin, S., Liu, Q., Ji, S., Lu, S., Cheng, L.: Investigating pose representations and motion contexts modeling for 3D motion prediction. IEEE Trans. Pattern Anal. Mach. Intell. 45(1), 681–97 (2022)CrossRef
30.
Zurück zum Zitat Liu, A.-A., Zhou, H., Nie, W., Liu, Z., Liu, W., Xie, H., Mao, Z., Li, X., Song, D.: Hierarchical multi-view context modelling for 3D object classification and retrieval. Inf. Sci. 547, 984–995 (2021)CrossRef Liu, A.-A., Zhou, H., Nie, W., Liu, Z., Liu, W., Xie, H., Mao, Z., Li, X., Song, D.: Hierarchical multi-view context modelling for 3D object classification and retrieval. Inf. Sci. 547, 984–995 (2021)CrossRef
31.
Zurück zum Zitat Rennie, S.J., Marcheret, E., Mroueh, Y., Ross, J., Goel, V.: Self-critical sequence training for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7008–7024 (2017) Rennie, S.J., Marcheret, E., Mroueh, Y., Ross, J., Goel, V.: Self-critical sequence training for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7008–7024 (2017)
32.
Zurück zum Zitat Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: a neural image caption generator. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156–3164 (2015) Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: a neural image caption generator. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156–3164 (2015)
33.
Zurück zum Zitat Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3128–3137 (2015) Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3128–3137 (2015)
34.
Zurück zum Zitat Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. Adv. Neural Inf. Process. Syst. 31 (2018) Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: Catboost: unbiased boosting with categorical features. Adv. Neural Inf. Process. Syst. 31 (2018)
36.
Zurück zum Zitat Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Process. Syst. 26 (2013) Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Process. Syst. 26 (2013)
37.
Zurück zum Zitat Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013) Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:​1301.​3781 (2013)
38.
Zurück zum Zitat Wu, B., Cheng, W.-H., Liu, P., Liu, B., Zeng, Z., Luo, J.: Smp challenge: an overview of social media prediction challenge 2019. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2667–2671 (2019) Wu, B., Cheng, W.-H., Liu, P., Liu, B., Zeng, Z., Luo, J.: Smp challenge: an overview of social media prediction challenge 2019. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2667–2671 (2019)
39.
Zurück zum Zitat Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: common objects in context. In: European Conference on Computer Vision, pp. 740–755. Springer (2014) Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: common objects in context. In: European Conference on Computer Vision, pp. 740–755. Springer (2014)
41.
Zurück zum Zitat Ding, K., Wang, R., Wang, S.: Social media popularity prediction: a multiple feature fusion approach with deep neural networks. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2682–2686 (2019) Ding, K., Wang, R., Wang, S.: Social media popularity prediction: a multiple feature fusion approach with deep neural networks. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 2682–2686 (2019)
42.
Zurück zum Zitat Xu, K., Lin, Z., Zhao, J., Shi, P., Deng, W., Wang, H.: Multimodal deep learning for social media popularity prediction with attention mechanism. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 4580–4584 (2020) Xu, K., Lin, Z., Zhao, J., Shi, P., Deng, W., Wang, H.: Multimodal deep learning for social media popularity prediction with attention mechanism. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 4580–4584 (2020)
43.
Zurück zum Zitat Wang, K., Wang, P., Chen, X., Huang, Q., Mao, Z., Zhang, Y.: A feature generalization framework for social media popularity prediction. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 4570–4574 (2020) Wang, K., Wang, P., Chen, X., Huang, Q., Mao, Z., Zhang, Y.: A feature generalization framework for social media popularity prediction. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 4570–4574 (2020)
Metadaten
Titel
SMPC: boosting social media popularity prediction with caption
verfasst von
An-An Liu
Xiaowen Wang
Ning Xu
Jing Liu
Yuting Su
Quan Zhang
Shenyuan Zhang
Yejun Tang
Junbo Guo
Guoqing Jin
Xuanya Li
Publikationsdatum
19.01.2023
Verlag
Springer Berlin Heidelberg
Erschienen in
Multimedia Systems / Ausgabe 2/2023
Print ISSN: 0942-4962
Elektronische ISSN: 1432-1882
DOI
https://doi.org/10.1007/s00530-022-01030-5

Weitere Artikel der Ausgabe 2/2023

Multimedia Systems 2/2023 Zur Ausgabe